ChangeLog

Version 0.4.0.1 (January 9, 2012)

• Reports
  • XML -- added missing Issue attributes
• Added draconian run-time exception handling to all components.

Version 0.4 (January 7, 2012)

• RPC Infrastructure (New)
  • Dispatcher
    • Dispatchers can now be connected to form a High Performance Grid and share scan workloads.
    • Users can now specify a range of ports to be used for spawned Instances. [Issue #76]
    • Now checks for signal availability before using trap(). (New) [Issue #71]
    • Now uses Windows compliant filenames for the logs. (New) [Issue #70]
  • Ruby's XMLRPC libraries have been replaced by Arachni-RPC, a light-weight and high-performance custom client/server RPC implementation.
• Added extras directory holding components that are considered too specialised, dangerous or in some way unsuitable for utilising without explicit user interaction. (New)
  • Modules
    • Recon
      • SVN Digger dirs -- Finds directories, based on wordlists created from open source repositories (Herman Stevens)
      • SVN Digger files -- Finds files, based on wordlists created from open source repositories (Herman Stevens)
      • RAFT dirs (Herman Stevens)
      • RAFT files (Herman Stevens)
• Framework
  • stats()
    • Fixed bug that caused the current_page to not be refreshed during timing attacks.
    • Fixed bug that caused a less than 100% progress at the end of scans. [Issue #86]
    • If the crawler is limited by link-count it will be taken under consideration.
  • Significantly reduced memory footprint by re-scheduling the consumption of Trainer generated pages.
• User Interfaces
  • WebUI
    • Sinatra
      • Updated to use the light-weight and high-performance Thin server.
      • Added async_sinatra to allow for asynchronous responses. (New)
    • Added support for HTTP Basic Auth (New)
    • Updated screens to provide access to HPG (High Performance Grid) features:
      • Home
        • Added option to enable HPG mode on a per scan basis (New)
      • Dispatchers
        • Added node information (Nickname, Pipe ID, Weight, Cost). (New)
        • Added neighbour inspection per dispatcher. (New)
        • Added log inspection per dispatcher. (New)
        • Improved accuracy of instance statuses.
        • Added percentages for memory and CPU usage per instance. (New)
      • Instance (scan management)
        • Provides an average of all stats of scanner instances. (New)
        • Added per instance progress bars. (New)
        • Added per instance statuses. (New)
        • Added est. remaining time. (New)
      • Settings
        • Added proxy settings. [Issue #74] (New)
        • Added settings for restrict and extend paths options. (New)
    • Fixed small typo in "Settings" screen. [Issue #62]
    • Reports -- AFR report is now served straight-up to avoid corruption. [Issue #55]
    • Add-ons -- Updated to use the new async libraries.
    • Added help buttons. (New)
  • CLI
    • Improved interrupt handler:
      • It now exits in a cleaner fashion and is more obedient.
      • Added est. remaining time. (New)
      • Added progressbar. (New)
• HTTP client
  • Added support for including custom headers. [Issue #90] (New)
  • Refactored in order for all methods to use request().
  • Bug-fixed cookie preservation.
• Spider
  • spider-first option removed and set to true by default.
  • Added "--depth" parameter. (New)
  • Fixed incorrect implementation of the inclusion filters.
  • Now follows "Location" headers directly and bypasses the trainer.
  • Added support for extending the crawl scope with a file that contains newline separated URLs. (New) [Issue #67]
  • Added support for restricting the crawl scope with a file that contains newline separated URLs. (New)
  • Made more resilient against malformed/non-standard URLs. [Issue #57]
• Parser
  • Encoded URLs with fragments right after the host caused URI.parse to fail. [Issue #66]
  • Auditable elements
    • If there are 2 or more password fields in a form an extra variation is added with the same inputs for all passwords in case it's a 'please repeat your password' thing. (New) [Issue #59]
• Plugins
  • API -- Added distributable?() and merge() class methods which declare if a plug-in can be distributed to all instances when running in Grid mode and merge an array of its own results respectively.
    • Distributable plug-ins:
      • Content-Types
      • Cookie collector
      • Healthmap
      • Profiler
      • AutoThrottle
  • Profiler -- Removed response time logging and moved it to defaults.
  • Proxy -- Fixed bug which caused some headers not to be forwarded. [Issue #64]
  • Discovery (accompanied by appropriate report formatters). (New) [Issue #81]
    • Performs anomaly detection on issues logged by discovery modules and warns of the possibility of false positives where applicable.
  • Added the 'defaults' subdirectory which contains plug-ins that should be loaded by default.
  • Added: (New)
    • ReScan -- It uses the AFR report of a previous scan to extract the sitemap in order to avoid a redundant crawl.
    • BeepNotify -- Beeps when the scan finishes.
    • LibNotify -- Uses the libnotify library to send notifications for each discovered issue and a summary at the end of the scan.
    • EmailNotify -- Sends a notification (and optionally a report) over SMTP at the end of the scan.
    • Manual verification -- Flags issues that require manual verification as untrusted in order to reduce the signal-to-noise ratio.
    • Resolver -- Resolves vulnerable hostnames to IP addresses.
• Reports
  • HTML report
    • Fixed replay forms to include URL params in the action attribute. [Issue #73]
    • Refactored and broken into erb partials.
    • Organised subsections into tabs. (New)
    • HTML responses of logged Issues are now rendered on-demand. [Issue #88]
    • Added graph showing issue trust totals. (New)
    • The main issue graph shows trusted and untrusted issues in 2 different series.
    • ALl JavaScript and CSS code is now included in the report for off-line viewing.
    • Removed manual-verification piechart, obsoleted by the trust chart.
    • Replaced Highcharts with jqPlot due to licensing reasons.
    • Removed false-positive reporting -- was causing segfaults on Mac OSX. [Issue #126]
  • Added (New)
    • JSON -- Exports the audit results as a JSON serialized Hash.
    • Marshal -- Exports the audit results as a Marshal serialized Hash.
    • YAML -- Exports the audit results as a YAML serialized Hash.
• Heeded Ruby's warnings (ruby -w).
• Modules
  • API
    • Auditor
      • Added helper methods for checking the existence of remote files and directories. (New)
      • Added helper methods for issue logging. (New)
  • Refactored modules replacing duplicate code with the new helper methods.
  • Audit
    • XSS -- Updated to actually inject an element, parse the HTML response and look for that element before logging in order to eliminate false positives. [Issue #59]
    • Path traversal -- Fixed broken regular expressions
    • SQL Injection -- Fixed broken regular expressions
    • XSS Path -- Updated to verify the injection using HTML parsing
    • XSS URI -- Made obsolete and will be removed from future releases -- loads and runs XSS Path instead.
  • Recon
    • Added MixedResource detection module (Reference) (New) [Issue #56]
• Meta-Modules
  • Have all been converted to regular plug-ins in order to make distribution across the Grid easier.
• Dependencies
  • Added
    • Arachni-RPC
    • EventMachine
    • EM Synchrony
    • AsyncSinatra
  • Updated
    • Typhoeus => 0.3.3
    • Sys-proctable => 0.9.1
    • Nokogiri => 1.5.0
    • Sinatra => 1.3.1
    • Datamapper => 1.1.0
    • Json => 1.6.1
    • Datamapper SQLite adapter => 1.1.0
    • Net-SSH => 2.2.1
  • Removed
    • Rack-CSRF
    • JSON (Provided by DataMapper)

Version 0.3 (July 26, 2011)

• HTTP client
  • Fixed race condition in timeout options.
• Spider (New)
  • Replaced Anemone with a lightweight custom-written spider.
• WebUI
  • Major refactoring.
  • Improved handling of connection errors during scan progress updates.
  • Added support for add-ons. (New)
  • Add-ons (New)
    • Scan scheduler
    • Auto-deploy -- Automatically converts any SSH enabled Linux box into an Arachni Dispatcher.
  • Fixed bug when IP addresses are used, instead of hostnames, for the Dispatchers.
• Parser
  • Form action attributes are now sanitized using iterative URI decoding.
  • Link variables are extracted before URL sanitization takes place in order to keep values with URL-encoded characters intact.
  • The link variables of any current page's URL are now pushed to 'page.links'.
• Auditor
  • Abstracted the rDiff audit methods from the "Blind (rDiff) SQL Injection" module and moved them in the Auditor.
  • Timing attack technique has been greatly improved and all timing attacks are now scheduled to run at the end of the scan.
• Modules
  • API
    • Added the "redundant()" method -- Allows a module to prevents itself from auditting elements that have been previously logged by other modules.
    • Modules are now passed an instance of the framework.
  • Audit
    • Blind (rDiff) SQL Injection
      • Updated to support all element types (Links, Forms, Cookies, Headers).
      • Optimized using the new "redundant()" method -- It will no longer audit elements that have been previously logged by the 'sqli' or 'sqli_blind_rdiff' modules.
    • OS command injection (timing)
      • Optimized using the new "redundant()" method -- It will no longer audit elements that have been previously logged by the 'os_cmd_injection' module.
    • Code injection (timing)
      • Optimized using the new "redundant()" method -- It will no longer audit elements that have been previously logged by the 'code_injection' module.

Version 0.2.4 (July 1, 2011)

• HTTP
  • Implemented a 10s time-out [Issue #40]
• Command Line Interface
  • The interrupt handler (Ctrl+C) now presents the option to generate reports mid-scan. [Issue #41]
  • Added a counter of timed-out requests in the stats.
• WebUI
  • The "Replay" form's action attribute now contains the full URL, including params. [Issue #38]
  • Fixed path clash that caused the "shutdown" button in the Dispatchers screen not to work. [Issue #39]
  • Fixed mix-up of output messages from different instances. [Issue #36]
  • Added a counter of timed-out requests in "Instance" screens.
• External
  • Metasploit
    • Updated SQL injection exploit module to work with SQLmap 0.9. [Issue #37]
• Reports
  • HTML
    • Fixed yet another error condition occuring with broken encodings. [Issue #31]
• Auditor
  • Timing attacks now have a "control" to verify that the server is indeed alive i.e. requests won't time-out by default.

Version 0.2.3 (May 22, 2011)

• WebUI
  • Added connection cache for XMLRPC server instances to remove HTTPS handshake overhead and take advantage of keep-alive support.
  • Added initial support for management of multiple Dispatchers.
• XMLRPC Client->Dispatch Server
  • Updated to always use SSL [Issue #28]
  • Added per instance authentication tokens [Issue #28]
• Modules
  • Audit
    • Path traversal: added double encoded traversals [Issue #29]
• Reports
  • HTML
    • Fixed "invalid byte sequence in UTF-8" using iconv [Issue #27]
    • Added false positive reporting. Data are encrypted using 256bit AES (with AES primitives encrypted using RSA) and sent over HTTPS. [Issue #30]
  • Metareport
    • Fixed bug caused by not explicitly closed file handle.

Version 0.2.2.2 (March 22, 2011)

• Added "arachni_web_autostart" under bin -- Automatically starts all systems required by the WebUI and makes shutting down everything easier too (Original by: Brandon Potter bpotter8705@gmail.com)
• Overrided Nokogiri to revert to UTF-8 when it comes across an unknown charset instead of throwing exceptions
• Dependency versions are now defined explicitly [Issue #23]
• Updated to Sinatra v1.2.1
• HTTP
  • Disabled peer verification on SSL [Issue #19]
  • Replaced callbacks with the new Observable mixin (also updated components to use the new conventions)
• WebUI
  • Plug-in options are preserved [Issue #19]
  • Check-all now skips disabled checkboxes
  • Report info is stored in a database [Issue #19]
  • Reports are now displayed in descending order based on scan completion datetime [Issue #19]
  • Any existing reports will be migrated into the new database [Issue #19]
• XMLRPC service
  • Fixed segfault on forced shutdown when spider-first was enabled
• Plug-ins
  • AutoLogin now registers its results
• Reports -- Added formatters for the AutoLogin [Issue #19] and Profiler plug-ins
  • HMTL
    • Fixed exception on empty issue list
    • Fixed encoding exceptions (cheers to Chris Weber chris@casaba.com)
• Path extractors
  • Generic -- fixed error on invalid encoding sequences
• Modules
  • Recon
    • Directory listing -- Now skips non-200 pages because it used to log false positives on redirections
• Plug-ins
  • Added Profiler -- Performs taint analysis (with benign inputs) and response time analysis

Version 0.2.2.1 (February 13, 2011)

• Web UI v0.1-pre (Utilizing the Client - Dispatch-server XMLRPC architecture) (New)
  • Basically a front-end to the XMLRPC client
  • Support for parallel scans
  • Report management
  • Can be used to monitor and control any running Dispatcher
• Changed classification from "Vulnerabilities" to "Issues" (New)
• Improved detection of custom 404 pages.
• Reports updated to show plug-in results.
• Updated framework-wide cookie handling.
• Added parameter flipping functionality ( cheers to Nilesh Bhosale )
• Major performance optimizations (4x faster in most tests)
  • All modules now use asynchronous requests and are optimized for highest traffic efficiency
  • All index Arrays have been replaced by Sets to minimize look-up times
  • Mark-up parsing has been reduced dramatically
  • File I/O blocking in modules has been eliminated
• Crawler
  • Improved performance
  • Added '--spider-first" option (New)
• Substituted the XMLRPC server with an XMLRPC dispatch server (New)
  • Multiple clients
  • Parallel scans
  • Extensive logging
  • SSL cert based client authentication
• Added modules (New)
  • Audit
    • XSS in event attributes of HTML elements
    • XSS in HTML tags
    • XSS in HTML 'script' tags
    • Blind SQL injection using timing attacks
    • Blind code injection using timing attacks (PHP, Ruby, Python, JSP, ASP.NET)
    • Blind OS command injection using timing attacks (*nix, Windows)
  • Recon
    • Common backdoors -- Looks for common shell names
    • .htaccess LIMIT misconfiguration
    • Interesting responses -- Listens to all traffic and logs interesting server messages
    • HTML object grepper
    • E-mail address disclosure
    • US Social Security Number disclosure
    • Forceful directory listing

• Added plugins (New)

  • Dictionary attacker for HTTP Auth
  • Dictionary attacker for form based authentication
  • Cookie collector -- Listens to all traffic and logs changes in cookies
  • Healthmap -- Generates sitemap showing the health of each crawled/audited URL
  • Content-types -- Logs content-types of server responses aiding in the identification of interesting (possibly leaked) files
  • WAF (Web Application Firewall) Detector
  • MetaModules -- Loads and runs high-level meta-analysis modules pre/mid/post-scan
    • AutoThrottle -- Dynamically adjusts HTTP throughput during the scan for maximum bandwidth utilization
    • TimeoutNotice -- Provides a notice for issues uncovered by timing attacks when the affected audited pages returned unusually high response times to begin with.
      It also points out the danger of DoS attacks against pages that perform heavy-duty processing.
    • Uniformity -- Reports inputs that are uniformly vulnerable across a number of pages hinting to the lack of a central point of input sanitization.

• New behavior on Ctrl+C

  • The system continues to run in the background instead of pausing
  • The user is presented with an auto-refreshing report and progress stats
• Updated module API
  • Timing/delay attacks have been abstracted and simplified via helper methods
  • The modules are given access to vector skipping decisions
  • Simplified issue logging
  • Added the option of substring matching instead of regexp matching in order to improve performance.
  • Substituted regular expression matching with substring matching wherever possible.
• Reports:
  • Added plug-in formatter components allowing plug-ins to have a say in how their results are presented (New)
  • New HTML report (Cheers to Christos Chiotis for designing the new HTML report template.) (New)
  • Updated reports to include Plug-in results:
    • XML report
    • Stdout report
    • Text report

Version 0.2.1 (November 25, 2010)

• Major performance improvements
• Major system refactoring and code clean-up
• Major module API refactoring providing even more flexibility regarding element auditing and manipulation
• Integration with the Metasploit Framework via: (New)
  • ArachniMetareport, an Arachni report specifically designed to provide WebApp context to the Metasploit framework.
  • Arachni plug-in for the Metasploit framework, used to load the ArachniMetareport in order to provide advanced automated and manual exploitation of WebApp vulnerabilities.
  • Advanced generic WebApp exploit modules for the Metasploit framework, utilized either manually or automatically by the Arachni MSF plug-in.
• Improved Blind SQL Injection module, significantly less requests per audit.
• XMLRPC server (New)
• XMLRPC CLI client (New)
• NTLM authentication support (New)
• Support for path extractor modules for the Spider (New)
• Path extractors: (New)
  • Generic -- extracts URLs from arbitrary text
  • Anchors
  • Form actions
  • Frame sources
  • Links
  • META refresh
  • Script 'src' and script code
  • Sitemap
• Plug-in support -- allowing the framework to be extended with virtually any functionality (New).
• Added plug-ins: (New)
  • Passive proxy
  • Automated login
• Added modules: (New)
  • Audit
    • XPath injection
    • LDAP injection
  • Recon
    • CVS/SVN user disclosure
    • Private IP address disclosure
    • Robot file reader (in the Common Files module)
    • XST
    • WebDAV detection
    • Allowed HTTP methods
    • Credit card number disclosure
    • HTTP PUT support
• Extended proxy support (SOCKS4, SOCKS4A, SOCKS5, HTTP/1.1 and HTTP/1.0). (New)

Version 0.2 (October 14, 2010)

• Improved output.
  • Increased context awareness.
  • Extensive debugging output capabilities.
  • Added simple stats at the end of scans.
• Rewritten HTTP interface.
  • High-performance asynchronous HTTP requests.
  • Adjustable HTTP request concurrency limit.
  • Adjustable HTTP response harvests.
  • Custom 404 page detection.
• Optimized Trainer subsystem.
  • Invoked when it is most likely to detect new vectors.
  • Can be invoked by individual modules on-demand, forcing Arachni to learn from the HTTP responses they will cause -- a great asset to Fuzzers.
• Refactored and improved Auditor.
  • No redundant requests, except when required by modules.
  • Better parameter handling.
  • Speed optimizations.
  • Added differential analysis to determine whether a vulnerability needs manual verification.
• Refactored and improved module API.
  • Major API clean up.
  • With facilities providing more control and power over the audit process.
  • Significantly increased ease of development.
  • Modules have total flexibility and control over input combinations, injection values and their formating -- if they need to.
  • Modules can opt for sync or async HTTP requests (Default: async)
• Improved interrupt handling
  • Scans can be paused/resumed at any time.
  • In the event of a system exit or user cancellation reports will still be created using whatever data were gathered during runtime.
  • When the scan is paused the user will be presented with the results gathered thus far.
• Improved configuration profile handling
  • Added pre-configured profiles
  • Multiple profiles can be loaded at once
  • Ability to show running profiles as CLI arguments
• Overall module improvements and optimizations.
• New modules for:
  • Blind SQL Injection, using reverse-diff analysis.
  • Trainer, probes all inputs of a given page, in order to uncover new input vectors, and forces Arachni to learn from the responses.
  • Unvalidated redirects.
  • Forms that transmit passwords in clear text.
  • CSRF, implementing 4-pass rDiff analysis to drastically reduce noise.
• Overall report improvements and optimizations.
• New reports
  • Plain text report
  • XML report