Arachni The Web App Security Scanner Framework Development Has Stopped

Arachni is a feature-full, modular, high-performance Ruby framework aimed towards helping penetration testers and administrators evaluate the security of web applications.

It is smart, it trains itself by monitoring and learning from the web application's behavior during the scan process and is able to perform meta-analysis using a number of factors in order to correctly assess the trustworthiness of results and intelligently identify (or avoid) false-positives.

Unlike other scanners, it takes into account the dynamic nature of web applications, can detect changes caused while travelling through the paths of a web application’s cyclomatic complexity and is able to adjust itself accordingly. This way, attack/input vectors that would otherwise be undetectable by non-humans can be handled seamlessly.

Moreover, due to its integrated browser environment, it can also audit and inspect client-side code, as well as support highly complicated web applications which make heavy use of technologies such as JavaScript, HTML5, DOM manipulation and AJAX.

Features

General

• Cookie-jar/cookie-string support.
• Custom header support.
• SSL support with fine-grained options.
• User Agent spoofing.
• Proxy support for SOCKS4, SOCKS4A, SOCKS5, HTTP/1.1 and HTTP/1.0.
• Proxy authentication.
• Site authentication (SSL-based, form-based, Cookie-Jar, Basic-Digest, NTLMv1, Kerberos and others).
• Automatic log-out detection and re-login during the scan (when the initial login was performed via the autologin, login_script or proxy plugins).
• Custom 404 page detection.
• UI abstraction:
  • Command-line Interface.
  • Web User Interface.
• Pause/resume functionality.
• Hibernation support -- Suspend to and restore from disk.
• High performance asynchronous HTTP requests.
  • With adjustable concurrency.
  • With the ability to auto-detect server health and adjust its concurrency automatically.
• Support for custom default input values, using pairs of patterns (to be matched against input names) and values to be used to fill in matching inputs.

Integrated browser environment

Arachni includes an integrated, real browser environment in order to provide sufficient coverage to modern web applications which make use of technologies such as HTML5, JavaScript, DOM manipulation, AJAX, etc.

In addition to the monitoring of the vanilla DOM and JavaScript environments, Arachni's browsers also hook into popular frameworks to make the logged data easier to digest:

• JQuery
• AngularJS
• More to come...

In essence, this turns Arachni into a DOM and JavaScript debugger, allowing it to monitor DOM events and JavaScript data and execution flows. As a result, not only can the system trigger and identify DOM-based issues, but it will accompany them with a great deal of information regarding the state of the page at the time.

Relevant information include:

• Page DOM, as HTML code.
  • With a list of DOM transitions required to restore the state of the page to the one at the time it was logged.
• Original DOM (i.e. prior to the action that caused the page to be logged), as HTML code.
  • With a list of DOM transitions.
• Data-flow sinks -- Each sink is a JS method which received a tainted argument.
  • Parent object of the method (ex.: DOMWindow).
  • Method signature (ex.: decodeURIComponent()).
  • Arguments list.
    • With the identified taint located recursively in the included objects.
  • Method source code.
  • JS stacktrace.
• Execution flow sinks -- Each sink is a successfully executed JS payload, as injected by the security checks.
  • Includes a JS stacktrace.
• JavaScript stack-traces include:
  • Method names.
  • Method locations.
  • Method source codes.
  • Argument lists.

In essence, you have access to roughly the same information that your favorite debugger (for example, FireBug) would provide, as if you had set a breakpoint to take place at the right time for identifying an issue.

Browser-cluster

The browser-cluster is what coordinates the browser analysis of resources and allows the system to perform operations which would normally be quite time consuming in a high-performance fashion.

Configuration options include:

• Adjustable pool-size, i.e. the amount of browser workers to utilize.
• Timeout for each job.
• Worker TTL counted in jobs -- Workers which exceed the TTL have their browser process respawned.
• Ability to disable loading images.
• Adjustable screen width and height.
  • Can be used to analyze responsive and mobile applications.
• Ability to wait until certain elements appear in the page.
• Configurable local storage data.

Coverage

The system can provide great coverage to modern web applications due to its integrated browser environment. This allows it to interact with complex applications that make heavy use of client-side code (like JavaScript) just like a human would.

In addition to that, it also knows about which browser state changes the application has been programmed to handle and is able to trigger them programatically in order to provide coverage for a full set of possible scenarios.

By inspecting all possible pages and their states (when using client-side code) Arachni is able to extract and audit the following elements and their inputs:

• Forms
  • Along with ones that require interaction via a real browser due to DOM events.
• User-interface Forms
  • Input and button groups which don't belong to an HTML <form> element but are instead associated via JS code.
• User-interface Inputs
  • Orphan <input> elements with associated DOM events.
• Links
  • Along with ones that have client-side parameters in their fragment, i.e.: http://example.com/#/?param=val&param2=val2
  • With support for rewrite rules.
• LinkTemplates -- Allowing for extraction of arbitrary inputs from generic paths, based on user-supplied templates -- useful when rewrite rules are not available.
  • Along with ones that have client-side parameters in their URL fragments, i.e.: http://example.com/#/param/val/param2/val2
• Cookies
  • Also supports nested cookies, containing key-value pairs inside individual cookies.
• Headers
• Generic client-side elements which have associated DOM events.
• AJAX-request parameters.
• JSON request data.
• XML request data.

Open distributed architecture

Arachni is designed to fit into your workflow and easily integrate with your existing infrastructure.

Depending on the level of control you require over the process, you can either choose the REST service or the custom RPC protocol.

Both approaches allow you to:

• Remotely monitor and manage scans.
• Perform multiple scans at the same time -- Each scan is compartmentalized to its own OS process to take advantage of:
  • Multi-core/SMP architectures.
  • OS-level scheduling/restrictions.
  • Sandboxed failure propagation.
• Communicate over a secure channel.

REST API

• Very simple and straightforward API.
• Easy interoperability with non-Ruby systems.
  • Operates over HTTP.
  • Uses JSON to format messages.
• Stateful scan monitoring.
  • Unique sessions automatically only receive updates when polling for progress, rather than full data.

RPC API

• High-performance/low-bandwidth communication protocol.
  • MessagePack serialization for performance, efficiency and ease of integration with 3rd party systems.
• Grid:
  • Self-healing.
  • Scale up/down by hot-plugging/hot-unplugging nodes.
    • Can scale up infinitely by adding nodes to increase scan capacity.
  • (Always-on) Load-balancing -- All Instances are automatically provided by the least burdened Grid member.
    • With optional per-scan opt-out/override.
  • (Optional) High-Performance mode -- Combines the resources of multiple nodes to perform multi-Instance scans.
    • Enabled on a per-scan basis.

Scope configuration

• Filters for redundant pages like galleries, catalogs, etc. based on regular expressions and counters.
  • Can optionally detect and ignore redundant pages automatically.
• URL exclusion filters using regular expressions.
• Page exclusion filters based on content, using regular expressions.
• URL inclusion filters using regular expressions.
• Can be forced to only follow HTTPS paths and not downgrade to HTTP.
• Can optionally follow subdomains.
• Adjustable page count limit.
• Adjustable redirect limit.
• Adjustable directory depth limit.
• Adjustable DOM depth limit.
• Adjustment using URL-rewrite rules.
• Can read paths from multiple user supplied files (to both restrict and extend the scope).

Audit

• Can audit:
  • Forms
    • Can automatically refresh nonce tokens.
    • Can submit them via the integrated browser environment.
  • User-interface Forms
    • Input and button groups which don't belong to an HTML <form> element but are instead associated via JS code.
  • User-interface Inputs
    • Orphan <input> elements with associated DOM events.
  • Links
    • Can load them via the integrated browser environment.
  • LinkTemplates
    • Can load them via the integrated browser environment.
  • Cookies
    • Can load them via the integrated browser environment.
  • Headers
  • Generic client-side DOM elements.
  • JSON request data.
  • XML request data.
• Can ignore binary/non-text pages.
• Can audit elements using both GET and POST HTTP methods.
• Can inject both raw and HTTP encoded payloads.
• Can submit all links and forms of the page along with the cookie permutations to provide extensive cookie-audit coverage.
• Can exclude specific input vectors by name.
• Can include specific input vectors by name.

SQL Injection

Active checks engage the web application via its inputs.

• SQL injection (sql_injection) -- Error based detection.
  • Oracle
  • InterBase
  • PostgreSQL
  • MySQL
  • MSSQL
  • EMC
  • SQLite
  • DB2
  • Informix
  • Firebird
  • SaP Max DB
  • Sybase
  • Frontbase
  • Ingres
  • HSQLDB
  • MS Access

Components

Arachni is a highly modular system, employing several components of distinct types to perform its duties.

In addition to enabling or disabling the bundled components so as to adjust the system's behavior and features as needed, functionality can be extended via the addition of user-created components to suit almost every need.

Supported Platforms, Web Servers, Programming Languages and Frameworks

Currently, the following platforms can be identified:

• Operating systems
  • BSD
  • Linux
  • Unix
  • Windows
  • Solaris
• Web servers
  • Apache
  • IIS
  • Nginx
  • Tomcat
  • Jetty
  • Gunicorn
• Programming languages
  • PHP
  • ASP
  • ASPX
  • Java
  • Python
  • Ruby
• Frameworks
  • Rack
  • CakePHP
  • Rails
  • Django
  • ASP.NET MVC
  • JSF
  • CherryPy
  • Nette
  • Symfony

Development Status

However, The developers recommended using other scanners as the project did not receive any update for the past few years.

License

Arachni Public Source License v1.0.

Resources and Downloads

• Download and Source-code.

GitHub - Arachni/arachni: Web Application Security Scanner Framework

Web Application Security Scanner Framework. Contribute to Arachni/arachni development by creating an account on GitHub.

GitHubArachni