This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

Machine Learning for Web

Vulnerability Detection
The Case of Cross-Site Request Forgery

Stefano Calzavara | Università Ca’ Foscari Venezia

Mauro Conti | University of Padua
Riccardo Focardi and Alvise Rabitti | Università Ca’ Foscari Venezia
Gabriele Tolomei | Sapienza University of Rome

We propose a methodology to leverage machine learning (ML) for the detection of web application
vulnerabilities. We use it in the design of Mitch, the first ML solution for the black-box detection
of cross-site request forgery vulnerabilities. Finally, we show the effectiveness of Mitch on real
software.

W eb applications are the most common inter-

face to security-sensitive data and functional-
ity currently available. They are routinely used to file
particularly popular.2–4 As opposed to white-box
techniques, which require access to the web applica-
tion source code, black-box methods operate at the
income taxes, access the results of medical screenings, level of HTTP traffic, that is, requests and responses.
perform financial transactions, and share opinions with Although this limited perspective might miss impor-
our circle of friends, just to mention a few popular use tant insights, it has the key advantage of offering a
cases. On the downside, this means that web applica- language-agnostic vulnerability detection approach,
tions are appealing targets to malicious users (attackers) which abstracts from the complexity of scripting
who are determined to force economic losses, unduly languages and offers a uniform interface to the wid-
access confidential data, or create embarrassment for est possible range of web applications. This sounds
their victims. appealing, yet previous work showed that such an
It is well known that securing web applications is analysis is far from trivial.5,6 One of the main chal-
difficult.1 There are several reasons for this, ranging lenges is how to expose to automated tools a critical
from the heterogeneity and complexity of the web ingredient of effective vulnerability detection, that is,
platform to the adoption of undisciplined script- an understanding of the web application semantics.
ing languages offering dubious security guarantees
and not amenable for static analysis. In such a set- Example: Cross-Site Request Forgery
ting, black-box vulnerability detection methods are Cross-site request forgery (CSRF) is a well-known
web attack that forces a user into submitting unwanted,
Digital Object Identifier 10.1109/MSEC.2019.2961649
attacker-controlled HTTP requests toward a vulner-
Date of current version: 22 January 2020 able web application in which he or she is currently

2 May/June 2020 Copublished by the IEEE Computer and Reliability Societies  1540-7993/20©2020IEEE
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

authenticated. The key concept of CSRF is that the

malicious requests are routed to the web application Log In
through the user’s browser; therefore, they might be 1
indistinguishable from intended benign requests that Get Session Cookie
were actually authorized by the user.
A typical CSRF attack works as follows (Figure 1). 3 2 Visit Unrelated Website

1. Alice logs into an honest yet vulnerable web appli-

cation, for example, her preferred social network. Get Page With mal. adv
Session authentication is implemented through a
session cookie that is automatically attached by the
browser to any subsequent request toward the web Like Dictator
application.
2. Alice opens another tab and visits an unrelated web-
site, such as a newspaper website, which returns a
web page including a malicious advertisement. Figure 1. An example of CSRF.
3. The malicious advertisement sends a cross-site
request to the social network using HTML or
JavaScript, for example, asking to “like” a given politi- ■■ checking the presence of unpredictable anti-CSRF
cal party. Since the request includes Alice’s cook- tokens, set by the server into sensitive forms.
ies, it is processed in her authentication context at
the social network. This way, the malicious adver- A recent article discusses the pros and cons of
tisement can force Alice into putting a “like” to the these different solutions.3 However, all three options
desired political party, which might skew the result suffer from the same limitation: they require a care-
of online surveys. ful and fine-grained placement of security checks. For
example, tokens should be attached to all and only
CSRF does not require the attacker to intercept or the security-sensitive HTTP requests, to ensure com-
modify the user’s requests and responses: it suffices plete protection without harming the user experience.
that the victim visits the attacker’s website, from which Using a token to protect a “like” button is useful to
the attack is launched. Thus, CSRF vulnerabilities are prevent the attack discussed previously, yet having a
exploitable by any malicious website. token on the social network home page is undesir-
able, because it might lead to rejection of legitimate
Preventing CSRF cross-site requests, such as from clicks on the results
To prevent CSRF, web developers must implement of a search engine indexing the social network.
explicit protection mechanisms.7 If adding extra user In the end, finding the optimal placement of
interaction does not affect usability too much, it is pos- anti-CSRF defenses is typically a daunting task for
sible to force reauthentication or use one-time pass- web developers. Modern web application develop-
words/captchas to prevent cross-site requests from ment frameworks provide automated support for this,
going through unnoticed. In many cases, however, auto- yet CSRF vulnerabilities are still routinely found even
mated prevention is preferred: the recently introduced in top-ranked websites.2 This underscores the need for
SameSite cookie attribute can be used to prevent cookie effective CSRF detection tools, but how can we provide
attachment on cross-site requests, which solves the root automated tool support for CSRF detection if we have
cause of CSRF and is highly recommended for new no mechanized way to detect which HTTP requests are
web applications. Unfortunately, this defense is not yet actually security sensitive?
widespread, and existing web applications typically fil-
ter out cross-site requests by using any of the following Machine Learning to the Rescue
techniques: The CSRF example in the previous section shows that
it is useful to enrich vulnerability detection tools with
■■ checking the value of standard HTTP request head- semantic information to minimize the numbers of
ers, such as Referrer and Origin, indicating the false positives and false negatives. At the very least, one
page originating the request would desire a method to automatically classify HTTP
■■ checking the presence of custom HTTP request head- requests as security sensitive or not to restrict the analy-
ers, such as X-Requested-With, which cannot be set sis to the former. However, this is particularly challeng-
from a cross-site position ing on the web, since HTTP requests have a relatively

www.computer.org/security 3
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

weak syntactic structure, and custom programming 1. Use supervised learning to automatically train a
practices abound. For example, there are many differ- classifier that partitions selected web objects of
ent plausible ways to implement a “like” button for some interest, such as HTTP requests, HTTP responses,
content identified by the unique string 3aa5bf, including or cookies, based on the web application semantics.
For example, in the case of CSRF detection, the clas-
■■ a GET request to the page like.php with a single sifier would be used to identify security-sensitive
parameter id = 3aa5bf HTTP requests.
■■ a GET request to the page manage.php with a param- 2. For each possible class returned by the classifier,
eter id = 3aa5bf and a parameter action = like define a heuristic for vulnerability detection. Even
■■ a POST request to the page manage.php including trivial heuristics marking every object in a given
a JavaScript Object Notation object {id: 3aa5bf, class as nonvulnerable are plausible. For exam-
action: upvote}. ple, insensitive requests cannot be exploited for
All of these requests look semantically similar to CSRF; hence, they can be immediately marked as
experienced security testers, yet they are syntactically nonvulnerable.
different, and it might be hard to identify all of the 3. Use the classifier to choose the appropriate vulner-
most common ways to encode the same information ability detection heuristic to run on each web object
in the wild. of interest, such as part of a browser extension.

Supervised Learning We successfully leveraged this methodology in a

Luckily, machine learning (ML) provides effective tools number of research papers.12–14 Here, we report our
to automate classification tasks. A classifier can be seen most up-to-date study on CSRF detection.14
as the function f : X " Y mapping any object from
the feature space X into a corresponding class from Mitch: ML-Based Detection of CSRF
Y. The subfield of supervised learning studies effective Mitch is the first tool for the black-box detection of
techniques to automatically generate classifiers starting CSRF vulnerabilities; its design is based on the method-
from a set of labeled data.8 Therefore, to fruitfully use ology presented in the previous section. Mitch is avail-
supervised learning, one must take the following steps. able online16 as a browser extension for Mozilla Firefox.
We refer readers to our recent research article for full
1. Collect a set of objects of interest O, for exam- details.14
ple, HTTP requests sent to representative web
applications. Overview
2. Define the set of classes Y. For example, one could set Mitch assumes the possession of two test accounts
Y = " +1, - 1 , to discriminate the security-sensitive (say, Alice and Bob) at the website where the secu-
requests (+1) from all the other requests (–1). rity testing is to be performed. This is used to simu-
3. Define the feature space X by manually identi- late a scenario in which the attacker (Alice) inspects
fying the salient aspects that look useful to assign sensitive HTTP requests in her session to force the
the objects in O to their correct class in Y. For forgery of such requests in the browser of the victim
example, one could leverage the request length, the (Bob). Having two test accounts is crucial for the
request method, or the presence of selected key- precision of the tool because if the forged requests
words in the request body. contain some information that is bound to Alice’s ses-
4. Build a training set D of pairs ^ xv, y h, where each sion, then CSRF against Bob may not be possible. For
xv is the encoding in X of an object o d O and y example, if a website defends against CSRF through
is its class. the use of anti-CSRF tokens, then Alice’s requests
will be rejected in Bob’s session. The use of two test
Once this is done, supervised learning can automatically accounts for CSRF detection has already been advo-
extract the best-performing classifier from a set of possible cated in previous work2 and is part of traditional man-
hypotheses H by estimating its performance on the train- ual testing strategies.17
ing set D. As long as there is enough manually curated data The architecture of Mitch is shown in Figure 2.
in D, the performance of supervised learning can compete After installing Mitch in his/her browser, the security
with or even outclass that of human experts.9–11 tester first navigates the website as Alice: for every
HTTP request detected as sensitive by the classi-
Web Vulnerability Detection fier, Mitch stores the content of the corresponding
The methodology we put forward can be summarized HTTP response. After completing the navigation,
as follows. Mitch uses the collected sensitive HTTP requests to

4 IEEE Security & Privacy May/June 2020

 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

generate new HTML elements in the extension ori- dynamically generated elements, which might realis-
gin, which allows for replaying them. The security tically differ even when the same idempotent opera-
tester then authenticates to the website as Bob, and tion is performed multiple times. Thus, Mitch builds
Mitch exploits the generated HTML to automati- on the notion of dissimilar HTTP responses. In gen-
cally replay the detected sensitive requests from a eral, the dissimilarity of HTTP responses is much
cross-site position, which simulates a CSRF attack. easier to check than their similarity, for example, due
Finally, the responses collected for Alice and Bob are to the use of different status codes or content types to
compared: if a response received by Bob “matches” denote failures (e.g., status codes 401 Unauthorized
the one received by Alice, it means that Alice was able and 403 Forbidden are typical ways to denote unau-
to forge a valid request for Bob’s session; hence, the thorized access). When Bob’s response is dissimilar
attack is considered successful, and Mitch reports a from Alice’s response, it is likely that Alice’s request
potential CSRF vulnerability. failed in Bob’s session, which might indicate the use
of a CSRF protection mechanism.
Challenges
The proposed CSRF detection heuristic is intuitive, yet Changes in Session State
there are several challenges to solve to make it work in Since the state of Alice and Bob at the website might be
practice. We provide a high-level view of these issues different, matching the response received by Bob against
and our proposed solutions. the one received by Alice might be an improper way to
detect a CSRF vulnerability. For instance, Bob might
Changes in HTTP Responses not be able to perform a sensitive operation because it
Defining a suitable notion of “matching” HTTP does not have access to the file foo, yet a CSRF attack
responses for Alice’s and Bob’s sessions is gener- would work if it targeted the file bar. When comparing
ally hard, because HTTP responses may include the response received by Bob against the one received

Credentials

Trace Generator

Sensitive
HTTP
Requests
ML
Ta T′a Tb Tu
Classifier

Insensitive

CSRF Detection
Algorithm

Mitch

Figure 2. The architecture of Mitch.

www.computer.org/security 5
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

by Alice, Mitch does not immediately consider their dis- have a very weak structure, and it is hard to come up
similarity as a definite evidence that the request of Bob with general yet accurate typing techniques for them.
had a different outcome than the one of Alice due to the
use of a CSRF protection mechanism. Rather, since dif- Textual
ferent outcomes might come from a difference in the This category of features captures textual characteris-
state of Alice’s and Bob’s sessions, Mitch also replays tics of HTTP requests and is based on a small manually
Alice’s original request in a fresh Alice session: if the curated vocabulary of keywords V that may occur in the
new response received by Alice is dissimilar to the origi- request, resulting from a manual inspection of sensitive
nal one, it is likely that session-dependent information requests from a sample of real-world websites consid-
is required to process the request, which might indicate ered in our data set. More specifically, we consider only
the adoption of an anti-CSRF token. binary features of the following forms:

Classification Errors ■■ wordInPath, where word ∈V means the presence of

Even a very accurate classifier might incorrectly mark the string word in the request path
an insensitive request as sensitive. In this case, there is ■■ wordInParams, where word ∈V means the presence of
no CSRF vulnerability, and the presence of matching the string word in any parameter name of the request.
responses for Alice’s and Bob’s sessions should not raise an
alarm. To detect potential false positives produced by the The vocabulary V includes the following 21 keywords,
classifier, Mitch replays Alice’s original request without first which have been selected as possible signals of sensitive
authenticating to the website, that is, outside any session: if requests, according to common sense and a preliminary
the received response is dissimilar from the original one, inspection of the part of our data set that is reserved for
then there is further evidence that the requested operation training: create, add, set, delete, update, remove, friend,
required an authenticated context to be performed, which setting, password, token, change, action, pay, login, log-
confirms that there exists potential room for CSRF. out, post, comment, follow, subscribe, sign, and view.

ML Classifier Functional
The ML classifier used by Mitch was trained from a This category of features indicates the HTTP method
data set of approximately 6,000 HTTP requests from associated with the request. We consider just the follow-
existing websites, collected and labeled by two human ing two binary features:
experts. The feature space X of the classifier has 49
dimensions, each one capturing a specific property of ■■ isGET: the HTTP request method is GET
HTTP requests. Those can be organized into three cat- ■■ isPOST: the HTTP request method is POST.
egories: structural, textual, and functional.
There are no additional alternatives, because our
Structural data set includes only GET and POST requests. All
This category of features describes structural properties other requests can be easily labeled as sensitive or not
of an HTTP request. More precisely, we define the fol- just based on their method; for example, OPTIONS
lowing set of numerical features: requests are always insensitive.

■■ numOfParams: the total number of parameters Experimental Evaluation

■■ numOfBools: the number of request parameters In this section, we evaluate the effectiveness of Mitch in
bound to a Boolean value detecting CSRF vulnerabilities. In particular, we show
■■ numOfIds: the number of request parameters bound that the number of false positives and false negatives
to an identifier, that is, a hexadecimal string, whose produced by Mitch is remarkably low and amenable for
usage was empirically observed to be common in our practical use.
data set
■■ numOfBlobs: the number of request parameters False Positives and False Negatives
bound to a blob, that is, any string that is not an Mitch produces a false positive when it returns a can-
identifier didate CSRF that cannot be actually exploited. This is
■■ reqLen: the total number of characters in the request, something relatively easy to detect by manual testing,
including parameter names and values. although this process is tedious and time consuming. In
general, it is not possible to reliably identify when Mitch
Although one might devise more sophisticated tech- produces a false negative, because this would require
niques to “type” request parameters, HTTP requests knowing all of the CSRF vulnerabilities on the tested

6 IEEE Security & Privacy May/June 2020

 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

websites. To estimate this important aspect, we keep in Table 1 and is discussed in the following sections.
track of all of the sensitive requests returned by the ML Many of the attacks we found targeted the social
classifier embedded into Mitch, and we focus our manual functionalities of the websites we tested, such as cast-
testing on those cases. This is a reasonable choice to make ing votes on public contents, adding or removing items
the analysis tractable, because we first showed that the from favorite lists, and posting comments under the
classifier performs well using standard validity measures. identity of the victim. Therefore, most of these attacks
may affect recommender systems, lead to social embar-
Assessment on Existing Websites rassment, and compromise user reputation. Worse, we
To test how effective Mitch is on existing websites, we were also able to find a number of attacks that seriously
sampled 20 websites from the Alexa Top 10,000 rank- compromised the website functionality; we responsibly
ing. We considered only websites with single sign-on disclosed all of the vulnerabilities to the respective web-
access via a major social network website so that we site owners. We discuss a few interesting cases here.
could leverage just two existing social accounts to per-
form our security testing. Bombas
Overall, Mitch found 191 sensitive requests and Bombas is an e-commerce website selling socks. It pro-
reported 47 potential CSRF vulnerabilities: we were able vides a functionality to store a list of shipping addresses
to immediately exploit 35 of them, exposing major secu- to simplify purchases, so that shipping details do not
rity issues in a few cases. We estimated only seven false need to be entered for each transaction. The form used
negatives in total, which means that our heuristics are to store a new shipping address is vulnerable to CSRF,
accurate enough to capture most of the vulnerabilities. so an attacker can force any address into the victim’s
The full breakdown of the individual websites is shown account to hijack deliveries. By default, the latest added

Table 1. CSRF detection on existing websites.

Website Sensitive requests Detected CSRFs False positives False negatives

9gag.com 10 3 1 0
ask.fm 16 0 0 0
askubuntu.com 16 0 0 0
bombas.com 2 1 0 1
brilio.net 2 1 0 1
eprice.it 11 3 0 3
flixbus.com 4 1 1 0
funnyjunk.com 17 8 2 2
gsmarena.com 3 3 0 0
imdb.com 10 0 0 0
imgur.com 12 3 3 0
indeed.com 8 4 0 0
instructables.com 11 4 0 0
mocospace.com 7 5 2 0
pornhub.com 13 2 1 0
smokecartel.com 5 2 0 0
starnow.com.au 8 4 0 0
tomshardware.com 13 1 1 0
wish.com 11 0 0 0
yelp.com 12 2 1 0
Total 191 47 12 7

www.computer.org/security 7
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

address is the one that is used, which makes the attack severe, because it affects the form used to set the email
even worse in terms of practical impact. address of user profiles. By exploiting this vulnerability,
Bombas is a customer of Shopify, which is a major the attacker can set the victim’s email address to his/her
e-commerce platform, so this attack may also affect own address and then use the password reset functional-
many other websites. We reported the issue to Shopify, ity of Starnow to get a fresh password for the victim in
which acknowledged the attack and is working on a fix her inbox, thus taking possession of the victim’s account.
but marked our report as duplicate due to the existence
of a previous independent disclosure. Assessment on Production Software
As a second set of experiments, we decided to run Mitch
Indeed on the testbed of open source web applications used to
Indeed is one of the largest websites hosting job offers. evaluate Deemon, a state-of-the-art automated detection
Registered users can send their CVs and apply to different tool for CSRF vulnerabilities.15 Since Deemon works
open positions around the world. We found three CSRF only on Hypertext Preprocessor applications whose
vulnerabilities that give an attacker the ability to fully source code is available for dynamic analysis, we could not
manage job offers associated with the account, including test it on the closed source websites from our first set of
the possibility of storing new offers and archiving exist- experiments. Out of the 10 applications considered in the
ing ones. Indeed also suffers from a CSRF vulnerabil- original testbed, we were able to find only three applica-
ity on the form used to set user preferences, which can tions at the same version: Oxid e-shop, Prestashop, and
severely affect the visibility of job offers. An attacker can Simple Machine Forums. No CSRF vulnerability was
exploit this vulnerability to hide job offers, for instance, detected by Deemon on these applications, according to
by restricting the search radius and changing the desired the experimental evaluation by Sudhodanan et al.15 The
publication date for displayed offers. results of the analysis performed by Mitch on the applica-
We find these vulnerabilities particularly interest- tions in their default configuration are shown in Table 2.
ing, because Indeed is making wide use of anti-CSRF Mitch was extremely effective on the tested applica-
tokens, and all of the vulnerable forms have their own tions, because it reported only two false positives and
token. However, it seems that not all of the tokens are it was able to catch three CSRF vulnerabilities on Oxid
correctly checked by the website, which may suggest e-shop that were not reported by Deemon.15 These vul-
a manual, error-prone placement of the tokens. More nerabilities allow an attacker to corrupt the integrity of
generally, this shows that checking the presence of the shopping cart, force the use of vouchers, and change
anti-CSRF tokens is not sufficient to say that a website the preferred payment method. All of the correspond-
is protected against CSRF and that the actual website ing functionalities are supposed to be protected by an
behavior should be tested instead. The security team anti-CSRF token, which however, is not checked by the
of Indeed acknowledged the issue and rewarded us Oxid back end. We reported the issues to the Oxid secu-
US$100 for the finding. rity team, who acknowledged the problem and worked
on a fix.
Starnow
Starnow is an Australian website designed to discover Freeware and Open Source Software
new talents, such as singers and actors. Users who are Penetration testers have been using a range of differ-
interested in pursuing an artistic career can register on ent tools to detect CSRF vulnerabilities in web appli-
the website to get access to a number of auditions and cations. Based on extensive research on blogs, forums,
job interviews. The first two CSRFs we found allow and resources for security practitioners, including the
an attacker to arbitrarily manipulate the watchlists of OWASP Testing Guide, we classified existing tools in
authenticated users, thus compromising a functionality the following categories.
offered by the website.
However, there are two much worse attacks. A CSRF ■■ Intercepting proxies allow penetration testers to inter-
vulnerability affects the form used to store the phone cept and modify arbitrary HTTP traffic, which can be
number associated with user profiles; this can be used used for an essentially manual detection of web vul-
for scams or to disrupt the functionality of the website, nerabilities, including CSRF. Popular tools in this cat-
such as by making it impossible to contact the victim for egory are Burp, ZAP, and WebScarab.
an audition. The request used to set the phone number ■■ Exploit generators simplify the generation of proof of
contains an anti-CSRF token, but it is not checked by concepts for attack finding, based on human guidance
the website, confirming that this type of mistake is not on the set of HTTP requests that need to be tested for
confined to Indeed but is apparently more widespread. CSRF. Examples tools in this category include CSRF-
The last CSRF vulnerability is definitely the most Tester and pinata-csrf-tool.

8 IEEE Security & Privacy May/June 2020

 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

Table 2. CSRF detection on production software.

Web application Sensitive requests Detected CSRFs False positives False negatives
Oxid e-shop 4.9.8 21 4 1 0
Prestashop 1.6.1.2 12 1 1 0
Simple Machine Forums 2.0.12 9 0 0 0
Total 42 5 2 0

■■ Web application scanners automatically detect a range

of web application vulnerabilities, including CSRF,
based on different heuristics. Scanners supporting
W eb applications are particularly challenging to
analyze due to their diversity and the wide-
spread adoption of custom programming practices.
modules for CSRF are Arachni, Skipfish, and w3af. ML is, thus, very helpful in the web setting, because it
can take advantage of manually labeled data to expose
Our work improves on intercepting proxies and the human understanding of web application semantics
exploit generators by providing effective automated to automated analysis tools. We validated this claim by
techniques for the detection and the exploitation of designing Mitch, the first ML solution for the black-box
sensitive HTTP requests, as opposed to manual inves- detection of CSRF vulnerabilities, and by experimentally
tigation and testing. The most important advances over assessing its effectiveness. We hope other researchers
web application scanners are, instead, the use of ML for will take advantage of our methodology for the detec-
sensitive request detection, a more sophisticated CSRF tion of other classes of web application vulnerabilities.
detection algorithm, and a systematic evaluation of the
performance of our detection tool, based on the analysis Acknowledgment
of false positives and false negatives produced on real This article was supported by the project Machine Learning
web applications. We noticed important design limita- for Web Security, funded by the Incentivi alla Ricerca Individ-
tions in the open source tools we analyzed that signifi- uale program of Università Ca’ Foscari Venezia.
cantly downgraded their accuracy.
For example, Arachni detects CSRF vulnerabili- References
ties only on forms requiring an authenticated context; 1. S. Calzavara, R. Focardi, M. Squarcina, and M. Tempesta,
therefore, it does not capture CSRF attempts via links “Surviving the web: A journey into web session security,”
or Ajax. The rationale behind this choice is likely the ACM Comput. Surv., vol. 50, no. 1, pp. 13:1–13:34, 2017.
complexity of detecting sensitive HTTP requests, doi: 10.1145/3038923.
which forced the developers of Arachni to limit their 2. A. Sudhodanan, R. Carbone, L. Compagna, N. Dol-
tool to HTML elements that are potentially danger- gin, A. Armando, and U. Morelli, “Large-scale analysis
ous yet easy to catch syntactically (forms). It is instruc- & detection of authentication cross-site request forger-
tive that w3af suffers from a somewhat opposite design ies,” in Proc. 2017 IEEE European Symp. Security and Pri-
choice: since any request, which includes cookies and vacy (EuroS&P 2017), Paris, France, pp. 350–365. doi:
parameters, is deemed as potentially sensitive by w3af, 10.1109/EuroSP.2017.45.
the tool is affected by many false positives, which led to 3. S. Calzavara, A. Rabitti, A. Ragazzo, and M. Bugliesi,
the opening of an issue on GitHub, where a major rede- “Testing for integrity flaws in web sessions,” in Proc.
sign of the tool is recommended.18 Computer Security 24rd European Symp. Research Com-
Other issues we found are related to the choice of deem- puter Security (ESORICS 2019), Luxembourg, Lux-
ing secure any HTTP request that includes an anti-CSRF embourg, Sept. 23–27, 2019, pp. 606–624. doi:
token, although we observed several cases in which tokens 10.1007/978-3-030-29962-0_29.
are not checked at the web application back end, and to the 4. “OWASP testing guide,” OWASP, Bel Air, MD, 2016.
use of just a single authenticated session, which loses preci- https://www.owasp.org/index.php/OWASP_Testing
sion when user-dependent secrets happen to thwart CSRF _Guide_v4_Table_of_Contents
attempts. Preliminary tests with existing web application 5. J. Bau, E. Bursztein, D. Gupta, and J. C. Mitchell, “State of
scanners on simple examples returned a high number of the art: Automated black-box web application vulnerabil-
false positives and false negatives, which is in line with the ity testing,” in Proc. 31st IEEE Symp. Security and Privacy
findings of previous research work that showed the ineffec- (S&P 2010), Berkeley/Oakland, CA, May 16–19, 2010,
tiveness of such scanners for CSRF detection.5,6 pp. 332–345. doi: 10.1109/SP.2010.27.

www.computer.org/security 9
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6. A. Doupé, M. Cova, and G. Vigna, “Why Johnny can’t 17. Portswigger Web Security, “Using Burp to Test for
pentest: An analysis of black-box web vulnerability scan- Cross-Site Request Forgery (CSRF),” Knutsford, UK.
ners,” in Proc. 7th Int. Conf. Detection of Intrusions and Mal- [Online]. Available: https://support.portswigger.net/
ware, and Vulnerability Assessment (DIMVA 2010), Bonn, customer/portal/articles/1965674-using-burp-to-test-
Germany, July 8–9, 2010. pp. 111–131. for-cross-site-request-forgery-csrf-
7. A. Barth, C. Jackson, and J. C. Mitchell, “Robust defenses 18. A. Rancho, “w3af,” GitHub. Accessed on: Jan. 15, 2020.
for cross-site request forgery,” in Proc. 2008 ACM Conf. [Online]. Available: https://github.com/andresriancho/
Computer and Communications Security (CCS 2008), Alex- w3af/issues/120
andria, VA, pp. 75–88. doi: 10.1145/1455770.1455782.
8. M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations Stefano Calzavara is a tenure-track assistant professor
of Machine Learning. Cambridge, MA: MIT Press, 2012. at Università Ca’ Foscari Venezia, Italy. His research
9. M. W. Kattan, D. A. Adams, and M. S. Parks, “A compari- interests include formal methods and web security.
son of machine learning with human judgment,” J. Man- Calzavara received a Ph.D. in computer science from
age. Inf. Syst., vol. 9, no. 4, pp. 37–57, Mar. 1993. doi: the Università Ca’ Foscari Venezia, Italy, in 2013.
10.1080/07421222.1993.11517977. Contact him at [email protected].
10. D. A. Ferrucci, “Introduction to ‘This is Watson’,” IBM J.
Res. Develop., vol. 56, no. 3.4, pp. 1:1–1:15, May 2012. doi: Mauro Conti is a full professor at the University of
10.1147/JRD.2012.2184356. Padua, Italy. His research interests include computer
11. D. Silver et al., “Mastering the game of Go with deep neu- security and privacy. Conti received a Ph.D. in com-
ral networks and tree search,” Nature, vol. 529, no. 7587, puter science from Sapienza University of Rome,
pp. 484–489, Jan. 2016. doi: 10.1038/nature16961. Italy, in 2009. Contact him at [email protected].
12. M. Bugliesi, S. Calzavara, R. Focardi, and W. Khan,
“CookiExt: Patching the browser against session hijacking Riccardo Focardi is a full professor at Università Ca’ Fos-
attacks,” J. Comput. Security, vol. 23, no. 4, pp. 509–537, cari Venezia, Italy. His research interests include com-
2015. doi: 10.3233/JCS-150529. puter security and formal methods. Focardi received
13. S. Calzavara, G. Tolomei, A. Casini, M. Bugliesi, and S. a Ph.D. in computer science from the University of
Orlando, “A supervised learning approach to protect cli- Bologna, Italy, in 1999. Contact him at focardi@
ent authentication on the web,” ACM Trans. Web, vol. 9, unive.it.
no. 3, pp. 15:1–15:30, 2015. doi: 10.1145/2754933.
14. S. Calzavara, M. Conti, R. Focardi, A. Rabitti, and G. Alvise Rabitti is a security officer at Università Ca’ Fos-
Tolomei, “Mitch: A machine learning approach to the cari Venezia, Italy. His research interests include web
black-box detection of CSRF vulnerabilities,” in Proc. security and privacy. Rabitti received a B.S. in com-
IEEE European Symp. Security and Privacy (EuroS&P puter science from the Università Ca’ Foscari Venezia,
2019), Stockholm, Sweden, June 17–19, 2019, pp. 528– Italy, in 2013. Contact him at [email protected].
543. doi: 10.1109/EuroSP.2019.00045.
15. G. Pellegrino, M. Johns, S. Koch, M. Backes, and C. Ros- Gabriele Tolomei is an associate professor at Sapienza
sow, “Deemon: Detecting CSRF with dynamic analysis University of Rome, Italy. His research interests
and property graph,” in Proc. 2017 ACM SIGSAC Conf. include machine learning and web search. Tolomei
Computer and Communications Security (CCS 2017), received a Ph.D. in computer science from the Univer-
Dallas, TX, Oct. 30–Nov. 3, 2017, pp. 1757–1771. doi: sità Ca’ Foscari Venezia, Italy, in 2011. Contact him at
10.1145/3133956.3133959. [email protected].
16. S. Calzavara, M. Conti, R. Focardi, A. Rabitti, and G.
Tolomei, “mitch,” GitHub. Accessed on: Jan. 15, 2020.
[Online]. Available: https://github.com/alviser/mitch

10 IEEE Security & Privacy May/June 2020