Google Online Security Blog: 2015

An update on SHA-1 certificates in Chrome

December 18, 2015

Posted by Lucas Garron, Chrome security and David Benjamin, Chrome networkingannounced last Septemberfurther recent researchStep 1: Blocking new SHA-1 certificates

is signed with a SHA-1-based signature

is issued on or after January 1, 2016

chains to a public CA

We are hopeful that no one will encounter this error, since public CAs must stop issuing SHA-1 certificates in 2016 per the Baseline Requirements for SSL.
In addition, a later version of Chrome in 2016 may extend these criteria in order to help guard against SHA-1 collision attacks on older devices, by displaying a certificate error for sites with certificate chains that:

contain an intermediate or leaf certificate signed with a SHA-1-based signature
contain an intermediate or leaf certificate issued on or after January 1, 2016
chain to a public CA

(Note that the first two criteria can match different certificates.)
Note that sites using new SHA-1 certificates that chain to local trust anchors (rather than public CAs) will continue to work without a certificate error. However, they will still be subject to the UI downgrade specified in our original announcement.
Step 2: Blocking all SHA-1 certificates
Starting January 1, 2017 at the latest, Chrome will completely stop supporting SHA-1 certificates. At this point, sites that have a SHA-1-based signature as part of the certificate chain (not including the self-signature on the root certificate) will trigger a fatal network error. This includes certificate chains that end in a local trust anchor as well as those that end at a public CA.
In line with Microsoft Edge and Mozilla Firefox, the target date for this step is January 1, 2017, but we are considering moving it earlier to July 1, 2016 in light of ongoing research. We therefore urge sites to replace any remaining SHA-1 certificates as soon as possible.
Note that Chrome uses the certificate trust settings of the host OS where possible, and that an update such as Microsoft’s planned change will cause a fatal network error in Chrome, regardless of Chrome’s intended target date.
Keeping your site safe and compatible
As individual TLS features are found to be too weak, browsers need to drop support for those features to keep users safe. Unfortunately, SHA-1 certificates are not the only feature that browsers will remove in the near future.
As we announced on our security-dev mailing list, Chrome 48 will also stop supporting RC4 cipher suites for TLS connections. This aligns with timelines for Microsoft Edge and Mozilla Firefox.
For security and interoperability in the face of upcoming browser changes, site operators should ensure that their servers use SHA-2 certificates, support non-RC4 cipher suites, and follow TLS best practices. In particular, we recommend that most sites support TLS 1.2 and prioritize the ECDHE_RSA_WITH_AES_128_GCM cipher suite. We also encourage site operators to use tools like the SSL Labs server test and Mozilla's SSL Configuration Generator.

Indexing HTTPS pages by default

December 17, 2015

Posted by Zineb Ait Bahajji, WTA, and the Google Security and Indexing teams[Cross-posted from the Webmaster Central Blog]GmailGoogle searchranking boost to HTTPS URLseavesdroppingman-in-the-middle attacksHTTPS everywhere

It doesn’t contain insecure dependencies.

It isn’t blocked from crawling by robots.txt.

It doesn’t redirect users to or through an insecure HTTP page.

It doesn’t have a rel="canonical" link to the HTTP page.

It doesn’t contain a noindex robots meta tag.

It doesn’t have on-host outlinks to HTTP URLs.

The sitemaps lists the HTTPS URL, or doesn’t list the HTTP version of the URL.

The server has a valid TLS certificate.

Although our systems prefer the HTTPS version by default, you can also make this clearer for other search engines by redirecting your HTTP site to your HTTPS version and by implementing the HSTS header on your server.
We’re excited about taking another step forward in making the web more secure. By showing users HTTPS pages in our search results, we’re hoping to decrease the risk for users to browse a website over an insecure connection and making themselves vulnerable to content injection attacks. As usual, if you have any questions or comments, please let us know in the comments section below or in our webmaster help forums.

Proactive measures in digital certificate security

December 11, 2015

Posted by Ryan Sleevi, Software Engineernotification by Symantec CorporationCA/Browser Forum’s Baseline Requirementscontact Symantec Technical SupportFurther Technical Details of Affected Root:Friendly Name:Subject:Public Key Hash (SHA-1):Public Key Hash (SHA-256):MD2 VersionFingerprint (SHA-1):Fingerprint (SHA-256):SHA1 VersionFingerprint (SHA-1)Fingerprint (SHA-256)

Year one: progress in the fight against Unwanted Software

December 9, 2015

Posted by Moheeb Abu Rajab, Google Security Team
“At least 2 or 3 times a week I get a big blue warning screen with a loud voice telling me that I’ve a virus and to call the number at the end of the big blue warning.”“I’m covered with ads and unwanted interruptions. what’s the fix?”“I WORK FROM HOME AND THIS POPING [sic] UP AND RUNNING ALL OVER MY COMPUTER IS NOT RESPECTFUL AT ALL THANK YOU.”
Launched in 2007More than a year agoWhat is UwS and how does it get on my computer?defining list of characteristics

It is deceptive, promising a value proposition that it does not meet.

It tries to trick users into installing it or it piggybacks on the installation of another program.

It doesn’t tell the user about all of its principal and significant functions.

It affects the user’s system in unexpected ways.

It is difficult to remove.

It collects or transmits private information without the user’s knowledge.

It is bundled with other software and its presence is not disclosed.

Next, we had to better understand how UwS is being disseminated.
This varies quite a bit, but time and again, deception is at the heart of these tactics. Common UwS distribution tactics include: unwanted ad injection, misleading ads such as “trick-to-click”, ads disguised as ‘download’ or ‘play’ buttons, bad software downloader practices, misleading or missing disclosures about what the software does, hijacked browser default settings, annoying system pop-up messages, and more.
Here are a few specific examples:

Deceptive ads leading to UwS downloads

Ads from unwanted ads injector taking over a New York Times page and sending the user to phone scams

Unwanted ad injector inserts ads on the Google search results page

New tab page is overridden by UwS

UwS hijacks Chrome navigations and directs users to a scam tech support website
One year of progress
Because UwS touches so many different parts of people’s online experiences, we’ve worked to fight it on many different fronts. Weaving UwS detection into Safe Browsing has been critical to this work, and we’ve pursued other efforts as well—here’s an overview:

We now include UwS in Safe Browsing and its API, enabling people who use Chrome and other browsers to see warnings before they go to sites that contain UwS. The red warning below appears in Chrome.

We launched the Chrome Cleanup Tool, a one-shot UwS removal tool that has helped clean more than 40 million devices. We shed more light on a common symptom of UwS—unwanted ad injectors. We outlined how they make money and launched a new filter in DoubleClick Bid Manager that removes impressions generated by unwanted ad injectors before bids are made.
We started using UwS as a signal in search to reduce the likelihood that sites with UwS would appear in search results.
We started disabling Google ads that lead to sites with UwS downloads.

It’s still early, but these changes have already begun to move the needle.

UwS-related Chrome user complaints have fallen. Last year, before we rolled-out our new policies, these were 40% of total complaints and now they’re 20%.
We’re now showing more than 5 million Safe Browsing warnings per day on Chrome related to UwS to ensure users are aware of a site’s potential risks.
We helped more than 14 million users remove over 190 deceptive Chrome extensions from their devices.
We reduced the number of UwS warnings that users see via AdWords by 95%, compared to last year. Even prior to last year, less than 1% of UwS downloads were due to AdWords.

However, there is still a long way to go. 20% of all feedback from Chrome users is related to UwS and we believe 1 in 10 Chrome users have hijacked settings or unwanted ad injectors on their machines. We expect users of other browsers continue to suffer from similar issues; there is lots of work still to be done.

Looking ahead: broad industry participation is essential
Given the complexity of the UwS ecosystem, the involvement of players across the industry is key to making meaningful progress in this fight. This chain is only as strong as its weakest links: everyone must work to develop and enforce strict, clear policies related to major sources of UwS.
If we’re able, as an industry, to enforce these policies, then everyone will be able to provide better experiences for their users. With this in mind, we’re very pleased to see that the FTC recently warned consumers about UwS and characterizes UwS as a form of malware. This is an important step toward uniting the online community and focusing good actors on the common goal of eliminating UwS.
We’re still in the earliest stages of the fight against UwS, but we’re moving in the right direction. We’ll continue our efforts to protect users from UwS and work across the industry to eliminate these bad practices.

A new version of Authenticator for Android

December 7, 2015

Posted by Alexei Czeskis, Software Engineer2-Step VerificationAndroid Wear

specificationshttps://u2fdemo.appspot.comSecurity KeyAuthenticator for Android on the Play Store

Protecting hundreds of millions more mobile users

December 7, 2015

Posted by Noé Lutz, Nathan Parker, Stephan Somogyi; Google Chrome and Safe Browsing Teamsmalware, unwanted software, and social engineering sitesHow To Get It
while preserving your privacy, just like on desktopWhat Came Before
What’s New

Social engineering

Mobile data costs money for most users around the world. Data size matters a lot.

Mobile data speeds are slower than Wi-Fi in much of the world. Data size matters a lot.

Cellular connectivity quality is much more uneven, so getting the right data to the device quickly is critically important. Data size matters a lot.

Maximum Protection Per Bit
More Mobile

Safe Browsing protection from even more deceptive attacks

November 13, 2015

Posted by Emily Schechter, Program Manager and Noé Lutz, Software EngineerSafe Browsingtraditional phishing attacks

The content pretends to act, or looks and feels, like a trusted entity — like a bank or government.

The content tries to trick you into doing something you’d only do for a trusted entity — like sharing a password or calling tech support.

This page tries to trick you into downloading and executing malware or unwanted software. It uses Chrome’s logo and name to confuse you into believing the site is operated by Google. Content like this may include an inconspicuous legal disclaimer that states it is not affiliated with Google. This does not change the deceptive nature of this content—as always, use caution when downloading files from the web.

This is a fake tech phone support page. This page mimics a warning and may trick you into calling a third-party company that pretends to be Google or some other trusted entity, but charges a fee for support. (Chrome does not offer paid remote support).

This is a fake Google login page. It might trick you into disclosing your account login credentials. Other phishing sites like this could trick you into giving up other personal information such as credit card information. Phishing sites may look exactly like the real site—so be sure to look at the address bar to check that the URL is correct, and also check to see that the website begins with https://. See more information here.
If we identify that a web page contains social engineering content, Chrome will warn you by displaying the following warning:

(If you believe Safe Browsing has classified a web page in error, please report it here.)
We'll continue to improve Google's Safe Browsing protection to help more people stay safer online. Check out the Safe Browsing Transparency Report to find out more.

New Research: Encouraging trends and emerging threats in email security

November 12, 2015

default HTTPSSafer Email Transparency reportmulti-year studySafer Email Transparency report
Email security strengthens, industry-wide

Newer security challenges and how we can address them
Our study identified several new security challenges as well.
First, we found regions of the Internet actively preventing message encryption by tampering with requests to initiate SSL connections. To mitigate this attack, we are working closely with partners through the industry association M3AAWG to strengthen “opportunistic TLS” using technologies that we pioneered with Chrome to protect websites against interception.
Second, we uncovered malicious DNS servers publishing bogus routing information to email servers looking for Gmail. These nefarious servers are like telephone directories that intentionally list misleading phone numbers for a given name. While this type of attack is rare, it’s very concerning as it could allow attackers to censor or alter messages before they are relayed to the email recipient.
While these threats do not affect Gmail to Gmail communication, they may affect messaging between providers. To notify our users of potential dangers, we are developing in-product warnings for Gmail users that will display when they receive a message through a non-encrypted connection. These warnings will begin to roll-out in the coming months.
All email services—Gmail included—depend on the trust of their users. Partnering with top researchers helps us make the email ecosystem as a whole safer and more secure for everyone. Security threats won’t disappear, but studies like these enable providers across the industry to fight them with better, more powerful protections today and going forward.
[This work was made possible thanks to the contribution of many Googlers including Vijay Eranti, Kurt Thomas, John Rae-Grant, and Mark Risher.]

Sustaining Digital Certificate Security

October 28, 2015

Posted by Ryan Sleevi, Software EngineerThis post updates our previous notification of a misissued certificate for google.coma report164 certificates2,458 certificates

A post-mortem analysis that details why they did not detect the additional certificates that we found.

Details of each of the failures to uphold the relevant Baseline Requirements and EV Guidelines and what they believe the individual root cause was for each failure.

Following the implementation of these corrective steps, we expect Symantec to undergo a Point-in-time Readiness Assessment and a third-party security audit. The point-in-time assessment will establish Symantec’s conformance to each of these standards:

WebTrust Principles and Criteria for Certification Authorities
WebTrust Principles and Criteria for Certification Authorities – SSL Baseline with Network Security
WebTrust Principles and Criteria for Certification Authorities – Extended Validation SSL

The third-party security audit must assess:

The veracity of Symantec’s claims that at no time private keys were exposed to Symantec employees by the tool.
That Symantec employees could not use the tool in question to obtain certificates for which the employee controlled the private key.
That Symantec’s audit logging mechanism is reasonably protected from modification, deletion, or tampering, as described in Section 5.4.4 of their CPS.

We may take further action as additional information becomes available to us.

Behind the red warning: more info about online site safety

October 20, 2015

Posted by Adrienne Porter Felt, Chrome Security Engineer and Warning WizardEmily Schechter, Safe Browsing Program Manager and Menace to MalwareKe Wang, Safe Browsing Engineer and Developer of DefenseSite Status sectionSafe Browsing diagnostic pageunwanted software

Search ConsoleGoogle Analyticshelp centerTransparency Report

Simplifying the Page Security Icon in Chrome

October 13, 2015

Posted by Lucas Garron and Chris Palmer, Chrome security team

There are two reasons for this:

This change is a better visual indication of the security state of the page relative to HTTP.

Chrome users will have fewer security states to learn.

(Not) Warning About Mixed Content
This change will mainly affect HTTPS pages that contain certain mixed content, such as HTTP images.
Site operators face a dilemma: Switching an HTTP site to HTTPS can initially result in mixed content, which is undesirable in the long term but important for debugging the migration. During this process the site may not be fully secured, but it will usually not be less secure than before.
Removing the yellow “caution triangle” badge means that most users will not perceive a warning on mixed content pages during such a migration. We hope that this will encourage site operators to switch to HTTPS sooner rather than later.
Fewer Security States
This change will reduce the number of page security states in Chrome from four to three.
We have to strike a balance: representing the security state of a webpage as accurately as possible, while making sure users are not overwhelmed with too many possible states and details. We’ve come to understand that our yellow “caution triangle” badge can be confusing when compared to the HTTP page icon, and we believe that it is better not to emphasize the difference in security between these two states to most users. For developers and other interested users, it will still be possible to tell the difference by checking whether the URL begins with “https://”.
In the long term, we hope that most sites on the internet will become secure, and we plan to reduce the icon to just two states: secure and not secure. The change announced in this post is a small step in that direction.

HTTPS support coming to Blogspot

September 30, 2015

Posted by Jo-el van Bergen, Software Engineer, Security.2008our ads productsHTTPS Everywherehttps://www.blogger.com

mixed contentfixable by the author themselvesOfficial Google BlogGoogle Online Security BlogfeedbackHelp Center

New research: The underground market fueling for-profit abuse

September 24, 2015

Posted by Kurt Thomas and Elie Bursztein, Google Anti-Fraud and Abuse Research
hereWorkshop on the Economics of Information SecurityProfiting from abuse: a bird's eye view
Collaborating to better understand the undergroundRedefining the abuse arms race30–40% more expensive on the black marketdisrupting payment processingcutting off access to fake accountsdisabling the command and control

Improved Digital Certificate Security

September 18, 2015

Posted by Stephan Somogyi, Security & Privacy PM, and Adam Eijdenberg, Certificate Transparency PMgoogle.comwww.google.comCertificate Transparency

Disabling SSLv3 and RC4

September 17, 2015

Posted by Adam Langley, Security Engineer
As the previously announced transition to SHA-256 certificates is nearing completion, we are planning the next changes to Google’s TLS configuration. As part of those changes, we expect to disable support for SSLv3 and RC4 in the medium term.
SSLv3 has been obsolete for over 16 years and is so full of known problems that the IETF has decided that it must no longer be used. RC4 is a 28 year old cipher that has done remarkably well, but is now the subject of multiple attacks at security conferences. The IETF has decided that RC4 also warrants a statement that it too must no longer be used.
Because of these issues we expect to disable both SSLv3 and RC4 support at Google’s frontend servers and, over time, across our products in general, including Chrome, Android, our webcrawlers and our SMTP servers. (Indeed, SSLv3 support has already been removed from Chrome.) The SSL Pulse survey of the top 200,000 HTTPS sites finds that, already, 42% of sites have disabled RC4 and 65% of sites have disabled SSLv3.
If your TLS client, webserver or email server requires the use of SSLv3 or RC4 then the time to update was some years ago, but better late than never. However, note that just because you might be using RC4 today doesn’t mean that your client or website will stop working: TLS can negotiate cipher suites and problems will only occur if you don’t support anything but RC4. (Although if you’re using SSLv3 today then things will stop working when we disable it because SSLv3 is already a last resort.)
Minimum standards for TLS clients
Google's frontend servers do a lot more than terminate connections for browsers these days; there are also lots of embedded systems talking to Google using TLS. In order to reduce the amount of work that the deprecation of outdated cryptography causes, we are also announcing suggested minimum standards for TLS clients today. This applies to TLS clients in general: certainly those that are using TLS as part of HTTPS, but also, for example, SMTP servers using STARTTLS.
We can't predict the future, but devices that meet these requirements are likely to be able to continue functioning without changes to their TLS configuration up to 2020. You should expect these standards to be required in cases where Google runs certification programs, but it’s a very good idea to meet them anyway.
Devices that don’t meet these standards aren’t going to stop working anytime soon (unless they depend on RC4 or SSLv3—see above), but they might be affected by further TLS changes in the coming years.
Specifically, we are requiring:

TLS 1.2 must be supported.

A Server Name Indication (SNI) extension must be included in the handshake and must contain the domain that's being connected to.

The cipher suite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 must be supported with P-256 and uncompressed points.

At least the certificates in https://pki.google.com/roots.pem must be trusted.

Certificate handling must be able to support DNS Subject Alternative Names and those SANs may include a single wildcard as the left-most label in the name.

In order to make testing as easy as possible we have set up https://cert-test.sandbox.google.com, which requires points 1–3 to be met in order to make a successful connection. Thus, if your TLS client can’t connect to that host then you need to update your libraries or configuration.
No longer serving a cross-sign to Equifax
At the moment the certificate chains that Google properties serve most often include a cross-sign from our CA, GeoTrust, to our previous CA, Equifax. This allows clients that only trust our previous CA to continue to function. However, this cross-sign is only a transitional workaround for such clients and we will be removing it in the future. Clients that include our required set of root CAs (at https://pki.google.com/roots.pem) will not be affected, but any that don’t include the needed GeoTrust root may stop working.

Cutting unwanted ad injectors out of advertising

September 10, 2015

Posted by Vegard Johnsen, Product Manager, Google Ads Traffic Qualityhurt user experiencejeopardize user securitygenerate significant volumes of unwanted adsour researchUnwanted ad injectors: disliked by users, advertisers, and publishers

recent “Superfish” incidentRemoving injected inventory from advertising

policiesagainstNo more injected ads?

Say hello to the Enigma conference

August 18, 2015

Posted by Elie Bursztein - Anti-abuse team, Parisa Tabriz - Chrome Security and Niels Provos - Security team
USENIX Enigma is a new conference focused on security, privacy and electronic crime through the lens of emerging threats and novel attacks. The goal of this conference is to help industry, academic, and public-sector practitioners better understand the threat landscape. Enigma will have a single track of 30-minute talks that are curated by a panel of experts, featuring strong technical content with practical applications to current and emerging threats.

Google is excited to both sponsor and help USENIX build Enigma, since we share many of its core principles: transparency, openness, and cutting-edge security research. Furthermore, we are proud to provide Enigma with with engineering and design support, as well as volunteer participation in program and steering committees. The first instantiation of Enigma will be held January 25-27 in San Francisco. You can sign up for more information about the conference or propose a talk through the official conference site at http://enigma.usenix.org

New research: Comparing how security experts and non-experts stay safe online

July 23, 2015

Posted by Symposium on Usable Privacy and SecurityExperts’ and non-experts’ top 5 security practices

Common ground: careful password management

Clearly, careful password management is a priority for both groups. But, they differ on their approaches.

Security experts rely heavily on password managers, services that store and protect all of a user’s passwords in one place. Experts reported using password managers, for at least some of their accounts, three-times more frequently than non-experts.

As one expert said, “Password managers change the whole calculus because they make it possible to have both strong and unique passwords.”

On the other hand, only 24% of non-experts reported using password managers for at least some of their accounts, compared to 73% of experts. Our findings suggested this was due to lack of education about the benefits of password managers and/or a perceived lack of trust in these programs. “I try to remember my passwords because no one can hack my mind,” one non-expert told us.

Key differences: software updates and antivirus software
Despite some overlap, experts’ and non-experts’ top answers were remarkably different.
35% of experts and only 2% of non-experts said that installing software updates was one of their top security practices. Experts recognize the benefits of updates—“Patch, patch, patch,” said one expert—while non-experts not only aren’t clear on them, but are concerned about the potential risks of software updates. A non-expert told us: “I don’t know if updating software is always safe. What [if] you download malicious software?” and “Automatic software updates are not safe in my opinion, since it can be abused to update malicious content.”
Meanwhile, 42% of non-experts vs. only 7% of experts said that running antivirus software was one of the top three three things they do to stay safe online. Experts acknowledged the benefits of antivirus software, but expressed concern that it might give users a false sense of security since it’s not a bulletproof solution.

Next Steps
In the immediate term, we encourage everyone to read the full research paper, borrow experts’ top practices, and also check out our tips for keeping your information safe on Google.
More broadly, our findings highlight fundamental misunderstandings about basic online security practices. Software updates, for example, are the seatbelts of online security; they make you safer, period. And yet, many non-experts not only overlook these as a best practice, but also mistakenly worry that software updates are a security risk.
No practice on either list—expert or non-expert—makes users less secure. But, there is clearly room to improve how security best practices are prioritized and communicated to the vast majority of (non expert) users. We’re looking forward to tackling that challenge.

Working Together to Filter Automated Data-Center Traffic

July 21, 2015

Posted by Vegard Johnsen, Product Manager Google Ad Traffic Quality
Today the Trustworthy Accountability Group (TAG) announced a new pilot blacklist to protect advertisers across the industry. This blacklist comprises data-center IP addresses associated with non-human ad requests. We're happy to support this effort along with other industry leaders—Dstillery, Facebook, MediaMath, Quantcast, Rubicon Project, TubeMogul and Yahoo—and contribute our own data-center blacklist. As mentioned to Ad Age and in our recent call to action, we believe that if we work together we can raise the fraud-fighting bar for the whole industry.
Data-center traffic is one of many types of non-human or illegitimate ad traffic. The newly shared blacklist identifies web robots or “bots” that are being run in data centers but that avoid detection by the IAB/ABC International Spiders & Bots List. Well-behaved bots announce that they're bots as they surf the web by including a bot identifier in their declared User-Agent strings. The bots filtered by this new blacklist are different. They masquerade as human visitors by using User-Agent strings that are indistinguishable from those of typical web browsers.
In this post, we take a closer look at a few examples of data-center traffic to show why it’s so important to filter this traffic across the industry.
Impact of the data-center blacklist
When observing the traffic generated by the IP addresses in the newly shared blacklist, we found significantly distorted click metrics. In May of 2015 on DoubleClick Campaign Manager alone, we found the blacklist filtered 8.9% of all clicks. Without filtering these clicks from campaign metrics, advertiser click-through rates would have been incorrect and for some advertisers this error would have been very large.
Below is a plot that shows how much click-through rates in May would have been inflated across the most impacted of DoubleClick Campaign Manager’s larger advertisers.

Two examples of bad data-center traffic
There are two distinct types of invalid data-center traffic: where the intent is malicious and where the impact on advertisers is accidental. In this section we consider two interesting examples where we’ve observed traffic that was likely generated with malicious intent.
Publishers use many different strategies to increase the traffic to their sites. Unfortunately, some are willing to use any means necessary to do so. In our investigations we’ve seen instances where publishers have been running software tools in data centers to intentionally mislead advertisers with fake impressions and fake clicks.
First example
UrlSpirit is just one example of software that some unscrupulous publishers have been using to collaboratively drive automated traffic to their websites. Participating publishers install the UrlSpirit application on Windows machines and they each submit up to three URLs through the application’s interface. Submitted URLs are then distributed to other installed instances of the application, where Internet Explorer is used to automatically visit the list of target URLs. Publishers who have not installed the application can also leverage the network of installations by paying a fee.
At the end of May more than 82% of the UrlSpirit installations were being run on machines in data centers. There were more than 6,500 data-center installations of UrlSpirit, with each data-center installation running in a separate virtual machine. In aggregate, the data-center installations of UrlSpirit were generating a monthly rate of at least half a billion ad requests— an average of 2,500 fraudulent ad requests per installation per day.
Second example
HitLeap is another example of software that some publishers are using to collaboratively drive automated traffic to their websites. The software also runs on Windows machines, and each instance uses the Chromium Embedded Framework to automatically browse the websites of participating publishers—rather than using Internet Explorer.
Before publishers can use the network of installations to drive traffic to their websites, they need browsing minutes. Participating publishers earn browsing minutes by running the application on their computers. Alternatively, they can simply buy browsing minutes—with bundles starting at $9 for 10,000 minutes or up to 1,000,000 minutes for $625.
Publishers can specify as many target URLs as they like. The number of visits they receive from the network of installations is a function of how long they want the network of bots to spend on their sites. For example, ten browsing minutes will get a publisher five visits if the publisher requests two-minute visit durations.
In mid-June, at least 4,800 HitLeap installations were being run in virtual machines in data centers, with a unique IP associated with each HitLeap installation. The data-center installations of HitLeap made up 16% of the total HitLeap network, which was substantially larger than the UrlSpirit network.
In aggregate, the data-center installations of HitLeap were generating a monthly rate of at least a billion fraudulent ad requests—or an average of 1,600 ad requests per installation per day.Not only were these publishers collectively responsible for billions of automated ad requests, but their websites were also often extremely deceptive. For example, of the top ten webpages visited by HitLeap bots in June, nine of these included hidden ad slots -- meaning that not only was the traffic fake, but the ads couldn’t have been seen even if they had been legitimate human visitors.
http://vedgre.com/7/gg.html is illustrative of these nine webpages with hidden ad slots. The webpage has no visible content other than a single 300×250px ad. This visible ad is actually in a 300×250px iframe that includes two ads, the second of which is hidden. Additionally, there are also twenty-seven 0×0px hidden iframes on this page with each hidden iframe including two ad slots. In total there are fifty-five hidden ads on this page and one visible ad. Finally, the ads served on http://vedgre.com/7/gg.html appear to advertisers as though they have been served on legitimate websites like indiatimes.com, scotsman.com, autotrader.co.uk, allrecipes.com, dictionary.com and nypost.com, because the tags used on http://vedgre.com/7/gg.html to request the ad creatives have been deliberately spoofed.
An example of collateral damage
Unlike the traffic described above, there is also automated data-center traffic that impacts advertising campaigns but that hasn’t been generated for malicious purposes. An interesting example of this is an advertising competitive intelligence company that is generating a large volume of undeclared non-human traffic.
This company uses bots to scrape the web to find out which ad creatives are being served on which websites and at what scale. The company’s scrapers also click ad creatives to analyse the landing page destinations. To provide its clients with the most accurate possible intelligence, this company’s scrapers operate at extraordinary scale and they also do so without including bot identifiers in their User-Agent strings.
While the aim of this company is not to cause advertisers to pay for fake traffic, the company’s scrapers do waste advertiser spend. They not only generate non-human impressions; they also distort the metrics that advertisers use to evaluate campaign performance—in particular, click metrics. Looking at the data across DoubleClick Campaign Manager this company’s scrapers were responsible for 65% of the automated data-center clicks recorded in the month of May.
Going forward
Google has always invested to prevent this and other types of invalid traffic from entering our ad platforms. By contributing our data-center blacklist to TAG, we hope to help others in the industry protect themselves.
We’re excited by the collaborative spirit we’ve seen working with other industry leaders on this initiative. This is an important, early step toward tackling fraudulent and illegitimate inventory across the industry and we look forward to sharing more in the future. By pooling our collective efforts and working with industry bodies, we can create strong defenses against those looking to take advantage of our ecosystem. We look forward to working with the TAG Anti-fraud working group to turn this pilot program into an industry-wide tool.

Google, the Wassenaar Arrangement, and vulnerability research

July 20, 2015

Posted by Neil Martin, Export Compliance Counsel,
Cross-posted on the Google Public Policy Blog
As the usage and complexity of software grows, the importance of security research has grown with it. It’s through diligent research that we uncover and fix bugs — like Heartbleed and POODLE — that can cause serious security issues for web users around the world.
The time and effort it takes to uncover bugs is significant, and the marketplace for these vulnerabilities is competitive. That’s why we provide cash rewards for quality security research that identifies problems in our own products or proactive improvements to open-source products. We’ve paid more than $4 million to researchers from all around the world - our current Hall of Fame includes researchers from Germany, the U.S., Japan, Brazil, and more than 30 other countries.
Problematic new export controls
With the benefits of security research in mind, there has been some public head scratching and analysis around proposed export control rules put forth by the U.S. Department of Commerce that would negatively affect vulnerability research.
The Commerce Department's proposed rules stem from U.S. membership in the Wassenaar Arrangement, a multilateral export control association. Members of the Wassenaar Arrangement have agreed to control a wide range of goods, software, and information, including technologies relating to "intrusion software" (as they've defined that term).
We believe that these proposed rules, as currently written, would have a significant negative impact on the open security research community. They would also hamper our ability to defend ourselves, our users, and make the web safer. It would be a disastrous outcome if an export regulation intended to make people more secure resulted in billions of users across the globe becoming persistently less secure.
Google comments on proposed rules
Earlier today, we formally submitted comments on the proposed rules to the United States Commerce Department’s Bureau of Industry and Security (BIS). Our comments are lengthy, but we wanted to share some of the main concerns and questions that we have officially expressed to the U.S. government today:

Rules are dangerously broad and vague. The proposed rules are not feasible and would require Google to request thousands - maybe even tens of thousands - of export licenses. Since Google operates in many different countries, the controls could cover our communications about software vulnerabilities, including: emails, code review systems, bug tracking systems, instant messages - even some in-person conversations! BIS’ own FAQ states that information about a vulnerability, including its causes, wouldn’t be controlled, but we believe that it sometimes actually could be controlled information.

You should never need a license when you report a bug to get it fixed. There should be standing license exceptions for everyone when controlled information is reported back to manufacturers for the purposes of fixing a vulnerability. This would provide protection for security researchers that report vulnerabilities, exploits, or other controlled information to any manufacturer or their agent.

Global companies should be able to share information globally. If we have information about intrusion software, we should be able to share that with our engineers, no matter where they physically sit.

Clarity is crucial. We acknowledge that we have a team of lawyers here to help us out, but navigating these controls shouldn’t be that complex and confusing. If BIS is going to implement the proposed controls, we recommend providing a simple, visual flowchart for everyone to easily understand when they need a license.

These controls should be changed ASAP. The only way to fix the scope of the intrusion software controls is to do it at the annual meeting of Wassenaar Arrangement members in December 2015.

We’re committed to working with BIS to make sure that both white hat security researchers’ interests and Google users’ interests are front of mind. The proposed BIS rule for public comment is available here, and comments can also be sent directly to publiccomments@bis.doc.gov. If BIS publishes another proposed rule on intrusion software, we’ll make sure to come back and update this blog post with details.

Security Blog