Cloudbleed bug: What you need to know
Serious ‘cloudbleed bug’ in Cloudflare infrastructure leaks user data from huge range of sites...
Widely-used content delivery network (CDN) Cloudflare has suffered a serious leak that has publicly exposed user data in a security error that has been compared with the Heartbleed flaw of 2014.
The leak was caused by edge servers running past the end of a buffer in certain circumstances, and returning memory that contained private information. It was uncovered by Google Project Zero security researcher Tavis Ormandy, and included encryption keys, cookies, passwords, and HTTPS session information.
The first public indication of disaster was this tweet, which as the Twitterverse pointed out, must have made several hearts at Cloudflare sink at the sight:
According to Cloudflare, the earliest date memory could have leaked is 2016-09-22, and although their investigations found no evidence of malicious usage of the leaked information, there is a possibility, as at least some of the leaked data was cached by search engines such as Google and Yahoo.
Although it’s impossible to be clear about exactly how many sites have been affected by the leaks, which Cloudflare has of course been downplaying, the company’s scale means that not only are big name brands such as Uber, FitBit, OKCupid potentially in the frame, but by some estimates up to 5,319,353 domains could be affected.
Ilia Kolochenko, CEO High-Tech Bridge said: “Network architecture becomes quite complicated and includes many different layers from different providers, introducing new vulnerabilities and attack vectors. This is a very good example of a security flaw that may occur at the fault of a third-party, and not web application developers, who are usually blamed for data breaches.
“Without full technical details, it’s difficult to assess the practical exploitability and related risks of the vulnerability. It seems to be not very critical, but under some circumstances a set of chunks of disclosed memory may be perfectly enough to compromise the remote website and even the server. The risk is also aggravated by the large scale of impacted websites, providing attackers with a great choice of potential victims. Chances that cybercriminals had found and exploited the issue much earlier than Google exist. Therefore, all websites owners are better off changinmg all their passwords for web applications and back ends.”
The bug as described by Cloudflare relates to the way in which HTML was parsed on the fly. Cloudflare was in the process of migrating from a legacy parser to a new version, but the changes caused during the migration activated the bug.
“The engineers working on the new HTML parser had been so worried about bugs affecting our service that they had spent hours verifying that it did not contain security problems. Unfortunately, it was the ancient piece of software that contained a latent security problem and that problem only showed up as we were in the process of migrating away from it”, wrote John Graham-Cumming, CTO of Cloudflare, in a blogpost explaining the full technical detail.
Of course, with such a potentially critical bug the disclosure crowd were set for a field day, and although the conversation between Cloudflare and Tavis Ormandy seemed to be cordial enough initially, some cracks appeared around disclosure after Cloudflare’s rapid initial investigation and mitigation.
Ormandy blogged: “I asked for a draft of their announcement, but they seemed evasive about it and clearly didn't want to do that. I'm really hoping they're not planning to downplay this. If the date keeps extending, they'll reach our "7-day" policy for actively exploited attacks.” He also pointed to the top bug bounty offered by Cloudflare on their HackerOne page - a t-shirt. Unfortunately the example of Yahoo’s T-shirt gate has obviously not sunk in as widely as hoped. However, overall the incident - from Ormandy’s report to coordinated disclosure - took a pretty impressive 22.5 business hours.
Kolochenko commented: “Speaking about the Bug Bounty at Cloudflare, I’d say that a t-shirt can be a reasonable solution for all minor submissions if the company does not want to spend cash on crowd security testing, and prefers to express their gratitude differently. However, in this particular case, they should probably make an exception to their award policy and offer something much more significant (not necessarily cash).”
Perhaps Ben Hawkes, manager of the Project Zero team, had the most philosophical take:
In summary, any traffic which passed through Cloudflare (including HTTPS) in recent months might have been made public, in spite of the clean-up operation, so change those passwords/tokens/etc.
Image credit: Unsplash, Marc Wieland