A vulnerability in a Facebook server that could have led to information disclosure and command execution has been patched by the social network.
At issue was a Sentry service, which is an open-source error tracking application that helps developers monitor and fix crashes in real time. It’s written in Python with the Django framework.
SCRT Information Security researchers were examining the application, and saw that stack traces – lists of the stack of functions that are in play at the time of an error – regularly popped on the page.
“The application seemed to be unstable regarding the user password reset feature, which occasionally crashed,” the company explained in a post on the flaw.
More importantly, these stack traces were triggering full printouts of the entire application, albeit with some critical information redacted.
“Django debug mode was not turned off, which consequently prints the whole environment when a stack trace occurs,” according to the post. However, Django snips critical information (passwords, secrets, key…) in those stack traces, therefore avoiding a massive information leakage.”
Even so, the stack traces did reveal enough information pieces to build an exploit for executing commands on the system. These included, the name of the session cookie; the session serializer that uses the Pickle binary protocol to unserialize Python object structures; the session engine; and a list of Sentry configuration information containing a secret key used for session signing.
This last piece was crucial given that the key used by Django for signing session cookies is redacted from the stack trace.
“The SENTRY_OPTIONS list contains a key named system.secret-key, that is not snipped,” SCRT Information Security said. “Quoting the Sentry documentation, system.secret-key is ‘a secret key used for session signing. If this becomes compromised it’s important to regenerate it as otherwise it’s much easier to hijack user sessions’; wow, it looks like it’s a sort of Django SECRET-KEY override!”
Armed with all of the pieces, the researchers found they were able to create their own session containing arbitrary pickle content, in order to execute commands on the system. To test the mechanism, they wrote a simple script with an arbitrary payload into a specially crafted sentrysid cookie.
“This code is a simple proof of concept; it takes the content of an existing sentrysid cookie, and replaces its content with an arbitrary object that will run…when unserialized.”
The payload was a “sleep 30” command: “When using this cookie, the page actually takes an additional 30 seconds to load, which confirms the presence of the flaw,” the company added.
“It looks like Facebook failed to fully sanitize error data returned by a public facing web app,” James Lerud, head of the behavioral research team at Verodin, told Threatpost. “This is a common problem in the world of web application security…It is likely that the server involved is used for testing and is not intended to be a production asset, although without Facebook’s confirmation it is hard to say for sure.”
Facebook acknowledged the vulnerability, and took the system down the same day (July 30), while prepping a patch. It fixed the problem a little over a week later.
Researchers told Threatpost that while the vulnerability did not affect Facebook’s users directly, it’s an important reminder of the challenge of patching at enterprise scale.
“It was an information disclosure in a third-party application that Facebook was running on some of their servers that appear to be disconnected from user information,” Allan Liska, senior security architect at Recorded Future, told Threatpost. “In any service the size of Facebook there will be any number of tangential systems running older or potentially vulnerable code.”
Sherban Naum, senior vice president for corporate strategy and technology at Bromium, meanwhile told Threatpost that “the question we look to answer on the host (not server) is maintaining integrity even if a vulnerability exists. Better yet, how can one survive a successful breach while maintaining the integrity and privacy of an enterprise’s high-value assets.”
Naum added, “Not all vulnerabilities lead to critical demise of high-value assists. It looks like both the researcher and Facebook did the right thing and addressed the matter. Being diligent and responsive will make securing enterprise infrastructure (for all) better. I’m guessing they weren’t the only ones that benefited from this research. Let’s hope others took notice and updated their own systems. In reality, vulnerabilities are built in. The question is how to operate even when compromised.”