Firefox: Default Content Process DACL Sandbox Escape
Platform: Firefox 73.0.1 and 75.0a1, 64-bit on Windows 10 1809/1909 (not tested earlier OS versions).
Class: Elevation of Privilege

Summary: The Firefox content processes do not sufficiently lockdown access control which can result in a sandbox escape.

Description:

According to https://wiki.mozilla.org/Security/Sandbox the current content process sandbox is Level 5 which runs with Chromium token level of USER_LIMITED. However, USER_LIMITED is sufficient to open any of the other content processes running at the same time.

The ability to open another process presents a problem, the initial token is set to USER_RESTRICTED_SAME_ACCESS. If a content process is compromised via an RCE then an attack waits for a new content process to be created (there’s also likely a way of making a new content process start programmatically) the process can be opened before the initial thread token is dropped with the call to LowerToken and the attack can run as USER_RESTRICTED_SAME_ACCESS.

Even though the content process runs as a Low Integrity Level this access is sufficient to escape the sandbox. There’s likely to be multiple routes to escape but one which works that I’ve provided as a POC is as follows:

1. Wait for a new content process to start. Immediately suspend the process.
2. Impersonate the main thread’s initial token to elevate privileges.
3. Create an out-of-process COM server using the privileged identity to escape the process Job object. This also bypasses the child process mitigation as that’s set on the primary token, not the impersonation token.
4. Migrate to the new COM server process, abuse a UAC “feature” documented at https://www.tiraniddo.dev/2019/02/accessing-access-tokens-for-uiaccess.html which allows us to create a new process with UI Access privilege under our control.
5. Use UI Access to bypass UIPI and programmatically access the Explorer Run Dialog to spawn a process outside of the sandbox.

Fixing wise, you should be able to call the Chromium sandbox target policy function SetLockdownDefaultDacl to restrict the access between content processes. Note that it’s possible this will cause some system code to fail, you’ll need to do extensive testing to make sure it’s not a problem.

On a related topic I’d also recommend calling:
sandbox::ApplyProcessMitigationsToCurrentProcess(sandbox::MITIGATION_HARDEN_TOKEN_IL_POLICY).

In the main browser process. This prevents related token stealing attacks, which while mostly mitigated in Windows could still pose a risk. This can be done in slow time, it doesn’t have to be part of this fix.

Proof of Concept:

I’ve provided a PoC as a Visual Studio 2019 project. It contains a C# DLL injector and a C++ DLL which will be injected into a running copy of Firefox. The exploit will send details to the debug output which you can capture using Sysinternals Debug View.

Note, I’ve tested this on the latest stable as well as Nightly. It seems at least in Nightly it hits a release assert at https://dxr.mozilla.org/mozilla-central/source/ipc/glue/Transport_win.h#68 if debugging, continuing the execution the exploit still works. I think this assert is probably due to the process suspension, though it’s not entirely clear. I’m of the opinion that it doesn’t block the exploit, and is not really related.

1. Compile the VS2019 solution project as Release build, x64. The C# It will need to grab the NtApiDotNet library from NuGet to work.
2. Copy the DLL from x64\Release and InjectDll.exe and NtApiDotNet.dll from InjectDll\bin\Release to a generally accessible directory (say C:\Test).
3. Start a copy of Firefox running.
4. From a command line run InjectDll.exe FirefoxSandboxEscape.dll. Check that prints a PID, this is the process the code has injected into which should be a sandboxed content process.
5. In firefox create a new tab and navigate to a new web page. This should cause a new content process to be created for the exploit to hijack. A command prompt running at medium should appear.

Expected Result:
Code running in one content process should not be able to open other content processes.

Observed Result:
Content process is opened, initial thread token repurposes and sandbox escaped.

This bug is subject to a 90 day disclosure deadline. After 90 days elapse,
the bug report will become visible to the public. The scheduled disclosure
date is 2020-05-28. Disclosure at an earlier date is also possible if
agreed upon by all parties.

This looks related to bug 1554110, is that right Bob?

(In reply to :Gijs (he/him) from comment #1)

This looks related to bug 1554110, is that right Bob?

Yes, that describes the fundamental issue and our problems with deploying that fix.
The only outstanding issue that we know about is a performance one.
There is another (I believe DirectWrite) performance issue when we have acceleration enabled, but not webrender.

James however gives a much more extensive account of how this can be abused - thanks for the detailed report.
Our original concern was access to other running content processes with different trust levels from the parent (or indeed privileges in the case of the file content process).
The ease of complete escape might mean we need to swallow those performance problems though.

Would it be possible to be added to issue 1554110 to see the discussion, or does it contain other sensitive information? I'm investigating possible better fixes at the Chromium repo level so understanding of issues would be useful.

(In reply to James Forshaw from comment #3)

Would it be possible to be added to issue 1554110 to see the discussion, or does it contain other sensitive information? I'm investigating possible better fixes at the Chromium repo level so understanding of issues would be useful.

Bob? :-)

Added.

Thanks for adding me on to the other bug. I'm currently working on a change to Chromium. In summary in entails:

• Generating a random SID and adding that as a new restricted SID.
• Added the random SID granting full access into the default DACL.
• Use SetLockdownDefaultDacl to remove the other DACL access.

This should provide isolation of processes, threads and their resources from each other running at the same level as long as the each process gets a random SID. However the process will be able to access itself, this should hopefully fix issues with DirectWrite cache, although of course that will get broken at some point if you decide to drop to a lower privilege. I've no idea on the impact of the audio service though. Enabling this will still be optional, it won't be applied to everything just in case of breakage.

It won't stop a process opening another at a similar level which doesn't employ these tricks, so for example an IE PM process running at the same time would still be accessible, but there's not much you can do about that other than running in a more restrictive sandbox.

I'll let you know when I having something in the Chromium repo to test. Of course if you have other ideas it'd be good to know.

(In reply to James Forshaw from comment #6)
...

I'll let you know when I having something in the Chromium repo to test. Of course if you have other ideas it'd be good to know.

Thanks James, no other ideas at the moment, but I'm continuing to try and track down the other performance regressions.

Sounds like that will definitely render my OpenProcessToken fix unnecessary, hopefully it will sort the other performance regressions as well.
We were already planning to break the DirectWrite cache deliberately at some point assuming that we would hit the same reconnection issue that you had, but we'll probably only do that when we have the accelerated rendering moved to the GPU process.

[Tracking Requested - why for this release]: Public disclosure date of 2020-05-28, which looks like will happen when 76 is in release.

[Tracking Requested - why for this release]:
The deadline per comment #8 would mean that, in principle, we'd need to figure something out for ESR. It's unclear to me what the feasibility of that would be.

I've implemented the new sandbox feature in Chromium. https://chromium.googlesource.com/chromium/src.git/+/c1ce57ea5d31208af589b4839390a44ab20b0c8f.

I don't know what you're process is to merge in updates. To enable you'd need to call both SetLockdownDefaultDacl() and AddRestrictingRandomSid() on the sandbox policy.

(In reply to James Forshaw from comment #10)

I've implemented the new sandbox feature in Chromium. https://chromium.googlesource.com/chromium/src.git/+/c1ce57ea5d31208af589b4839390a44ab20b0c8f.

I don't know what you're process is to merge in updates. To enable you'd need to call both SetLockdownDefaultDacl() and AddRestrictingRandomSid() on the sandbox policy.

Thanks for the speedy implementation.
We normally take full updates, with some shimmed files to reduce dependency creep.
We also have a set of our own patches for various reasons.

In cases like these, I'll try and merge in just that update, we're not too far behind, so hopefully it won't be too painful.

Local testing with James's fix looks good, thanks again for this!

• Audio issue appears to be fixed on Windows 7 even without cubeb sandbox (remote audio) enabled
• Gecko profiler issue with stacks fixed
• Performance issues appear to be fixed, although it's a little difficult to be sure of this with local runs. Given that I tried to land the SetLockdownDefaultDacl fix on bug 1557282 (an open bug) last year and I have pushed try pushes with various attempts to fix performance issues I didn't think pushing this fix to try added any new risk. Performance issues appear to be fixed:
  https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=291846719&revision=3576fb045d820f0fcb977a916d7e6391d6826d24

As far as I can see, we'll need to add these settings to all of our sandboxed processes, because if a way can be found to cause them to start (or restart) then they would also be initially running at low integrity and with a permissive access token. So, I think I'm going to have to push a full try push with those changes added, to get some test coverage. My feeling is that if the content process issues are fixed, we have a reasonable chance it won't cause issues in other processes.

One concern I have is when firefox is being run from a network drive.
For legacy reasons a number of people expect this to work.
When we detect this we stop using restricting SIDs in the access tokens, instead just relying on deny only settings.
For some reason, we never got to the bottom of, restricting SIDs don't seem to work for certain things when accessing a network drive.
As the random SID is only added as a restricting SID, it's obviously not there when we're not using them.

James - in the case above is there an easy way (with existing sandbox code or otherwise) to add the random SID to the Access Token as a normal SID, so we still keep access to our own process?

Unfortunately it's not really possible to add arbitrary groups, at least not without running with SeTcbPrivilege which basically requires you to be running a system service. The problem with UNC paths is by default Windows blocks access to network resources from restricted tokens, regardless of how much access those tokens have. i.e. even if you add every SID restricted as you have in the list of groups it won't. I believe there's a system setting you could change to allow this to work but of course that'd have to be enabled by the user.

From a unprivileged user process really the only possibility would be to disable the sandbox in your current implementation. There might be a theoretical possibility to use AppContainers on modern Windows 10 versions (which is supported in the Chromium sandbox) but that's fraught with difficult as well.

Could you try the POC I sent to ensure it breaks with the new sandbox changes? I've don't manual analysis in Chrome but not full end to end testing in FF.

(In reply to James Forshaw from comment #13)

Unfortunately it's not really possible to add arbitrary groups, at least not without running with SeTcbPrivilege which basically requires you to be running a system service. The problem with UNC paths is by default Windows blocks access to network resources from restricted tokens, regardless of how much access those tokens have. i.e. even if you add every SID restricted as you have in the list of groups it won't. I believe there's a system setting you could change to allow this to work but of course that'd have to be enabled by the user.

From a unprivileged user process really the only possibility would be to disable the sandbox in your current implementation. There might be a theoretical possibility to use AppContainers on modern Windows 10 versions (which is supported in the Chromium sandbox) but that's fraught with difficult as well.

Thanks, I guess we have two other choices then, dveditz, tjr - what do you think?:

• Don't lockdown the dacl when running from a network drive, even though that fundamentally weakens the sandbox.
• Land the other fix for the gecko profiler stack issue and just leave the performance problems when we're running from a network drive. Of course there is a chance this will cause other currently unknown issues, but maybe that is better than having a known effectively unsandboxed browser.

Could you try the POC I sent to ensure it breaks with the new sandbox changes? I've don't manual analysis in Chrome but not full end to end testing in FF.

I've compiled up the POC in VS2017 and checked that it was working without the fix.
With the fix the injected into process crashes, but the medium integrity command prompt doesn't appear.

You can still apply the DACL lockdown, but without the restricting SIDs it won't have any effect. The fix I implemented relies on both a token being sufficiently restricted and the potential target processes running with a restricted DACL. If you've effectively turned off the Restricted SIDs then you'll no longer prevent other processes from opening each other.

As for the POC I'm not sure it should crash, but I guess I've not actually checked it running. Is there a way of getting a test build so I can verify myself? Alternatively I can provide some instruction on using some of my analysis tools in PowerShell which you can use to verify process access.

(In reply to James Forshaw from comment #15)

You can still apply the DACL lockdown, but without the restricting SIDs it won't have any effect. The fix I implemented relies on both a token being sufficiently restricted and the potential target processes running with a restricted DACL. If you've effectively turned off the Restricted SIDs then you'll no longer prevent other processes from opening each other.

Ah, I guess the User's SID isn't set to deny only.

As for the POC I'm not sure it should crash, but I guess I've not actually checked it running. Is there a way of getting a test build so I can verify myself? Alternatively I can provide some instruction on using some of my analysis tools in PowerShell which you can use to verify process access.

You could use the build from the try push:
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IKBzjJx4ReCaH6BkeHZ5Tg/runs/0/artifacts/public/build/install/sea/target.installer.exe

That only has the fix enabled for content processes and I think your injection could pick up any child process because of the confusing -contentproc arg (which I think actually means we're using firefox.exe and not plugin-container.exe or something), if you also added command contains or possibly ends with "tab", then you should be guaranteed to get a content process.

(In reply to Bob Owen (:bobowen) from comment #16)

(In reply to James Forshaw from comment #15)

You can still apply the DACL lockdown, but without the restricting SIDs it won't have any effect. The fix I implemented relies on both a token being sufficiently restricted and the potential target processes running with a restricted DACL. If you've effectively turned off the Restricted SIDs then you'll no longer prevent other processes from opening each other.

Ah, I guess the User's SID isn't set to deny only.

I tried this by flipping the flag to always not use restricting SIDs and sure enough the fix no longer works (the exploit starts working again, I had to tweak it a bit to inject into processes without a restricted token).
However, I added into the USER_LIMITED code when not using restricted SIDs: restricted_token.AddUserSidForDenyOnly(); and that was enough for it to block the exploit again.

I'm not sure what else that might break, in theory for the normal case we don't have the User's SID because it's not in the restricted set, but Windows permissions can be unpredictable in some cases.

I wouldn't recommend blocking the user SID through a deny only, at least if you want anything to work correctly. Certainly opening the process would almost certainly fail, although it'll seem to work from in-process because there's a sandbox hook to handle it. The big issue with system services comes down to making a call to a service, that service impersonates the caller then tries to open the calling process which fails.

(In reply to James Forshaw from comment #18)

I wouldn't recommend blocking the user SID through a deny only, at least if you want anything to work correctly. Certainly opening the process would almost certainly fail, although it'll seem to work from in-process because there's a sandbox hook to handle it. The big issue with system services comes down to making a call to a service, that service impersonates the caller then tries to open the calling process which fails.

Right, we're back to the same regressions as before, one of which I have a fix for and the other performance ones caused presumably by the mechanism you describe.
However, having a performance hit only when running from a network drive, is probably better than having a totally compromised sandbox in that case. We may of course find other things that are broken in other ways than just performance.

(In reply to Bob Owen (:bobowen) from comment #19)

However, having a performance hit only when running from a network drive, is probably better than having a totally compromised sandbox in that case. We may of course find other things that are broken in other ways than just performance.

Agreed.

I've tested running from a network drive with the User's SID set to deny only and that does stop the POC.
Things seemed to work OK.
I tried things that run the various different child processes.
It was slow, but that mainly seemed to be down to running from the network rather than this change.

So, I propose to land these patches using open bug 1557282, given that has already had related traffic on it.
Two try pushes:
https://treeherder.mozilla.org/#/jobs?repo=try&duplicate_jobs=visible&selectedJob=291996630&revision=29d18576ca967f5ec79c491beed287f008cc5f3d
https://treeherder.mozilla.org/#/jobs?repo=try&duplicate_jobs=visible&selectedJob=292000034&revision=319875f253dde422da5c154e8815bbb0efa52c29

The first shows that adding this to the socket process breaks tests, the second is without the settings on the socket process.
I don't think we need to worry about that immediately, because:

• the socket process isn't shipping just yet (although it is soon)
• it starts before any other child processes and as far as I can tell currently doesn't attempt to restart, if that changes, which I guess it may well, then we'll need to work out what the settings are breaking

Not adding the rules to the socket process, has the added bonus of meaning all three patches apply to Beta and Release cleanly.
Unfortunately not ESR68 though, I'm having trouble getting that to build locally at the moment.
I've reproduced with the POC on a Beta and Release build and tested it was blocked with the new settings.

I'll upload the proposed patches to this bug first.
CCing handyman and dmajor as a heads up for the reviews on bug 1557282.

This adds AddRestrictingRandomSid feature, which fixes our issues with
SetLockdownDefaultDacl, apart from when we are running from a network drive.

When we are running from a network drive the new feature in part 1 doesn't work.
So this uses DuplicateHandle instead of OpenThread to get the thread handle used
by the profiler.
It also removes a DuplicateHandle THREAD_ALL_ACCESS call that also fails and a
DuplicateHandle to get a real process handle, which only seems to have been to
fix something on Windows XP.
The handle passed in is always the profiler one, so already has the necessary
permissions. If no thread handle is passed then the pseudo handle is used.

Finally won my fight with the ESR68 build, patches incoming.
I can see the permissions being set properly, but even without them the POC doesn't seem to work for some reason.

I worked out what was going wrong in the POC for esr68.
In RunExploit HANDLE proc was uninitialised and it was then picking up a terminated process and trying to suspend it.
Why this only caused a problem for esr68, I'm not sure.
When I initialised it to nullptr the POC started working and is blocked with the ESR68 patches attached.

(In reply to Bob Owen (:bobowen) from comment #21)
...

The first shows that adding this to the socket process breaks tests, the second is without the settings on the socket process.
I don't think we need to worry about that immediately, because:

• the socket process isn't shipping just yet (although it is soon)
• it starts before any other child processes and as far as I can tell currently doesn't attempt to restart, if that changes, which I guess it may well, then we'll need to work out what the settings are breaking

I also realised this morning that this is wrong, because we ought to also worry about already running compromised low integrity processes (another firefox instance for example or chrome GPU process (much less likely to be compromised I know)). If firefox with the socket process is then opened the exploit works as the socket process initialises.

One way to break this exploit at least is to run the socket process at USER_LIMITED from the start, this seems to break the creation of the out-of-process COM server I think.
The media socket process tests still work locally.
One other idea I had was to start the socket process at medium integrity not low (to prevent it being opened), before dropping to untrusted, but this breaks the tests.

I'll try the USER_LIMITED change on try with the socket process tests and update patch part 3 later, as I see no reason not to do this if it doesn't break anything.

James - do you think starting the socket process without the locked down dacl but at USER_LIMITED from the start is enough of a mitigation here or do we need to work out why the dacl lockdown is breaking things? (My guess is that it is to do with socket opens, but I haven't proved that.)

Try push with USER_LIMITED from start of socket process. Just socket process tests (spi):
https://treeherder.mozilla.org/#/jobs?repo=try&duplicate_jobs=visible&revision=903e6f3b1b05932defce0be4025c016795b743a7

Try push with the socket process using USER_LIMITED from the start looks good.
I've decided to upload this as a fourth part, because it wouldn't need to be uplifted as the socket process is only enabled on Nightly at the moment.

By the way, I've tried to get everything ready for landing because I will have some time today and tomorrow to work on this, but I'm then on PTO from Monday.

Sorry for the buggy POC.

The fix is all best efforts IMO. At the current FF's content sandbox level (and Chromium GPU) I'm sure they're probably still a way out given enough time and effort spent. Chromium has the advantage that finding a GPU RCE is slightly more difficult than finding a renderer sandbox escape so that's the only reason it's less of a concern (however I did enable the lockdown in Chromium on Friday).

I think the biggest risk is the initial token, if you can get it working with USER_LIMITED then that's a good idea. Still would be interesting to find out why your tests are failing as it might identity some new misfeature in Windows which needs to be taken into consideration in the future.

(In reply to James Forshaw from comment #33)

Sorry for the buggy POC.

The fix is all best efforts IMO. At the current FF's content sandbox level (and Chromium GPU) I'm sure they're probably still a way out given enough time and effort spent. Chromium has the advantage that finding a GPU RCE is slightly more difficult than finding a renderer sandbox escape so that's the only reason it's less of a concern (however I did enable the lockdown in Chromium on Friday).

I think the biggest risk is the initial token, if you can get it working with USER_LIMITED then that's a good idea. Still would be interesting to find out why your tests are failing as it might identity some new misfeature in Windows which needs to be taken into consideration in the future.

Thanks, looks like we can use USER_LIMITED from the start for the socket process.
Hopefully when I get back I can find out what is broken in the socket process with the locked down dacl.

I don't think uplifting to beta 75 is realistic if Bob's out until March 23, but we'll see how rough it goes trying to land on nightly in bug 1557282.

Bug 1557282 landed on central.

James, what's the Chrome disclosure timeline on this? If we ship the fix in Firefox 76 (scheduled for 2020-05-05), is that early enough? (It should be per the timeline in comment 0, just want to make sure that still stands.)

Disclosure wise the Project zero issue will be opened on 2020-05-28 as per the original report. We recently changed our policy and you get the whole 90 days from reporting date before the issues are disclosed. 2020-05-05 should therefore more than meet schedule.

From the Chrome (non-P0) perspective the fixes are merged into Canary/Dev but the issue is not public and won't be for a similar time period. However the fixes don't make any reference to FF in any way.

From discussion, there's an additional complication: for ESR 68, when we're on a network drive and use the fallback fix, we can't rely on AudioIPC to fix the audio issues. So that configuration might end up with broken audio. Not sure how common that configuration is, but Enterprise environments might be exactly the situation where running from a network drive happens. Blergh!

(In reply to Gian-Carlo Pascutto [:gcp] from comment #39)

From discussion, there's an additional complication: for ESR 68, when we're on a network drive and use the fallback fix, we can't rely on AudioIPC to fix the audio issues. So that configuration might end up with broken audio. Not sure how common that configuration is, but Enterprise environments might be exactly the situation where running from a network drive happens. Blergh!

I've tested running ESR68 from a network drive with the patches applied on two windows 10 installations and audio appears to be working OK.
Given bug 1567662, from the last time I tried to land this (without the new chromium fix), I suspect that there will be some cases where it is broken.

On windows 7 ESR68 from a network drive had broken audio.
It appeared to be working fine when running locally.

RyanVM reminded me that this isn't in ESR68 yet.
The patches landed using publicly open bug 1557282.

The fixes for this don't seem to have caused any issues on Nightly or Beta Fx76 as far as I know.

We know that we will get some problems with broken audio on ESR68 when running from a network drive (Firefox binaries installed on a network share).
That's because we don't have the benefit of cubeb remoting there.
The preferred fix for this, if people can't live with broken audio, is to install Firefox locally on the machine or possibly use Release version if the network installation is a must.

Is everyone happy that we should still take this on ESR68 given the security implications?
If we are, then I'll get the patches updated with any further rebasing / review comments and get them uploaded to bug 1557282 for uplift approval.

That's fine with me. And worst case on ESR, can't they change the sandbox value?

(In reply to Mike Kaply [:mkaply] from comment #42)

That's fine with me. And worst case on ESR, can't they change the sandbox value?

They can, but I don't think we should recommend that as a solution.

Fine with me -- I prefer to fix the security bug

The blog is not about this issue. I identified this issue while implementing the exploit chain for the Windows kernel bug in that blog post. I originally targeted FF's Content Sandbox, but the blog post I only briefly mention FF and focus on Chrome to avoid people looking too closely. The P0 issue for this vulnerability is still hidden and within the 90 days of disclosure.

Ah, sorry. I haven't had a chance to read the blog post yet. Thanks for the correction.

Hopefully with ESR 78 (or before on normal release) we will gain the ability to do audio remotely on Windows. So yes lets fix the security issue.

Fine with me

Thanks everyone, patches are now up on bug 1557282 and ESR68 uplift approval requested.

No point in keeping it closed after P0 releases their report. Opening up as per conversation with Dan.

Project Zero blog post: https://googleprojectzero.blogspot.com/2020/06/ff-sandbox-escape-cve-2020-12388.html