Bypassing Identity-Aware Proxy - Google Cloud Vulnerability

on

Bypassing Identity-Aware Proxy

Summary

I discovered a way, how an attacker could leak tokens from other users who are authorized to access an IAP-secured web application (Identity-Aware Proxy). This allows an attacker to hijack sessions and hence access IAP-secured web applications. 

To perform such an attack, the following steps are required:
Whenever a user who has access to the targeted application accesses the attackers IAP-secured app, a valid redirect token for the original OAuth client is sent to the attackers (second) backend service, where the redirect token can be read by the attacker.
This redirect token can be used by the attacker to access the targeted IAP-secured web application.

Target: An introduction to Identity-Aware Proxy

Identity-Aware Proxy (IAP) is a security component in Google Cloud Platform (GCP) that lets you establish a more granular, application-level access control model instead of simply relying on network-level firewall rules.

IAP can be used to secure web applications (hosted on Compute Engine VMs, App Engine, etc.). Additionally it is possible to tunnel TCP connections in general through IAP.

Either way, IAP enforces an OAuth authentication flow before access to the actual application is granted. Using the OAuth token information, granular access rules can be enforced for authorization (user identity but also context-aware parameters).

For this research, I focused on the web application part. 

For further explaining my approach, the actual vulnerability but also the reasoning behind choosing IAP as a target in the first place I want to explain the request flow for such an IAP-secured web application in more detail:



  1. A client tries to access the web application (e.g. https://my-secure-app.com/). This request goes to an External HTTPS Load Balancer. 
  2. The Load Balancer routes the request to the configured Backend Service. However, as Identity Aware Proxy is enabled for the backend service, the request is intercepted by IAP checked for a valid authorization token in the cookies.
  3. As the request did not contain a valid Token, IAP redirects the client to the Google Login for OAuth (https://accounts.google.com/o/oauth2/v2/auth?client_id=${CLIENT_ID}...), using the OAuth client ID configured for Identity Aware proxy.
  4. After successful authentication the client is redirected to the IAP API (https://iap.googleapis.com/v1/oauth/clientIds/${CLIENT_ID}:handleRedirect?...), exchanging the OAuth auth code for a redirect token.
  5. Now the client is redirected back to the initial web application, but including additional query parameters (e.g. https://my-secure-app.com/?gcp-iap-mode=AUTHENTICATING&redirect_token=...) especially the exchanged redirect token.
  6. The Load Balancer once again forwards this request to the backend service but is intercepted by IAP. IAP now notices the additional query parameters and exchanges both for a Cookie containing the final valid token (only if the authenticated user fulfills the configured authorization requirements).
  7. Finally the client is redirected to the web application and initally requested URL (e.g. https://my-secure-app.com/).
  8. The Load Balancer forwards the request to the backend service where it is once again intercepted by IAP. and checked for a valid authorization token in the cookies.
  9. As the token is valid, the request passes through IAP and reaches the backend service and hence the actual web application.
This detailed list of redirects and components involved is not only key for my further research but also illustrates several insights:
  • First of all, Identity Aware proxy is a great way for adding an additional layer of security in front of an application. Only authorized clients (in terms of identity and conext aware parameters) can reach the application network-wise. Implemented properly it can reduce the attack surface of an application tremendously.
  • In addition, the complexity is handled by Google, so it is pretty easy for Google Cloud customers to use Identity Aware proxy.
These features make IAP also a great target for bug hunting, because:
  • The amount of components and redirects involved provide quite some surface for potential attacks
  • Being a security component increases the likelihood that applications "hiding" behind IAP are sensitive
So I decided to dig further.

Approach: Digging for attack vectors

When searching for a way to access an application secured via IAP, there are 2 main categories of potential attack scenarios:
  1. Bypassing the IAP without valid credentials
  2. Bypassing the IAP with valid credentials leaked from another user
Because the first approach does not require some way of interaction with another user, this was my first area to research.

At first I tried fiddling around with the various requests involved (see figure above), changing some parameters (slightly modified or utter nonsense), sending invalid tokens (e.g. tokens for a different IAP secured application), etc. Apart from getting a good feeling for the causes of various IAP error codes, nothing really seamed interesting let alone a potential attack vector.
However, getting to know the target is always a valuable first step and you never know, which low hanging fruits you might stumble over.

After these failed attempts I tried to focus on the second approach and looked for potential attack vectors for stealing valid credentials from authorized users. It took some time until I stumbled over a "convenience feature" for Identity-Aware Proxy: Sharing OAuth Clients.

Whereas by default, each IAP-secured application creates its own OAuth Client (consisting of client id and secret), this feature allows to reduce the number of OAuth Clients created by reusing them for multiple applications.

For me, this caused a sudden shift in perspective and led to a breakthrough for bypassing IAP:
What would happen if I tried to create an IAP-secured application as the attacker, but specified the targeted OAuth client using the sharing OAuth clients feature?

Breakthrough(s): Completing the puzzle

So I gave it a try which led to a series of small steps forward, each of which created a new problem to solve and after finding various puzzle pieces I was able to construct an end to end exploit.

First I had to check, which information I had to know in order to use the Sharing OAuth Clients feature. These were Client ID and Client Secret for the OAuth client. The Client ID was easy to get, as it is included in the query parameters of the initial redirect when trying to access the target application. However, there was simply no way to get or guess the Client Secret. Either way, I decided to give it try.

First, using a "victim" Google Account and GCP project, I created a dummy target application consisting of a simple nginx webserver running on a Compute Engine VM which I added to an Instance Group. Then I created an External HTTPS Load Balancer, added the Instance Group as a new Backend Service and activated Identity-Aware proxy on the Backend Service (The automatically created OAuth client ID was: 1016913171751-u4ncjlfs09ckr8bho0praksql8uj71u2.apps.googleusercontent.com). I granted the "victim" Google Account the "IAP-secured Web App User" role, hence making this account the only one allowed to access the target application. Let's say this application is reachable under https://victim.app/.

Then, using another "attacker" Google Account and GCP project, I basically did the same thing again (simple webserver VM exposed via an External HTTPS Load Balancer), however skipped the IAP setup initially. Let's say this application is reachable under https://attacker.app/.
I then proceeded and activated IAP on the backend service, using the Client ID from the victims application. As the "attacker" had no clue about the Client Secret, I simply used a random and incorrect value (1234567890):

gcloud compute backend-services update my-attacker-backend --global --iap=enabled,oauth2-client-id=1016913171751-u4ncjlfs09ckr8bho0praksql8uj71u2.apps.googleusercontent.com,oauth2-client-secret=1234567890

Even with the incorrect secret value and the OAuth client even existing in a completely different GCP Organization, the command succeeded.
Then I tried to access https://attacker.app/ and got successfully redirected to the sign in page along with the targeted client Id (https://accounts.google.com/o/oauth2/v2/auth?client_id=1016913171751-u4ncjlfs09ckr8bho0praksql8uj71u2.apps.googleusercontent.com&...). 
After logging in I was redirected to the IAP API (https://iap.googleapis.com/v1/oauth/clientIds/1016913171751-u4ncjlfs09ckr8bho0praksql8uj71u2.apps.googleusercontent.com:handleRedirect?...) and afterwards back to https://attacker.app/ including the redirect_token as a query parameter.
However, here I recieved an IAP Error Code: Error Code 11 - Your OAuth client ID is incorrectly configured. So this was the location, where the incorrect client secret caused an issue. The redirect_token was related to the client secret and my attackers IAP configuration threw an error, because the OAuth flow was not completed using the configured client secret (1234567890).

However, I figured, that if the attacker would lure the victim to https://attacker.app/, a somewhat valid redirect_token would at least end up at the Load Balancer under the attackers control. The Error Code 11 was only thrown after the Load Balancer forwarded the request to the backend service where it was intercepted by the Identity-aware Proxy, which caused the error. Hence, if I could find a way to extract the redirect_token before it reached the Identity-aware Proxy, an attacker could steal a victim's token.

Luckily (for me) a more or less recent feature of HTTPS Load Balancers was query parameter-based routing. Using this method, the attacker could route the requests in question (containng the query parameter gcp-iap-mode=AUTHENTICATING and redirect_token) to another backend service (obviously without IAP activated so the request would actually reach the backend).

Hence, (as the attacker) I created a second backend service pointing to the same instance group but skipped the IAP configuration part entirely. I modified the Load Balancer to use the following URL Map configuration, making use of the query parameter-based routing:


Using a browser session with the victim's Google Account I navigated to https://attacker.app/ and the typical IAP redirect sequence happened (accounts.google.com, iap.googleapis.com and back to https://attacker.app/). However, due to the additional URL Map configuration, the final redirect no longer got intercepted by IAP and actually ended up on the backend (a Compute Engine Instance under the attacker's control).

This seemed like the final puzzle piece, as it would allow an attacker to simply lure a victim to a URL (which should be less suspicious than https://attacker.app/ ;) ). After a bunch of redirects without any further interactions by the victim (as long as the Google Account was already signed in in the browser session), the attacker would have extracted a valid redirect_token created with the same OAuth Client used by https://victim.app/.

Then it occured to me (probably a little late in the research), that it could make sense to check the actual contents of such a redirect_token (e.g. using https://jwt.io/ or any other way of decoding JWT tokens). And this is when my initial feeling of having found a vulnerability in IAP broke (for the moment). The decoded redirect_token contained https://attacker.app in the target_uri:



Without much hope, I crafted a request using curl  to https://victim.app/?gcp-iap-mode=AUTHENTICATING&redirect_token=${REDIRECT_TOKEN}. Basically sending the exact request (including the same headers), my attackers backend received but to https://victim.app/.
The response was a redirect (Code 302) as I feared, however, the response contained GCP_IAAP_AUTH_TOKEN and GCP_IAP_UID cookies.
In a new incognito browser session I navigated to https://victim.app/ and used the Chrome DevTools to create GCP_IAAP_AUTH_TOKEN and GCP_IAP_UID cookies to the retrieved values. A quick refresh of the page and I had access to the IAP-secured web application https://victim.app/.


Exploit: Crafting an end to end attack scenario

Finally, after finding multiple puzzle pieces an end to end scenario seemed possible. It is always worth taking a step back and put the individual pieces together step by step. Because on the one hand it might help finding flaws in the own reasoning, on the other hand additional vulnerabilities sometimes might become apparent during this process. 

In this case, the bug research was a little convoluted from my perspective and I wanted to re-ensure myself of the findings.

For an end-to-end attack scenario, an attacker would have to perform the following steps to access an IAP-secured web application:
  1. Find the OAuth Client ID for the target application (looking at the redirect parameters)
  2. Create an own GCP HTTPS Load Balancer with 2 Backend Services pointing to the same web server (could be different, but does not make a difference)
  3. Create an unsuspicious URL with a DNS record pointing to this Load Balancer. Or use services like nip.io and/or tinyurl.com to obfuscate it.
  4. Ensure, the first backend service has IAP enabled using the target OAuth Client ID and a random Client Secret (using Sharing OAuth Clients)
  5. Ensure, requests containing ?gcp-iap-mode=AUTHENTICATING&redirect_token are routed to the second, non-IAP-enabled backend service (using query parameter-based routing)
  6. On the backend server ensure that requests are properly persisted, especially the redirect_tokens. To further hide the attack, it makes sense to also redirect an unsuspicious website after storing the request data.
  7. Finally, try to lure a victim (who is allowed to access the target application) onto the prepared attackers URL. Think of any social engineering approach here. One entry point may be the admin email displayed on the error screen after a failed attempt to login to the target application (always present for IAP-secured web applications).
  8. Once a victim clicks the URL, a bunch of redirects happen in quick succession. When the browser is logged in with a Google Account permitted in the target application no user interaction is required at all. Finally the redirect progression end on the unsuspicious website, the attackers web server redirects to.
  9. During the redirects, the victims redirect_token has been captured by the attackers web server. The attacker can use this to retrieve valid session cookies from and create an authorized session for the target application.

Conclusion: Nice catch! ... and what I learned from it.

Re-ensured, that I indeed found a valid and exploitable security issue I opened a bug report with Google including the 3 security flaws, that led to this exploit when combined:
  1. OAuth Clients can be shared across GCP Organizations without further authorization checks (not knowning the OAuth Client secret is an insufficient hurdle)
  2. Query parameter based routing can be abused to intercept the IAP OAuth flow, allowing to capture intermediate states (i.e. redirect_tokens)
  3. IAP-backend services create IAAP_AUTH_TOKEN cookies even when the target_uri in the redirect_token does not match the requested hostname
Individually, these 3 issues probably do not pose significant risk but combined create an end to end attack scenario.

Brief timeline of the bug report:
May 5, 2021: Bug Report issued
May 10, 2021: Bug Report triaged
May 20, 2021: Nice catch! Bug Report accepted
June 1, 2021: The VRP panel has decided to issue a reward of 5,000$
The issue has been fixed by Google in June 2021.

Update:
June 3, 2022: This report has been selected for the GCP VRP Prize 2021 and an additional 133,337$ have been rewarded on top.

Many thanks to Google's Vulnerability Reward Program for the prompt reaction and the nice reward, very much appreciated :)

On top of the reward, I also took some additional insights from this research experience:
  • Minor flaws can add up to bigger ones. When stumbling over some minor couriosity, it is always worth digging deeper and try to find the next puzzle piece.
  • When searching for vulnerabilities, it might be worth to take a reverse approach. In this case, instead of trying to make a target application accept malicious authentication attempts, the vulnerability became apparent, when trying to make a malicious application accept valid credentials.
For me this was a fun and informative research that probably helps me in further developing both, my bughunting and my engineering skills.