Firebase functions emulator does not work with Yarn 2

[cjmyles]

[REQUIRED] Environment info

firebase-tools:
8.2.0

Platform:
macOS

Node:
10.16.0

Yarn:
2.0.0-rc.33

[REQUIRED] Test case

When generating, installing and running a fresh Typescript version of a functions project using the firebase-tools CLI tool and Yarn 2, 20 errors are produced when executing yarn serve - 3 unique:

Cannot find module 'express-serve-static-core'.
Cannot find type definition file for 'node'.
'functions' is declared but its value is never read.

I can resolve these in the following way, but the resolution causes other errors which are not curable:

1. yarn add @types/node -D
2. yarn add @types/express-serve-static-core -D
3. Modify src/index.ts to export the helloWorld function present in the comments.

This causes the following CLI output:

i  Starting emulators: ["functions"]
✔  functions: Emulator started at http://localhost:5001
i  functions: Watching "/Users/craig/dev/sandbox/firebase-functions-yarn-2" for Cloud Functions...
⚠  Error: Cannot find module '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip/node_modules/firebase-admin/package.json'
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:957:15)
    at Function.module_1.Module._resolveFilename (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12283:54)
    at Module._load (internal/modules/cjs/loader.js:840:27)
    at Function.module_1.Module._load (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12160:36)
    at Module.require (internal/modules/cjs/loader.js:1019:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:123:32
    at Generator.next (<anonymous>)
    at fulfilled (/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:4:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
⚠  Your function was killed because it raised an unhandled error.

I can confirm that the zip file '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip' contains the path 'node_modules/firebase-admin/package.json'.

I tried downgrading to firebase-tools v8.0.3 as per this issue to no avail.

I tried upgrading Yarn to use the latest sources version to no avail.

I tried using a newer version of Node (v12.16.3) to no avail.

I tried using a newer version of tslint (v6.1.2) to no avail.

Could be an issue with the PnP resolver (https://classic.yarnpkg.com/en/docs/pnp/troubleshooting)? Although I tried running the emulator using the following command, and the error still prevailed.

NODE_OPTIONS="--require ./.pnp.js" firebase emulators:start --only functions

I've created a repository containing the generated project so it's as minimal as possible: https://bitbucket.org/cjmyles/firebase-functions-yarn-2/src. I've included firebase-debug.log as well as the Yarn 2 Zero Install files.

[REQUIRED] Steps to reproduce

1. Install Yarn 2.
2. Execute firebase init functions from the CLI - choose a Firebase project; Typescript (Y); TSLint (Y); install dependencies (N).
3. Execute yarn to install dependencies.
4. Execute yarn serve to see the first set of errors.

[REQUIRED] Expected behavior

Generated project should execute without issues, same as if the project's dependencies were installed and the project executed using Yarn 1.

[REQUIRED] Actual behavior

See test case above.

[samtstern]

Thank you @cjmyles for the really detailed feedback. This is all way over my level of JS knowledge, I'm hoping @abeisgoat can chime in.

[cjmyles]

Thanks @samtstern. Interestingly, as soon as I add express to the mix (as per this basic example), I get another error:

../../.yarn/$$virtual/firebase-functions-virtual-bbad2b487f/0/cache/firebase-functions-npm-3.6.1-9aae15f59c-3.zip/node_modules/firebase-functions/lib/handler-builder.d.ts:21:123 - error TS2307: Cannot find module 'express-serve-static-core'.

Running yarn add @types/express @types/express-serve-static-core -D doesn't fix the issue.

So a few Yarn 2 related issues here. I'd like to use Yarn 2 as it resolves transitive dependencies correctly (I have a React/Functions monorepo for example using Yarn workspaces, which references "core" packages on my machine and Yarn 1 has link issues with libraries such as React which need to exist singularly but still be referenced in both), it's a lot quicker than Yarn 1 and offers some other nice features. I'd imagine a lot of people will be migrating over.

[cjmyles]

@samtstern @abeisgoat Any ideas?

[abeisgoat]

Will take a look this week and report back. I'm not familiar with the differences between yarn and yarn@2 so it'll take a moment I'm sure

[cjmyles]

@abeisgoat did you get a chance to take a look? Yarn 2 is a life saver for me in terms of the transitive dependency resolution feature - I have a core React library that can be imported into multiple other React projects no problem, all running within Workspaces. The missing piece of the jigsaw is getting Firebase to run properly in the same setup, so any help would be greatly appreciated.

[cjmyles]

@samtstern @abeisgoat any update on this? Or is Yarn 2 something that shouldn't be used with Firebase?

[samtstern]

@cjmyles I'll take a look at this today. We do a lot of manual module resolution in the emulator and I expect that PNP is tripping us up.

[samtstern]

Yikes PNP is doing some really nasty stuff, like patching fs so it looks inside ZIPs:

https://yarnpkg.com/features/pnp#packages-are-stored-inside-zip-archives-how-can-i-access-their-files

[samtstern]

Ok I got realllllly far with this but yarn got in my way at the very end:
yarnpkg/berry#1467

I have a few workarounds possible but none are great so I would like for the Yarn2 team to get back to me first.

[samtstern]

@cjmyles I have a work in progress here:
#2356

Could you try this:

1. Clone that branch
2. Run npm install && npm build && npm link to install my branch of the Firebase CLI as the firebase command on your machine
3. See if you're able to load your Yarn2 project in the emulator?

[samtstern]

After a bunch of discussion in #2356 we realized that there's no safe way for us to support Yarn2 with PnP in the local emulators that won't break in production. But we're going to add a helpful warning to make this less confusing.

[cjmyles]

@samtstern Apologies for the radio silence. Work commitments got the better of me. Thanks for debugging the issue and the work on the PRs, especially adding the warning, it's greatly appreciated. I'm tempted to try a Lerna workspace with Yarn 1 or NPM to see if it fits the setup I need and avoids the issues with Yarn 2.

[ericvera]

FYI, In case someone else ends up here to make the emulator run, instead of running firebase directly from the command line run it as a yarn command. After some reading, it seems yarn takes care of virtualizing node_modules if necessary.

Two things are needed:

1. Latest yarn yarn set version from sources

2. In package.json

{
  ...
  "scripts": {
    ...
    "start": "firebase emulators:start --project project-name"
    ...
  },
  "dependencies": {
    ...
    "exegesis": "^2.5.6",
    "firebase-tools": "^9.6.1",
    "typescript": "^4.2.3",
    "winston-transport": "^4.4.0"
  },
}

[aaronncfca]

@ericvera I did end up here trying to make the emulator run, but I was already using the yarn serve command, which per the initial package.json resolves to firebase emulators:start --only functions. My yarn version is 3.2.1. Still, yarn serve produces the error:

Error: Error parsing triggers: Cannot find module 'firebase-functions'

Is it possible that new versions of yarn/Firebase CLI no longer work with this? Or perhaps it's just because my code is set up in a monorepo? (I have tried with and without setting installConfig.hoistingLimits as shown below, to keep yarn from hoisting packages out of the functions directory up to the root).

In any case, for me (specifically a yarn monorepo with yarn 3.2.1) I was only able to proceed by disabling Yarn PnP for the functions workspace (setting nodeLinker: node-modules in the functions/.yarnrc.yml file), and I set the following in functions/package.json to keep it from hoisting anything to the root directory:

 "installConfig": {
    "hoistingLimits": "workspaces"
  }

[yuchenshi]

@aaronncfca Would you mind opening a new issue with more details?

[ericvera]

@aaronncfca

If you are using yarn PnP checkout #4275. I did not have the time to create a smaller repro so that issue it was closed. It seems to be specific to the functions emulator (auth and Firestore emulators run fine using yarn commands). I was able to get around the issue by installing firebase-tools globally using npm and running outside of yarn. Not ideal, but works.