Firebase-tools: Could not read source directory. Remove links and shortcuts and try again.

[ahaverty]

Version info

firebase-tools: 3.17.4

Platform Information

OS X

Steps to reproduce

Deploy a firebase function as normal
firebase deploy --only functions:myFunction

Expected behavior

Function is deployed or a descriptive error is displayed

Actual behavior

I received:

✔  functions: Finished running predeploy script.
i  functions: ensuring necessary APIs are enabled...

Error: Server Error. getaddrinfo ENOTFOUND servicemanagement.googleapis.com servicemanagement.googleapis.com:443

However, I kept re-attempting a few times without making changes and also received the following about ~5 times:

Error: Could not read source directory. Remove links and shortcuts and try again.

The "Could not read source directory" error also sometimes took up to 2 minutes to appear after executing the command, which leads me to believe its network related and nothing to do with my local setup, especially as the function eventually deployed after a few re-attempts.

[laurenzlong]

Seems like a network issue with servicemanagement.googleapis.com, which is the Google service for checking and enabling APIs on the project, unfortunately there's not much we can do from the tools perspective. If you see issues like this persistently, please file a support ticket at firebase.google.com/support.

[ahaverty]

Thanks @laurenzlong
I opened here because of the second error

Error: Could not read source directory. Remove links and shortcuts and try again.

Could firebase-tools add a description to the second error saying that it may be network related?
It currently reads like it's a local non-network issue about the local src directory.

[laurenzlong]

Thanks for clarifying. I think that error message is unrelated to the network issues, since it comes from this line: https://github.com/firebase/firebase-tools/blob/master/lib/prepareFunctionsUpload.js#L64. So it happens when the zipping process encounters an issue. Can you run the deploy command again with the --debug flag? That'll give us more info, and tell us if the error message is wrong.

[ahaverty]

Thanks @laurenzlong
I just got the same error again today, and retrying straight away without editing files worked.
I was looking into that prepareFunctionsUpload.js and saw the error log is there twice. Could the second error be network related? https://github.com/firebase/firebase-tools/blob/master/lib/prepareFunctionsUpload.js#L90

I'll try catch it with --debug next time!

[laurenzlong]

I don't believe that error can happen form network issues, but --debug would capture the original error message, so would be helpful to get your logs next time you encounter this :)

[ahaverty]

Hi @laurenzlong, I'm still intermittently hitting this unfortunately. I've sent my --debug log via support form. Thanks!

[ahaverty]

Thanks @laurenzlong,
I hope it ok that I copy your email into here:

---

I traced the error through the code, it came from the Archiver module, which we use to zip up the source code for functions prior to development. It came from this line: https://github.com/archiverjs/node-archiver/blob/master/lib/core.js#L576, and the description of the error is here: https://github.com/archiverjs/node-archiver/blob/master/lib/error.js#L22. So it seems like something in your functions directory could not be converted to a Stream or Buffer. I have some theories:

• Are you running anti virus software? Or perhaps a software that periodically scans your disk and backs it up? That can sometimes interfere with file operations, which may explain the intermittent nature of your error

- Are you using any non-traditional file formats?

I'm not using anything like antivirus or backups, but I am using typescript and tsc watch. Could it be me editing code while in mid firebase deploy? At what stage is it safe to touch code if so 🤔

[laurenzlong]

I think it might be the tsc watch that's interferring with deployment, since it scans all the files. I would stop tsc watch while you are deploying

[kiril]

This error message is almost definitely a red herring - I get it intermittently whenever there are network issues talking to the firebase servers (which seems to be ALL THE TIME recently).

[imjared]

For any future weary Google searchers, I get this issue when connected to my VPN (TunnelBear). If I turn the VPN off, the deploy works just fine.

[manmanmaaaaan]

Still an issue for me. Here is what I get with debug turned on.
Also I only get this error when I do firebase use production deploying to other environments works.

HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 12 Apr 2019 14:56:01 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,44,43,39", accept-ranges=none, transfer-encoding=chunked
i  functions: preparing functions directory for uploading...
 >>> HTTP REQUEST GET https://runtimeconfig.googleapis.com/v1beta1/projects/ABCXYZ/configs  

<<< HTTP RESPONSE 403 vary=X-Origin, Referer, Origin,Accept-Encoding, content-type=application/json; charset=UTF-8, date=Fri, 12 Apr 2019 14:56:01 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,44,43,39", accept-ranges=none, transfer-encoding=chunked
 <<< HTTP RESPONSE BODY code=403, message=The caller does not have permission, status=PERMISSION_DENIED
 FirebaseError: HTTP Error: 403, The caller does not have permission
 Error: input source must be valid Stream or Buffer instance
    at Archiver.append (/usr/local/lib/node_modules/firebase-tools/node_modules/archiver/lib/core.js:578:24)
    at /usr/local/lib/node_modules/firebase-tools/lib/prepareFunctionsUpload.js:64:17

[ajpocus]

In case anyone else runs into this, I had a temporary Emacs file in the functions directory that was causing this error. The --debug flag helped me find it. :)

[divyesm]

I m getting the same issue , the debug keeps same error either while preparing functions folder to deploy or while deploy the only function I m trying to deploy . The debug log is not even telling me the file in which it is getting the mentioned error. Tried everything. Please help me resolve this issue , it is blocking my release for 2 days.

[bkendall]

@divyesm please open a new issue with the appropriate information and we can try to help debug it. Include as much of the --debug output or firebase-debug.log file as possible. Thanks.

[alexlouden]

I had the same issue, so I re-ran my deploy with --debug and discovered that my deploy user was getting a 403 on the GET https://runtimeconfig.googleapis.com/v1beta1/projects/myproject/configs call.

I added the Cloud RuntimeConfig Admin role, and a handful of permissions including these, and that resolved it:

iam.serviceAccounts.actAs
iam.serviceAccounts.get
iam.serviceAccounts.list

(I'm not sure which of these are necessary but it might provide someone in the future some pointers!)

[dylangolow]

I was able to use this thread to solve my issues with Cloud Build when deploying, if anyone is looking for this later.

The Cloud Build Service Account was the account I added permissions to, which altogether included these to end up with a successful build and deploy:

• Cloud Build Service Account
• Cloud Functions Admin
• Service Account User
• Cloud RuntimeConfig Admin (pre-release state)
• Storage Admin

[Thkasis]

I get the archiver package issue which @ahaverty mentioned above,

":"nosniff","alt-svc":"h3-29=\":443\"; ma=2592000,h3-27=\":443\"; ma=2592000,h3-T050=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","accept-ranges":"none","transfer-encoding":"chunked"}
[2020-08-14T02:26:48.863Z] FirebaseError: Server Error. socket hang up
[2020-08-14T02:26:49.992Z] Error: input source must be valid Stream or Buffer instance
at Archiver.append (C:\Users\Main\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\archiver\libcore.js:575:24)
at C:\Users\Main\AppData\Roaming\npm\node_modules\firebase-tools\lib\prepareFunctionsUpload.js:63:17
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Error: Could not read source directory. Remove links and shortcuts and try again.

[garyo]

I m getting the same issue , the debug keeps same error either while preparing functions folder to deploy or while deploy the only function I m trying to deploy . The debug log is not even telling me the file in which it is getting the mentioned error. Tried everything. Please help me resolve this issue , it is blocking my release for 2 days.

I found the problem (a temp emacs file, which is a dead symlink) using --debug after finding this issue. It would be really helpful to just print the name of the offending file in the error message!

[Thkasis]

I m getting the same issue , the debug keeps same error either while preparing functions folder to deploy or while deploy the only function I m trying to deploy . The debug log is not even telling me the file in which it is getting the mentioned error. Tried everything. Please help me resolve this issue , it is blocking my release for 2 days.

I found the problem (a temp emacs file, which is a dead symlink) using --debug after finding this issue. It would be really helpful to just print the name of the offending file in the error message!

where did you find that? are you on windows? Same issue here for weeks, only way I can get it to work is if I try multiple times it finally goes through sometimes but it's a pain and no good

[garyo]

Just do find . -type l; that'll print out all symlinks. There shouldn't be any symlinks in your functions dir or it complains.

[Thkasis]

@garyo thanks for the reply. There's none. It's been an error I've been getting for a long time but could manage to pass through it randomly. Now seems like I'm stuck ...
The --debug gives me this.

FirebaseError: Server Error. socket hang up
[2020-09-23T03:52:25.349Z] Error: input source must be valid Stream or Buffer instance
at Archiver.append (C:\Users\Main\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\archiver\libcore.js:575:24)

[solrac97gr]

In my case I had this error te solution was deactivated the VPN (TunelBear)