Sp-dev-docs: 🐞 Unable to restore dependencies of a project created using SPFx v1.0

[michel-weber]

Category

• [ ] Question
• [ ] Typo
• [x] Bug
• [ ] Additional article idea

Expected or Desired Behavior

npm install should be able to restore the depencies of an SPFx WebPart. On each and every publicly released version of the @microsoft/generator-sharepoint.

Observed Behavior

On a freshly checked out version of the project (no node_modules folder), the build fails on npm install step with the following error message:

npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "--cache-min" "86400"
npm ERR! node v6.11.2
npm ERR! npm  v3.10.10
npm ERR! code E404

npm ERR! 404 Not found : @microsoft/node-core-library
npm ERR! 404
npm ERR! 404  '@microsoft/node-core-library' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of '@microsoft/gulp-core-build'
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

As we have fully automated the build and deploment of SPFx WebParts, I run a daily test build and deployment. Yesterday it was fine, this error started happening today (30.08.2017). So there must have been some recent changes.

The project was created with the 1.0 release of SPFx.

Steps to Reproduce

This can be reproduced by just creating a new HelloWorld WebPart - but using version 1.0.2 of the yeoman generator template.

1) Uninstall the current version: npm uninstall --global @microsoft/generator-sharepoint
2) Install version 1.0.2 of the generator: npm install --global @microsoft/[email protected]
3) Create a new WebPart: yo @microsoft/sharepoint
4) Now it fails trying to install the dependencies with error npm ERR! 404 Not found : @microsoft/node-core-library

My thoughts

As a work around, I now have to recreate my project with the newest version of the generator.
Please fix this. It would also be good if you set up more rigorous automated tests as breaking an officially released SPFx version is an absolute no-go. I also shouldn't be forced to recreate SPFx WebParts just in order to get them to build again.

This could maybe be related to another error that @waldekmastykarz is experiencing:
Unable to restore dependencies of a project created using SPFx v1.1.0 #813

[VelinGeorgiev]

I am getting the exact same issue.

[qz2017]

node-core-library is published. Can you please try again?

[qz2017]

Feel free to reopen if it is still happening to you.

[VesaJuvonen]

Let's ensure that the issue is fixed and close the issue after that. @michel-weber and @VelinGeorgiev - can you double check the situation also from your side to ensure that this is now fixed. Thanks for your input advance.

[qz2017]

@VesaJuvonen , I tried the repro. It works now.

[VelinGeorgiev]

Hey @VesaJuvonen, will test tomorrow and let you know.

[pedropablo]

It happens also with v1.1.1. Build in VSTS failed.

2017-08-30T21:47:21.7508260Z [command]C:\NPM\Modules\gulp.cmd bundle --gulpfile d:\a\1\s\code\gulpfile.js --ship
2017-08-30T21:47:23.3536281Z [21:47:23] Working directory changed to d:\a\1\s\code
2017-08-30T21:47:24.9706265Z { Error: Cannot find module '@microsoft/node-core-library'
2017-08-30T21:47:24.9706265Z     at Function.Module._resolveFilename (module.js:469:15)
2017-08-30T21:47:24.9706265Z     at Function.Module._load (module.js:417:25)
2017-08-30T21:47:24.9706265Z     at Module.require (module.js:497:17)
2017-08-30T21:47:24.9706265Z     at require (internal/module.js:20:19)
2017-08-30T21:47:24.9706265Z     at Object.<anonymous> (d:\a\1\s\code\node_modules\@microsoft\api-extractor\lib\Extractor.js:8:27)
2017-08-30T21:47:24.9706265Z     at Module._compile (module.js:570:32)
2017-08-30T21:47:24.9706265Z     at Object.Module._extensions..js (module.js:579:10)
2017-08-30T21:47:24.9706265Z     at Module.load (module.js:487:32)
2017-08-30T21:47:24.9706265Z     at tryModuleLoad (module.js:446:12)
2017-08-30T21:47:24.9706265Z     at Function.Module._load (module.js:438:3)
2017-08-30T21:47:24.9706265Z     at Module.require (module.js:497:17)
2017-08-30T21:47:24.9706265Z     at require (internal/module.js:20:19)
2017-08-30T21:47:24.9706265Z     at Object.<anonymous> (d:\a\1\s\code\node_modules\@microsoft\api-extractor\lib\index.js:5:19)
2017-08-30T21:47:24.9706265Z     at Module._compile (module.js:570:32)
2017-08-30T21:47:24.9706265Z     at Object.Module._extensions..js (module.js:579:10)
2017-08-30T21:47:24.9706265Z     at Module.load (module.js:487:32) code: 'MODULE_NOT_FOUND' }
2017-08-30T21:47:24.9746253Z [21:47:24] Error - Unknown 
2017-08-30T21:47:24.9746253Z  Cannot find module '@microsoft/node-core-library'

[OliverZeiser]

I am also seeing this issue. After installing the latest generator and doing an npm install in a project created with 1.1.0. Even uninstalling the latest generator and installing @1.1.0 again does not work. The project is broken:
Error: Cannot find module '@microsoft/node-core-library'

[michel-weber]

Hey @VesaJuvonen , i tested again and the npm install now works.
I downgraded the generator to 1.0.2 as before.

But it is failing now at gulp build --ship.
The exactly same error is happening for a completely new project (yo @microsoft/sharepoint) - and also for my existing WebPart.

λ gulp build --ship
Build target: SHIP
TypeError: Cannot read property 'readCommentedJsonFile' of undefined
TypeError: Cannot read property 'readCommentedJsonFile' of undefined
[09:13:06] Error - Unknown
 undefined
[09:13:06] ==================[ Finished ]==================
[09:13:07] Project testdeployment-spfx-webpart version: 0.0.1
[09:13:07] Build tools version: 2.10.0
[09:13:07] Node version: v6.11.2
[09:13:07] Total duration: 2.29 s
[09:13:07] Task errors: 2

Steps to reproduce (same as before just run gulp build --ship afterwards):
1) Uninstall the current version: npm uninstall --global @microsoft/generator-sharepoint
2) Install version 1.0.2 of the generator: npm install --global @microsoft/[email protected]
3) Create a new WebPart: yo @microsoft/sharepoint
4) Run gulp build --ship

I can also create another issue but I think it is strongly related.

[VelinGeorgiev]

I am still getting the same error:

Steps to reproduce:

• Downgrade yeoman generator from version 1.2 to @microsoft/[email protected]
• Go to existing SPFx solution folder
• Delete the node_modules folder
• Do fresh npm install
• Run gulp serve

{ Error: Cannot find module '@microsoft/node-core-library'
    at Function.Module._resolveFilename (module.js:485:15)
    at Function.Module._load (module.js:437:25)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\Projects\SharePointIntranet\src\xxx-webparts\node_modules\@microsoft\api-extractor\lib\Extractor.js:8:27)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\Projects\SharePointIntranet\src\xxx-webparts\node_modules\@microsoft\api-extractor\lib\index.js:5:19)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32) code: 'MODULE_NOT_FOUND' }
[12:03:32] Error - Unknown
 Cannot find module '@microsoft/node-core-library'
[12:03:33] ==================[ Finished ]==================
[12:03:33] Project xxx-webparts version: 0.0.1
[12:03:33] Build tools version: 2.5.3
[12:03:33] Node version: v8.2.1
[12:03:33] Total duration: 2.86 s
[12:03:33] Task errors: 2

There is my config:

Node version: v8.2.1
NPM version: v3.10.10
NPM global packages:

λ npm -g list --depth=0
C:\Users\xxx\AppData\Roaming\npm
+-- @microsoft/[email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
`-- [email protected]

packages installed in the webpart

λ npm list --depth=0
[email protected] C:\Projects\SharePointIntranet\src\xxx-webparts
+-- @microsoft/[email protected]
+-- @microsoft/[email protected]
+-- @microsoft/[email protected]
+-- @microsoft/[email protected]
+-- @microsoft/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- @types/[email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
`-- [email protected]

[VesaJuvonen]

Thx everyone for following up on the issue and confirming if the fix works or not. We'll follow up on this internally based on your updated input.

[rgylesbedford]

We have having this problem via VSTS builds. VSTS is configured to do the following based on https://dev.office.com/blogs/vsts-build-and-release-pipelines-for-sharepoint-framework-solutions

1. Get Sources
2. npm install
3. gulp update-manifest --cdnpath "$(PublicCdnPath)"
4. gulp bundle --ship
5. etc.

It is currently failing on gulp update-manifest

2017-08-31T15:46:12.3139253Z ##[section]Starting: gulp update-manifest
2017-08-31T15:46:12.3139253Z ==============================================================================
2017-08-31T15:46:12.3139253Z Task         : Gulp
2017-08-31T15:46:12.3149247Z Description  : Node.js streaming task based build system
2017-08-31T15:46:12.3149247Z Version      : 0.5.31
2017-08-31T15:46:12.3149247Z Author       : Microsoft Corporation
2017-08-31T15:46:12.3149247Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613721)
2017-08-31T15:46:12.3149247Z ==============================================================================
2017-08-31T15:46:13.4165614Z [command]C:\NPM\Modules\gulp.cmd update-manifest --gulpfile d:\a\9\s\gulpfile.js --cdnpath https://publiccdn.sharepointonline.com/gb365.sharepoint.com/cdn/spfx
2017-08-31T15:46:14.7529313Z [15:46:14] Working directory changed to d:\a\9\s
2017-08-31T15:46:15.9879720Z { Error: Cannot find module '@microsoft/node-core-library'
2017-08-31T15:46:15.9879720Z     at Function.Module._resolveFilename (module.js:469:15)
2017-08-31T15:46:15.9879720Z     at Function.Module._load (module.js:417:25)
2017-08-31T15:46:15.9879720Z     at Module.require (module.js:497:17)
2017-08-31T15:46:15.9879720Z     at require (internal/module.js:20:19)
2017-08-31T15:46:15.9879720Z     at Object.<anonymous> (d:\a\9\s\node_modules\@microsoft\api-extractor\lib\Extractor.js:8:27)
2017-08-31T15:46:15.9879720Z     at Module._compile (module.js:570:32)
2017-08-31T15:46:15.9879720Z     at Object.Module._extensions..js (module.js:579:10)
2017-08-31T15:46:15.9879720Z     at Module.load (module.js:487:32)
2017-08-31T15:46:15.9879720Z     at tryModuleLoad (module.js:446:12)
2017-08-31T15:46:15.9879720Z     at Function.Module._load (module.js:438:3)
2017-08-31T15:46:15.9879720Z     at Module.require (module.js:497:17)
2017-08-31T15:46:15.9879720Z     at require (internal/module.js:20:19)
2017-08-31T15:46:15.9879720Z     at Object.<anonymous> (d:\a\9\s\node_modules\@microsoft\api-extractor\lib\index.js:5:19)
2017-08-31T15:46:15.9879720Z     at Module._compile (module.js:570:32)
2017-08-31T15:46:15.9879720Z     at Object.Module._extensions..js (module.js:579:10)
2017-08-31T15:46:15.9879720Z     at Module.load (module.js:487:32) code: 'MODULE_NOT_FOUND' }
2017-08-31T15:46:15.9909924Z [15:46:15] Error - Unknown 
2017-08-31T15:46:15.9909924Z  Cannot find module '@microsoft/node-core-library'
2017-08-31T15:46:16.4942188Z [15:46:16] ==================[ Finished ]==================
2017-08-31T15:46:16.9976981Z [15:46:16] Project gb365-intranet-spfx version: 0.0.1
2017-08-31T15:46:16.9976981Z [15:46:16] Build tools version: 2.5.3
2017-08-31T15:46:16.9976981Z [15:46:16] Node version: v6.10.0
2017-08-31T15:46:16.9976981Z [15:46:16] Total duration: 1.98 s
2017-08-31T15:46:16.9976981Z [15:46:16] Task errors: 2
2017-08-31T15:46:17.0357002Z ##[error]Gulp failed with error: C:\NPM\Modules\gulp.cmd failed with return code: 1
2017-08-31T15:46:17.0376985Z ##[section]Finishing: gulp update-manifest

I can provide the full logs from VSTS if needed.

[octogonz]

We are working on a fix

[rgylesbedford]

I can confirm the VSTS build has completed successfully for me. Thanks @pgonzal

[octogonz]

We republished a new version of the api-extractor package, which should fix the SPFx 1.1.x releases. We are working on a more comprehensive fix for SPFx 1.0.0, which should also insulate old releases from these types of regressions in the future.

[pedropablo]

VSTS build completed succesfully. SPFx v1.1.1

[octogonz]

We have republished a patch increment of the old libraries, which should restore the old behavior for SPFx 1.x. Going forward, we're going to remove all SemVer tildes/carets from our package.json files, which should avoid this sort of trouble in the future.

(The purpose of SemVer is to answer the question "Will the API contract change if I upgrade this package?" Whereas for SPFx tooling, the question should be "Did Microsoft test this combination of package versions?" We realize that this approach won't lock down the version specifiers for our indirect dependencies. We're still considering the best long term solution for that.)

Anyway, sorry about the trouble. Everything should be resolved now. Please let us know if anyone is still having trouble installing any of the previous SPFx releases.

@qz2017 @VesaJuvonen

[VesaJuvonen]

Hi @VelinGeorgiev and @OliverZeiser,
would appreciate quick re-test now that the situation should be resolved related to the older version. Just to ensure that we truly got this solved and that you guys are also unblocked. Thx for your confirmation advance.

[OliverZeiser]

I can confirm that it is working again now! But I had to uninstall the generator and reinstall it with @1.1.0 . Before doing that it did not work.
Thx!

[andrewconnell]

@pgonzal said in 326445803:

Going forward, we're going to remove all SemVer tildes/carets from our package.json files, which should avoid this sort of trouble in the future.

This is AWESOME news that I suspect most won't notice. Huge kudos for doing this & thanks! Granted, you can't control what upstream does, but appreciate this change!

[michel-weber]

Hi @VesaJuvonen
I can also confirm that it is working again now!

Thanks everyone for the swift resolution and for taking steps to ensure this never happens again :1st_place_medal: .

[VesaJuvonen]

Thanks, everyone. Closing the issue, since root cause has been resolved. Like Pete noted above, we are working on making sure that this will not happen in future.

[lucmoco]

Hello @pgonzal,
One step forward would be to upgrade to npm 5. This version of npm creates by default a package.lock file which fixes the dependency tree (including package versions). Thus a working web part will not break anymore if an incompatible dependency is published (however newly created web parts will still break).

[waldekmastykarz]

@lucmoco unfortunately there are other problems with using npm@5 for example when starting a project with a web part and adding an extension afterwards. When using npm@5 the additional dependencies required by the extension won't be installed and your project won't build anymore.

[michel-weber]

@waldekmastykarz Are there issues with using npm@4? As far as I know this is officially supported by SPFx.

[waldekmastykarz]

@michel-weber correct. npm@4 is indeed the currently supported version

[lucmoco]

@waldekmastykarz sorry if I was not clear enough. My suggestion was that the SPFx infrastructure upgrades to npm5, not its current users (I know that npm5 is not currently supported with the SPFx).

[waldekmastykarz]

@lucmoco gotcha! That would be the best of course!

[octogonz]

FYI we have tried several times to upgrade to NPM 5 internally for ourselves, but each time we had to roll it back due to bugs in NPM. People with smaller projects seem to do okay with it, but for larger code bases NPM 5 has had a pretty rocky start. If you pick through their Release Notes for each 5.x release, you will find many serious regressions.

Due to this frustration, we also investigated Yarn and pnpm. Yarn is simply a reimplementation of NPM's approach but with better coding practices, whereas PNPM is a smarter node_modules folder layout that potentially can be radically simpler and faster. (It simulates a flat layout like a conventional package manager, which avoids all the tree balancing nonsense) This past month we have been experimenting with switching our internal systems to use pnpm. Of course, we will continue to support NPM for SPFx developers, and eventually will get to version 5. :-)

[andrewconnell]

@pgonzal

we have been experimenting with switching our internal systems to use pnpm

Interesting... I've had very good success with Yarn the last few months with one exception (React typings). Really curious to hear about your experiences with PNPM. Any chance you plan to share your internal results?

[octogonz]

Yes, we will certainly post something when we have some results to share. The tooling changes are open source.

[msft-github-bot]

Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues