Cdnjs: Support SRI (Subresource Integrity)

[nikolay]

You CloudFlare just had an article regarding SRI, so, you should start publishing the integrity value for each.

[PeterDaveHello]

cc @terinjokes

[fletom]

+1

[jonathanKingston]

So is the script which imports the resource data to be indexed by algolia on GitHub? (I can't see it anywhere).

If it is then I can help with making the relevant pull requests as have done other work which does this.

Does cdnjs ever change the libraries like Google has done in the past?

[PeterDaveHello]

@jonathanKingston

Does cdnjs ever change the libraries like Google has done in the past?

What's the change actually? Thanks.

[jonathanKingston]

@PeterDaveHello it patches certain versions of their code if there are critical versions (Don't quote me on this but I think that was even if you provided a full semver version path). Either way they offer the ability to say Jquery@2 and it will self update.

I assume cdnjs doesn't do this ever right?

[PeterDaveHello]

Yeah, we don't.

[jonathanKingston]

@PeterDaveHello how does the search get populated as I think that would be the best candidate for storing the SRI hashes precalculated so when the client loads the page they can see the files with the hashes.

[jonathanKingston]

How does public/packages.min.json get populated? I can see the indexing code mostly uses that with the json containing file size etc.

[xPaw]

Chrome 45+ now supports SRI, https://googlechrome.github.io/samples/subresource-integrity/index.html

[PeterDaveHello]

@terinjokes do you have any idea? Thanks!

[mozfreddyb]

What's blocking this? Would love to use cdnjs with subresource integrity ;D

[terinjokes]

I'm hoping to chat with @PeterDaveHello soon to see where I can help in getting SRI displayed on cdnjs.com.

[ImAbhishekTomar]

Hi Team,Please share C# code - for bind chart with SQL database. In asp.net.

Regards,
Abhishek Tomar

On Tue, Sep 22, 2015 at 1:08 PM -0700, "Terin Stock" [email protected] wrote:
I'm hoping to chat with @PeterDaveHello soon to see where I can help in getting SRI displayed on cdnjs.com.

---

Reply to this email directly or view it on GitHub:
https://github.com/cdnjs/cdnjs/issues/4599#issuecomment-142404229

[PeterDaveHello]

hmmmm ... I have no idea yet, cc @cdnjs/team-cdnjs for comments.

[ryankirkman]

@PeterDaveHello We have kicked off a room in gitter to figure out a plan for adding SRI support to cdnjs. Stay tuned everyone!

[PeterDaveHello]

@jonathanKingston sorry that I missed your comment, public/packages.min.json will be built by this js app: packages.json.js, and this is the app to push data to algolia: reindex.js.

[PeterDaveHello]

Adding SRI into public/packages.min.json is an option I think.

[PeterDaveHello]

• 402e1b2 add integrity by sha384 for css/js files in packages.min.json
• e9c21ee add dependency sri-toolbox
• 11f366d Revert "simplify structure of packages.json to lower the size"
• cdnjs/newwebsite@e9d8bc5 remove filename remap, see cdnjs/cdnjs@11f366d

[PeterDaveHello]

Got a problem here ... packages.min.json with SRI is too huge ...

$ git push
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 25.04 MiB | 2.97 MiB/s, done.
Total 6 (delta 4), reused 0 (delta 0)
remote: error: GH001: Large files detected.
remote: error: Trace: 7dc2082b8c4103cad477100d620117a6
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File public/packages.min.json is 132.66 MB; this exceeds GitHub's file size limit of 100.00 MB
To [email protected]:cdnjs/new-website.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:cdnjs/new-website.git'

[PeterDaveHello]

The compressed packages.min.json by xz is 8.8MB only, I wonder if we just ship that data with xz and then decompress it in program.

[PeterDaveHello]

hmmmm ... the compressing process uses too much memory and then make nodejs crashed ...

[Salmatron]

100 MB? Really Github?! Maybe it's time to move to BitBucket.

[terinjokes]

@PeterDaveHello what is packages.min.json used for?

[PeterDaveHello]

@terinjokes it's all the meta data, a little bit huge, will try to move some data on a real DB, the reason why not move to a real DB is the difficulty to build/deploy the artifacts, especially for the contributors not be familiar with cdnjs's structure, but seems we can not do it any more, it just keep growing, though git-lfs can handle this, we still have a problem on heroku, the packages.min.json.xz, not sure it's the part of xz or just the file is too huge, will use memory unlimitly on heroku, though I have no any problem locally with 4GB ram, but on heroku, even I setup 14GB ram, it can still exceeded the quota of memory ... very strange, and looks not like memory leak issue because it crashes from the begin.

[PeterDaveHello]

@SalmanPK I don't think it's a reason to move to bitbucket, in honesty, we didn't use it correctly, but I am currently too busy, I'll move it on git-lfs when I have time.

[PeterDaveHello]

https://github.com/cdnjs/new-website/issues/104

[terinjokes]

@PeterDaveHello let's talk on gitter about this.

[PeterDaveHello]

Sure.

[FavioGalvis]

Hello everyone, I was trying to update my Libs in Netbeans, and the download process got stuck. After a little of research I see github is blocking the download, Is there como way to solve this temporally?

[PeterDaveHello]

@FavioGalvis this issue is not for that problem.

[jonathanKingston]

Hey, I know you were the hardest to implement based upon your stack and structure etc.

However has there been any progress on this at all @PeterDaveHello

Sorry to nag and thank you!

[PeterDaveHello]

@jonathanKingston sorry for the late reply, and I have to say sorry again, there is still no more progress.

Actually, I tried to force add them in our packages.min.json but the app crashed again and again on Heroku due to the memory limited, even though I changed to a Performance-L(cost hundreds per month) Dyno on heroku, which has 14GB memory, the funny thing is that it works great on my local 4GB ram computer, so that plan just been dropped from then on, because I don't have enough time to debug it then.

I'm not begging, but the truth I'm still a full-time master student, and there is not a company behinds CDNJS to support us, all the members in the owner group of CDNJS still have to work to earn money, so the time is limited, sometimes we can not respond so quickly, or you can just say - very slow, I can not guarantee the schedule of finishing the feature, but trust me, this feature is on the high priority todo list, cause we now have a robot now(See https://github.com/cdnjs/cdnjs/commits/master?author=PeterBot, https://github.com/PeterBot), so I can save some time to focus on features and the real issues, I'll be back here!

Not sure if you would like to help us building this feature, if you would like to, we can talk on gitter or via emails, thanks.

[drewfreyling]

@PeterDaveHello ping me on gitter if there is anything I can do to help out here.

[jakesyl]

Definitely look forward to seeing this!

[ScottHelme]

Hey guys, thought I'd check in on this issue and see where things were up to. It'd be great if the script tags provided by CDNJS included the integrity/crossorigin attributes for SRI!

[ImJustToNy]

Yes, please <3

[PeterDaveHello]

Guys I'm here, sorry for the late reply, still testing something there, will update soon!

[ScottHelme]

@PeterDaveHello I'm looking forward to hearing what you have in store!

[PeterDaveHello]

Hello guys,

Thanks for all of you being interested in CDNJS.

We just started to test the Subresource Integrity support with sha256 on each library's page on CDNJS. You can not try it! (Please don't forget to flush your browser cache!) Note that it's only on libraries' page, not the search result.

Hope it works well!

Thanks you all.

[ScottHelme]

@PeterDaveHello Maybe I'm missing something, but where should I see this sorry?

[PeterDaveHello]

@ScottHelme since I found the npm package we're using to caculate the hash have some different results for some files, that's crasy, I tested 3 npm package and 2 of them have this problem, and just on some files, not all of them, so I spent a while to rewrite the code, just directly call the openssl command to caculate the hash and testing also takes me a while, will be online again with in 30 mins.

[nikolay]

[PeterDaveHello]

The whole rebuild process seems to take longer time than I expect ... sorry.

[PeterDaveHello]

Should be there now, please don't forget to purge the cache on your browser, and please feel free to report bug, thanks.

[PeterDaveHello]

This issue ticket will still be open for the next two weeks and will be closed if there is no problem for a continuous 2 weeks.

[mozfreddyb]

I suggest you to include SRI all the time and just keep the user interface ("Copy" and "Copy Script Tag"). There are no downsides: Firstly, SRI will be ignored in browsers that do not understand it and secondly your URLs bear a version number, so accidental integrity mismatches should be impossible.

(I would name the buttons differently: "Copy URL" & "Copy HTML", but that's not related to this issue)

[PeterDaveHello]

I'll let the users decide use it or not, especially most of our users are developers, force SRI is not an option in 2016, may be in 2017 or 2018.

[ScottHelme]

This looks awesome @PeterDaveHello!! Thanks for the work :+1:

Is there any chance we could get the sha384 and sha512 hashes in there for future compatibility? What do people think to adding them? They are given in examples in the spec.

[PeterDaveHello]

@ScottHelme on sha256, we still need time to improve the performance, and make sure the stablity, and maybe few more features(like SRI support in the search result), so for the sha384 or sha512, actually no plan for them yet, as we don't have full time developer and enough sorponsors to have one, we still have other works to do beside the SRI support, so maybe it'll be the plan for the next year.

[PeterDaveHello]

@ScottHelme BTW, I'll let our API also support SRI before using sha384 or sha512, we only support SRI on the website's copy button currently.

[indolering]

Is there any chance we could get the sha384 and sha512 hashes in there for future compatibility?

Are you talking about quantum computers? If SHA2 is broken through traditional means then we will all be switching to SHA3.

we only support SRI on the website's copy button currently.

I came here to report this as a missing feature. In my not-so-humble opinion (I'm a usability engineer) the UI should default to the full src attribute, anyone wanting less than that can easily select the raw URL directly. Let's make this thing secure-by-default!

[PeterDaveHello]

@indolering I didn't get your idea, can you explain it in detail? Thanks.

[indolering]

I didn't know that you supported SRI until I found this ticket and saw the GIF. I think you should switch to displaying the entire <script src="https://..." integrity="sha256-SAex..." crossorigin="anonymous"> tag by default. If people want the _just_ the URL, they can easily select the URL directly or use the copy button's dropdown.

[PeterDaveHello]

@indolering It may be too long to display, but we'll try to push SRI funtion harder, thanks for your suggestion!

[indolering]

You don't need to display the whole thing, just the initial <script src="url" part, the rest can be hidden from view. I would bet money that most people use tricks like triple clicking or moving the cursor below the text display to select the entire text field at once.

[PeterDaveHello]

Close due to we already implemented it for a while, other things like display or copy method could be discussed in the new issues, thanks.

[jakesyl]

Really glad to have seen this implemented @PeterDaveHello! Thanks!