Docs: Use https instead of http for links whenever possible

[mairaw]

See https://twitter.com/Duncanma/status/968369013234569216 for background info

Anyone jumping on this issue should break the PR into multiple smaller PRs so we're not overwhelmed to review this. An example can be seen with PR #4506 that only contains the links in the F# folder.

[ChrisMaddock]

This sounded like a fun little scripting task. 😄

I started with the low-hanging fruit, and wrote a tool that changes the URL to https if and only if https is supported, _and_ there's no redirection involved. I restricted it to just the docs/standard subdir, and still ended up with 245 changes across 162 files.

I did a check through those initial 245, and they looked good - but manually reviewing changes like this across the entire repository sounds like it would be an enormous task.

The code I wrote is currently under 100 lines. Would the docs team be interested instead in reviewing that code, and then accepting an automated PR covering a wider area of the repository instead? Or was the intention of this issue more to allow an easier contribution option for newcomers? Either is fine for me, I've got what I wanted out of it!

For reference: the docs/standard changes are here: https://github.com/dotnet/docs/compare/master...ChrisMaddock:master. I won't PR them yet, due to size.

[BillWagner]

@ChrisMaddock Thanks for working on this. I like the process you've taken.

We do the same internally: reviewing script code, and then comparing output. If you can put the code in a gist, or somewhere else where it is publicly available, we'll review it, spot check the output, and then merge it.

Thanks again for the continued involvement.

[ChrisMaddock]

Great, thanks @BillWagner 😄

Wasn't expecting such a quick response - I'd like to tidy it up a little first, and am out of time for tonight. I'll come back to this in the next couple of days and put it in a PR. Thanks!

[BillWagner]

You happened to catch me online. 😸

[mairaw]

No rush @ChrisMaddock! Excellent way to approach this. My approach had some issues on the F# PR. 😢

It would be interesting to look at the ones that redirect on a later pass too.

[ChrisMaddock]

It would be interesting to look at the ones that redirect on a later pass too.

Absolutely! I wanted to keep this review-simple to start with however, given there's so many of them. 😄

I think the next pass I'd want to do would be to update all the fwlinks/msdn links that redirect to a docs.microsoft.com page - as I think that's a fairly large amount, and they would be nice to get sorted. That would of course involved working out the relative links etc

[ChrisMaddock]

I've just got back to this - but spotted https://github.com/dotnet/docs/issues/4789. Will hold off until after that's gone though. 😄

[mairaw]

That's fine @ChrisMaddock!

[ChrisMaddock]

@mairaw - Now the code's been reviewed and the first batch merged - I'd like to run this across the rest of the repo. (if that's a useful contribution!)

I've just done this in dotnet-api-docs, and it's made changes to 1611 files. What's the easiest way for me push that - as a single PR, or in batches of e.g. 100? Or are we better just leaving things as they are, given the number of changes needed - most of these are msdn links, which redirect to https anyway.

I wouldn't want this to be something that needs hours of review time - I'm aware how low priority it is. 😄

For interests sake, links from the following domains were found and redirected:
download.microsoft.com
code.msdn.microsoft.com
www.rfc-editor.org
www.iana.org
www.unicode.org
http2.github.io
www.w3.org
unicode.org
msdn.microsoft.com
www.microsoft.com
en.wikipedia.org
referencesource.microsoft.com
www.ietf.org
www.regular-expressions.info
www.hanselman.com
opensource.org
www.ecma-international.org

[mairaw]

Thanks @ChrisMaddock. I just noticed your question, I'm sorry. The way you've been doing the recent PRs is perfect.

[ChrisMaddock]

Thanks @mairaw! 🙂

On the above, I realised that roughly 1400/1611 links were all msdn. I may do one PR for all of those at some point given it's a single domain - but msdn redirects to https anyway, so I didn't think it particularly worth the effort at the time. 😄

[mairaw]

Yeah that sounds appropriate @ChrisMaddock. And we'll be redirecting the msdn content over to Docs, so we'll have to revisit all the msdn links anyway.

[mairaw]

Adding here a spreadsheet with the http links identified by our internal report that should be https to help with this task for the dotnet/docs repo:
https-links-results-full.xlsx

I broke down by folder

edit by @mairaw: uploaded a new version of the spreadsheet that contains more results but PRs should be broken down

[ChrisMaddock]

@BillWagner - was that PR meant to close this entire issue? 🙂

(Sorry - I got a notification from when I commented here two years ago. Was quite impressed that it had all been finished! 😅)

[BillWagner]

Thanks @ChrisMaddock Reopening.

It was a hacktoberfest PR that used "fixed" rather than "contributes to"