Git: Use MSVC2013+ compiled Perl to fix DEP/ASLR issues

[dori4n]

The current version of Perl that ships with Git for Windows is a mingw/cygwin derivative build, which has known problems with DEP and 64-bit ASLR on Windows (versions 8 and newer), in that it causes Git to crash on simple pull and merge operations with custom parameters, if DEP is set to AlwaysOn in the BCD configuration via bcdedit.exe /set {current} nx alwayson from an elevated command prompt and rebooting, or more commonly, by using the Enhanced Mitigation Experience Toolkit (EMET) user interface application for the purpose. Fixing the issue should be simple, and building Perl with the MSVC compiler or using prebuilt binaries should be fairly straightforward. You can use the free Community edition for this.

[whoisj]

Well here you can sample the fun of just getting Git to be loaded by Visual Studio - this doesn't even get things building with the MSVC toolchain. #1093 #552 etc.

We're always happy to accept contributions :wink:

[dori4n]

Hey, I'm not asking you to switch toolchains for the build process, just the Perl bits, of which you use pre-compiled binaries, if I'm not mistaken. Should be an easy drop-in replacement. I don't have the time to set up my own build environment for Git. I'll just use a build VM with DEP set to optin for the purpose...
You can look at how slic3r (3D printer software) did it.

[dscho]

First of all, i much prefer contributions in the form of Pull Requests than ticket that tell me what somebody thinks I should do.

Second, it is easy to misunderstand the ramifications of exchanging the Perl interpreter for another one. The ramifications are quite serious, though, and it would be an enormous mistake to underestimate (or even ignore, let alone not bother to learn about) them:

• The number one reason why Git for Windows ships with Perl is to support git svn. That means, we have to have the Subversions bindings for Perl. It is a non-trivial endeavour to get them to compile, and so far it was enough to suggest to anyone who suggested any change in our Perl setup to go and try to build the native bits behind SVN.pm to "silence them". Everybody felt smart enough to suggest to switch to this or that Perl (Strawberry being an all-time favorite), but strangely those voices were all of a sudden quiet when the same persons were asked how they would go about building Subversion support. Maybe you are the first one not to shrink away from this requirement.
• The way Git uses Perl is littered with the assumption that the filesystem is a Unix one. That is, whether a path is absolute is often verified by looking at the first letter of the path and interpreting a slash (and only a slash) as meaning that the path is absolute. This assumption is hard to please with anything by Cygwin's or MSYS2' Perl. Git's scripts would have to be vetted and fixed to allow running in pure Win32 Perl interpreters.
• In addition to git svn, there are other Perl scripts in Git, most notably git send-email. That script in particular makes use of Perl modules that may not come with the stock distribution of your favorite Perl flavor du jour. When that happens, the module needs to be built, meaning that the development environment (read: Git for Windows' SDK) has to provide everything to compile the native C sources with the same compiler as the Perl and with the same compile options. As the Git for Windows SDK is a derivative of the MSYS2 build system, it is easy to get GCC. What is not easy is to integrate any other compiler.

In short, my take is that the idea put forth in this ticket is based on an incomplete understanding of the size of the task.

And I do not think that the benefit merits the size of the task.

@dorianmuthig if you disagree, please do not hesitate even one second to tackle the project. I will definitely welcome high quality Pull Requests.

[dori4n]

Like I said, I understand what Perl is used for in git, SVN support, recursive repository editing and commit action helpers being the big ones. Using them on a system with DEP set to alwaysOn will consistently have the Git process crash in a somewhat violent manner (and can corrupt a repository, if unlucky). I do not have the time to spare to fix it, otherwise I definitely would. For the time being, this issue ticket should mostly serve to let people know. As for your mention of Strawberry Perl, that distribution has the same issue and won't run with DEP enabled.
Personally, however, I do quite sincerely believe this type of system security configuration is what every developer should use to build software that works properly from the get go. Requiring users to relax their security configurations to run software to complete a task, or even having the audacity to do it automatically, is a really bad habit. DEP/NX has been around for over 15 years, ASLR for 10 years. It is quite reasonable to expect them to work and not run into this kind of trouble, though I can understand this not being a priority for you.

[dscho]

I do quite sincerely believe this type of system security configuration is what every developer should use to build software that works properly from the get go.

I am a bit tired of reading people telling other people what they need to do.

Imagine for a second what your reaction would be if I told you: you want Perl to run with DEP set to alwaysOn? Then you need to drop everything else and work on this project until it is done.

See how that sounds?

[dori4n]

I am a bit tired of reading people telling other people what they need to do.

Are you now? Get off your high horse with the "works on my machine" argument. Software engineers who's first answer to making software work with security systems is to remove or disable said security systems and offering similar suggestions to all their users are the worst. I do sincerely hope you are not one of them.

Imagine for a second what your reaction would be if I told you: you want Perl to run with DEP set to alwaysOn? Then you need to drop everything else and work on this project until it is done.

Been there, done that. Perl works.

See how that sounds?

Yes. Do you?

[PhilipOakley]

Hi Dorian

Are you now?

You appear to think that @dscho is being paid to work 24/7 on GfW and its tool chain. He is also a volunteer who has worked tirelessly to maintain the GfW fork of git.git. He is fortunate that his $dayjob does support his activities, so at least GfW has remained pretty functional.

He would desperately like more help with some of the deeper issues that are concerning other devs, who should be able to help. Perl is one of them.

As @dscho explained, a lot of careful work is needed to get the perl/svn/MSYS2 _tool chain_ to be properly functional for GfW. It is "easy" to replace the current perl with an alternatively broken version of perl that cannot bind to svn, but it would still be "broken" (relative to full functionality and being up to date).

We all have our time (and capability) limitations, but it's not helpful to suggest that @dscho is not trying hard (#1162). (Apologies if my wording has fallen into the same trap).

If you can contribute even a little bit time to help with research, trial code, information, etc. to build up the body of knowledge / code to get the perl updated that would be a help. It could be a 'do not merge' PR that contains the relevant info so that it gets brought together to help others.

It would be nice to get a perl that works for everybody. I hope you can help a little.

[dori4n]

We all have our time (and capability) limitations, but it's not helpful to suggest that @dscho is not trying hard (#1162).

Neither is doing the same to anyone else, I'm sure...
I do not believe this type of discussion is what the issue queue is for, either, however.

It would be nice to get a Perl that works for everybody.

Build it from the official project source code on the platform you are targeting for the platform you are targeting using the build tools the platform vendor provides. Simple enough, I would think... but by all means, do take all the time you need. ;)

[whoisj]

Build it from the official project source code on the platform you are targeting for the platform you are targeting using the build tools the platform vendor provides. Simple enough, I would think...

You would think, but then the Git for Windows project would own the bugs; or at least the responsibility to triage any issues to prove it wasn't because the forked build and not because of an upstream issue. The Git for Windows community has enough of that on its plate re: upstream Git and Msys.

I understand what you're looking for, and I appreciate your need. The polite answer is we would love to help you here, but lack capacity to do anything about it in the near term. You have two choices: you can wait until capacity becomes less scarce and the priority of this request rises to the top; or you can add capacity and contribute the work yourself.

This would be your decision, your choice. We'll leave it to you. Thanks.

[dscho]

It would be nice to get a Perl that works for everybody.

Build it from the official project source code on the platform you are targeting for the platform you are targeting using the build tools the platform vendor provides.

By platform, you probably refer to Windows (Vista, 7, 8, 8.1, 2008, 10)? Or MSYS2? Or MINGW? Or VisualStudio? Or Cygwin? Or all of the above, in which case you are talking about at least 8 platforms, and more if you count the ARM devices.

Have you even bothered to see for what purposes Perl is used by Git for Windows? It matters in this context, because that adds some very serious limitation on what version of Perl we require.

Simple enough, I would think...

Yes, you would think. And by saying it that smugly, you would not only show that you think wrongly, but it also demonstrates that you are not even aware how much knowledge you lack to make that judgement. And it also shows an inclination to try making other people feel bad.

It makes me sad that I have to point out all of this.

but by all means, do take all the time you need. ;)

More of the same smugness. Do you really think it is for you to decide who works for your ends? Do you really think that a statement like this will get you what you wanted?

I understand that it feels to you that you are not asking for much. The reality is: you are asking for a lot.

And you are not just asking, the way you phrase it makes everybody else feel a lot worse than necessary, which I do not appreciate, not one bit. It is so bad, even, that I would consider it to violate our stated Code of Conduct that tries to foster a positive environment. That, combined with the fact that you have shown a strong inclination to expect other people to do all the work while you do nothing apart from commenting from the sidelines (which stance flies in the face of what makes Open Source great) makes this discussion a pretty unhappy read.

So let's just close it.

[dori4n]

dscho commented on Sep 14, 2017*

Yes, you would think. And by saying it that smugly, you would not only show that you think wrongly, but it also demonstrates that you are not even aware how much knowledge you lack to make that judgement. And it also shows an inclination to try making other people feel bad.

It makes me sad that I have to point out all of this.

[...]

And you are not just asking, the way you phrase it makes everybody else feel a lot worse than necessary, which I do not appreciate, not one bit. It is so bad, even, that I would consider it to violate our stated Code of Conduct that tries to foster a positive environment. That, combined with the fact that you have shown a strong inclination to expect other people to do all the work while you do nothing apart from commenting from the sidelines (which stance flies in the face of what makes Open Source great) makes this discussion a pretty unhappy read.

Let's be perfectly frank here: The only person doing any of those things here was you. You have been nothing but rude. Nothing about the simplicity of building the project on Windows using the MSVC2013 or newer MSVC compilers is difficult. Yes, it may not work with your current deployment methods or integrate with any automation you have currently set up, but it is easy and very simple. So easy and simple in fact, that any seasoned software engineer can clone the repository from GitHub, download Perl for Windows binaries and start the build in a few minutes.

However, since you have been so very rude and unfriendly, instead of simply offering civil disagreement, I am definitely not going to share it with you now. Tough luck, sir.

[PhilipOakley]

@NightmareJoker2, Is 2 months (14-Sept .. 5-Nov) sufficient time to show how easy it is to get started on that simple build and demonstrate some progress? Or have you also been busy with other things rather than this important (but simple and easy) task.

Fortunately, at the moment I don't have to deal with SVN, but it may be coming. @dscho certainly hasn't had sufficient time for what he estimated would be a significant task. I'm thankful for the all work he has done.

Yours in engineering,

Philip

[dscho]

@PhilipOakley let's not feed the troll.

[dori4n]

Wow, guys... seriously?

@PhilipOakley I'm done, and could send in a pull request in less than 5 minutes, if I wanted to. As previously stated, I won't. This also isn't merely SVN related. The issue has gotten so bad, that the version you have can clone a repository, but not switch branches, nor perform pull, rebase or merge operations with the DEP and/or ASLR security options enabled at the system level on Windows 10 1709 (Fall Creators Update).
Of note is also CERT's Vulnerability Note VU#817544 as that is public now. It may help you understand.

@dscho Rude. Just rude.

[dscho]

I'm done, and could send in a pull request in less than 5 minutes, if I wanted to.

No, you could not. It would take substantially longer than 5 minutes to get Git (including most notably git svn) to work with MSVC-built Perl. Your assessment is just uninformed, and the form in which you brought it to us is very unfortunate.

It is the opposite of what the people who are active in the project deserve or want (being it contributors developing code, or end users trying to help others, or end users reaching out for help). Our covenant says as much: we want a friendly, competent exchange. You are welcome here as long as you abide by those rather simple and obvious rules.