Cryptopp: Infinite compilation time on blake2.cpp

[Xottab-DUTY]

I'm using Visual Studio 15.5 Preview 2.0 compiling on Release x64 with enabled default optimizations setting.

With this configuration compilation of blake2.cpp goes forever... Only after disabling optimizations on this file, I managed to compile the library..

Edit: just tested Release x32 – infinite compilation too...

It seems that compiler can't optimize it normally..

[noloader]

@Xottab-DUTY,

Please file a bug report with Microsoft at Microsoft Connect.

Pinging @sneves to give him a heads up. Samuel is one of the authors of BLAKE2.

[noloader]

@Xottab-DUTY,

Can you provide a link to the Microsoft Connect issue so that we can cross-reference it?

We can probably provide you with a #define to make it easier to disable SSE4 for BLAKE2. Would you like us to add it?

[sneves]

Visual Studio 15.5 Preview 2.0 is not a supported release; the currently supported release 15.4.0 has no such problems compiling cryptopp in any configuration I could try. In the meaning Preview 3 has been released. Does the crash remain there?

Also, if the file that hangs the compiler is blake2.cpp and not blake2-simd.cpp, it seems unlikely that SSE4.1 has anything to do with it; more likely that the generic compression function optimization is the culprit.

[noloader]

@sneves,

... if the file that hangs the compiler is blake2.cpp and not blake2-simd.cpp, it seems unlikely that SSE4.1 has anything to do with it; more likely that the generic compression function optimization is the culprit.

Oh, you are right. I'm glad someone is paying attention.

[sneves]

A little debugging tells me the issue is in the new SSA optimizer of theirs, which can be disabled with the -d2SSAOptimizer- flag. The compiler appears to get stuck in an infinite loop during common subexpression elimination.

Correction: not infinite, but seemingly exponential on the number of BLAKE2 rounds. Up to 3 rounds terminate with increasingly high running time, 4 rounds is already too much. De-unrolling the loop fixes it.

[Xottab-DUTY]

I tried to login to my Microsoft account and failed. Then I forgot about the issue. =(

I have created an issue on VS Dev Community

The newest Preview 3.0 still falls in forever compilation of blake2.cpp. blake2-simd is not even included in my project, so the SSE is not the issue.

[noloader]

@Xottab-DUTY,

I tried to login to my Microsoft account and failed. Then I forgot about the issue. =(

Lol... I know what you mean. My account is locked out, too. I cannot find a Microsoft contact to get it unlocked. I guess they want me to ask on Stack Overflow :o.

@sneves ,

A little debugging tells me the issue is in the new SSA optimizer of theirs, which can be disabled with the -d2SSAOptimizer- flag. The compiler appears to get stuck in an infinite loop during common subexpression elimination.

Thanks for looking at that. I guess we should roll that thing back up. I think that's going to hurt ARM performance a bit. That was one of the tradeoffs we made to get better performance on all gadgets. Let me put it on the TODO list.

[noloader]

@Xottab-DUTY,

The newest Preview 3.0 still falls in forever compilation of blake2.cpp. blake2-simd is not even included in my project, so the SSE is not the issue.

It sounds like you are using Crypto++ 5.6.5. Samuel and I we discussing Master. It sounds like wires got crossed. Sorry about that.

I don't know what your use case is, so this may not apply. We switched to Base+SIMD after Crypto++ 5.6.5 was released. Base provides straight C++ and SIMD provides Intel, ARM and Power8 implementations.

Base+SIMD is needed for Unix & Linux distros, where distros build for a "least capable" machine. At runtime we use SIMD if available; otherwise we use straight C++. The "straight C++" only uses the baseline architecture, which is SSE2 in the case of x86_64. Base can run anywhere.

In general, the model does not apply to Microsoft compilers. Microsoft compilers don't need special architectural switches to compile a source file. Once Microsoft makes a feature available, like AES-NI, its just available. Unix and Linux are different, and they need -maes for AES-NI on Intel machines; and -march-armv8-a+crypto on Aarch64; and -mcpu=power8 -maltivec on Power8.

Base+SIMD is kind of a pain in the ass. I like Microsoft's model much better, where we can use an instruction if the compiler supports the feature. But we have to support Unix & Linux equally well, so we felt compelled to make the architectural change. Otherwise, on Unix & Linux, a new 7th gen Core i7 would only use SSE2, and not AVX{2}, BMI{2}, AES, SHA, RDRAND, RDSEED, CRC32, SSE4 (and other goodies).

[noloader]

@Xottab-DUTY,

The newest Preview 3.0 still falls in forever compilation of blake2.cpp. blake2-simd is not even included in my project, so the SSE is not the issue.

By the way, the latest BLAKE2 source files are at:

• blake2.cpp
• blake2-simd.cpp

The source files do not include the specialized SSE2 implementation. They only include the SSE4 and ARM implementations. SSE2 was removed because its use case was so small.

[Xottab-DUTY]

@noloader I have used version 5.6.5 but I updated it to 1 or 2 of November master branch. But not all *-simd files was included in the project

[noloader]

@Xottab-DUTY,

Try this: Commit f1386ad89470. The pain point we are trying to workaround is, specialize a data-only template class with alignment requirements. Its difficult to find something that works under all compilers and all C++ versions.

The f1386ad89470 commit clears an error under SunCC. We added the alignment to the specialization because another compiler complained it needed the alignment on the specialization.

I'm thinking this could be one of the problems the Microsoft compiler may be experiencing.

[noloader]

@Xottab-DUTY,

Here's another BLAKE2 commit of interest: Commit e8bed05b7dc8.

I don't believe its related to your issue, however.

[noloader]

Closing. I think we are done here.

[Xottab-DUTY]

Sorry for the late answer. Just can't get enough time to try your new commits (trying to fix other issues)

As soon as I test it, I say to you if issue was fixed or not =)

[sneves]

For what it is worth, Visual Studio 15.5 has been released just now, and it would appear that the fix was not included---blake2.cpp never terminates compiling. Might require a workaround.

[noloader]

@Xottab-DUTY, @sneves,

Sorry about the delay. I was able to duplicate the issue under the x64 compiler that was released recently.

It looks like the problem was the related to the BLAKE2B_ROUND and BLAKE2S_ROUND macros. Once the macros were changed to inline functions the problems went away.

Cleared at Commit b436411de523.

[sneves]

For what it's worth, the compiler bug has now been fixed in the latest update.

[noloader]

@sneves, @Xottab-DUTY,

Botan is going through testing for upcoming 2.4.0 release. Botan hangs, too.

Pinging JL @randombit as an FYI...

[Xottab-DUTY]

By the way I still not updated my Crypto++ submodule to the latest commit.. Sorry for that..