Nixpkgs: Steam: openssl / libcrypto error

[FRidh]

Issue description

The change in #20269 broke the game "Wargame: European Escalation" for me.

relocation error: /home/freddy/.local/share/Steam/steamapps/common/Wargame European Escalation/Wargame: symbol ENGINE_load_builtin_engines, version OPENSSL_1.0.0 not defined in file libcrypto.so.1.0.0 with link time reference

Technical details

• NixOS unstable

[corngood]

@FRidh Using #20721 I've checked every instance of libcrypto.so.1.0.0 (both 32 and 64 bit) in my built steam-fhs, and they all define OPENSSL_1.0.0.

Could you confirm that you still get this same error message, and if so, we'll probably need some more info (strace or LD_DEBUG?).

[FRidh]

So, something else seems to be going on as well. If I start Steam from my K menu, it remains broken, even after reverting your commit, but if I start it from a shell, it does function. I'll have a closer look at it later.

[FRidh]

I kind of forgot about this issue. Tomorrow I should have some time to check.

[yochai]

Using nixos-unstable (17.03.git.daae1f1) I'm getting the same error on a different game

/home/yochai/.local/share/Steam/steamapps/common/Total War WARHAMMER/bin/TotalWarhammer: relocation error: /home/yochai/.local/share/Steam/steamapps/common/Total War WARHAMMER/bin/../lib/x86_64/libcurl.so.4: symbol ENGINE_load_builtin_engines, version OPENSSL_1.0.0 not defined in file libcrypto.so.1.0.0 with link time reference

[corngood]

@yochai Could you see if #20721 fixes it for you?

[yochai]

@corngood #20721 seems to have no affect on the error for me, still crashing with the same message.

[corngood]

@yochai Are you just launching it normally through steam, using the steam package from nixos-unstable?

Could you check the symbols as I mentioned in https://github.com/NixOS/nixpkgs/pull/20721#issuecomment-263156965? Could you also grep for ENGINE_load_builtin_engines?

[corngood]

I'm guessing it's because we're building openssl with --no-engine. It's there because the steam client is statically linked against a version of openssl with --no-engine, and it needs to be ABI compatible. Perhaps we could remove --no-engine from the 64-bit library? I believe the steam client is still 32-bit?

Edit: here's the bug on steam explaining what's actually happening in the steam client: https://github.com/ValveSoftware/steam-for-linux/issues/4619

[corngood]

Some more threads about the same issue on other distros:

https://github.com/ValveSoftware/steam-for-linux/issues/4537#issuecomment-267264192
https://forums.opensuse.org/showthread.php/520922-Steam-libcrypto-error
http://steamcommunity.com/app/203160/discussions/0/343787920132866394/

They all fix it by removing the 32-bit libcrypto from steam-runtime, but why does that help? Is the steam client 64-bit, and all the games they are testing 32?

[yochai]

nm -D result/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 | grep OPENSSL_1
0000000000000000 A OPENSSL_1.0.0
0000000000000000 A OPENSSL_1.0.1

nm -D result/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 | grep ENGINE_load_builtin_engines
00000000000dc400 T ENGINE_load_builtin_engines

[yochai]

Commenting out line 125,
configureFlags = [ "no-engine" ];
in the openssl file fixed it, the game load now (might face other issues, but unrelated).

[corngood]

So did you need the changes from the PR as well, or did you just need to remove that configure flag?

I'll test it myself when I get a chance. I was already going to buy that game on the humble monthly bundle.

[yochai]

Sorry for the delay, the PR is not needed, removing the configure flag is enough.

[corngood]

I just checked again, and the latest steam client still needs the --no-engine openssl to avoid crashing w/ mesa (https://github.com/NixOS/nixpkgs/pull/20269). I'm not really sure how best to fix this, but there are at least a couple of options that come to mind:

• build mesa with libnettle instead of openssl
  I don't really like this solution because it's not actually a problem with mesa, and it could theoretically happen with any indirect dependency on openssl.

• force only the steam executable to load the --no-engine openssl?
  Could be tricky because the steam client self updates, and games inherit its environment.

[FRidh]

Removing --no-engine solved the issue for me. I did not test in combination with https://github.com/NixOS/nixpkgs/pull/20721.

[corngood]

So I tested reverting 'openssl-steam' and adding the 32-bit openssl from nixpkgs LD_PRELOAD (in the steam /etc/profile). This causes the steam client to immediately load the version with engine support, and fixes the crash w/ mesa.

Doing this results in a bunch of spam like:

ERROR: ld.so: object '/home/david/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.

@FRidh @yochai The PR is #23034. Would you mind testing it with the games you were having trouble with?

Does anyone have a better idea of how to inject the library into the steam client without impacting child processes? Unfortunately we can't use patchelf on the steam client elf, or steam will detect tampering and replace it.

[yochai]

After merging #23034 I got segmentation faults when I tried to start steam, reverting the PR solved it.

[corngood]

@yochai thanks for testing it. Could you pastebin any output you get?

Also, which OpenGL driver are you using?

[yochai]

This is weird, merged and built -> segmentation fault -> reverted and built -> works again -> merged again to get the dump, but now it's working.

I think the change messed with something internal to Steam, as it's working (with the PR merged) now. I will update you if things change.

[yochai]

@corngood after using the PR for the past 6 days, I didn't experienced any farther crashes after the initial problem which was solved by rolling back, starting stream, closing steam, and then going to the version with the PR again.

I failed to reproduce the bug, so I think it might be some internal bug in steam, exposed by the change.
I'm using Nvidia's driver.

[corngood]

@yochai Thanks again. Just to be clear you were testing #23034 and had no problems?

[yochai]

@corngood yes, I will be able to check against latest nixos-unstable with your latest change in a few days, do you think it's needed ?

[corngood]

@yochai I don't think so. But if it gets merged and you have problems afterward, let me know.

[benley]

In theory this should be fixed by #23034.