Proton 3.7-4 beta - Games don't start anymore

[dragonfly-lover]

i have switched from proton 3.7.3 to 3.7.4 beta, but all "unsupported" games don't start anymore (Gas guzzlers extreme, call of juarez, PES 2018...)

The one that starts is Tekken 7 which was in the official list.

Python 2.7 and 3 packages are installed (verified using synaptic) as they were before but nevertheless it's probably something python related. Switching back to proton 3.7.3 games start again.

System
Steam client version: 25/08/2018
Distribution: Ubuntu 18.04 LTS
Opted into Steam client beta?: Yes
Have you checked for system updates?: Yes
CPU: AMD Ryzen5 1500
RAM: 8GB
Video Card: NVIDIA GTX750ti
Driver version: NVidia 396.54
Video memory: 2GB

[ProPanek]

Yeah i have the same problem but even in the 3.7.3 version. Trying to launch MONSTER HUNTER: WORLD and zero reaction. I don't even have the black screen as others. Just nothing after pressing Play button.

Edit: I heard that i need a python2, so i need to check it out.

[nstgc]

So far I've not had any more problems with 3.7.4 compared to 3.7.3.

[dubigrasu]

I can confirm that I can no longer start some games (Bioshock 2 for example) with 3.7-4 beta.
The moment I switch back to 3.7-3, it starts working again.

The rest of my tested games are unaffected.

[Zero86Sk]

For me Pac Man Museum didnt run on 4.7.3 but does on 4.7.4
but it can be i maybe just needed to perform a restart.

[Elkasitu]

You should post your system python version to see if there's a pattern

[nstgc]

Python 3.7.0
Python 2.7.15

[dragonfly-lover]

May It Be a distro-specific issue regarding python path? Which distro are you using? I use plane Ubuntu repos on Bionic (18.04).

[ProPanek]

After installing python2 all is working as expected.

@dragonfly-lover Im using the same distro as you.

[silelmot]

can confirm, that all VR games dont start anymore

[aeikum]

Can someone who is experiencing this please set the launch options for an affected game to PROTON_LOG=1 %command%, try run the game, and attach the resulting $HOME/steam-*.log file here? If no log file is generated, please launch Steam from a Terminal and watch out for any Python or Proton errors in the Steam output.

[bumbar1]

With proton 3.7.4-beta I'm getting a black screen with ERR01: Failed to create DirectInput device. Attached a log below:
steam-582010.log

Using nvidia 396.54 drivers with 4.15.0-33-generic #36~16.04.1-Ubuntu kernel.

[aeikum]

@bumbar1 344965.134:0008:002b:err:module:load_builtin_dll failed to load .so lib for builtin L"dinput.dll": /home/user/.local/share/Steam/steamapps/common/Proton 3.7 Beta/dist/bin/../lib64/wine/dinput.dll.so: undefined symbol: SDL_JoystickGetVendor

That function exists in the runtime version of SDL. Did you disable the Steam runtime?

[bumbar1]

I'm not sure, it's possible I did something but I've been playing around with getting gamepads to work and have been (re)installing some packages, amongst others were: steamos-xpad-dkms, xinput, xpad, xboxdrv. How would I check that?

[kisak-valve]

Hello @bumbar1, the Steam Runtime Version line in Steam's system information (Steam -> Help -> System Information).

[bumbar1]

system info as requested

[btegs]

I don't understand why there isn't a standard bash script that checks for Python 3 and if it doesn't work, look for Python 2 and run the .py script with that interpreter. You can also add code to Python script to check the platform and version easily:

from platform import python_version
python_version()

Or if you want to get the main release (2 or 3)

from sys import version_info
version_info[0]

[ghost]

@aeikum I tried to use PROTON_LOG=1 %command% but no log is created when I tested mount and blade. Running Steam from terminal using steam steam://run/22100 only shows:

Running Steam on ubuntu 18.04 64-bit
STEAM_RUNTIME is enabled automatically
Pins up-to-date!

For the purpose of this test I removed Python 2.7 but if I re-add it, it works fine. Python 3.6.5 is installed as referenced in #772 .

[aeikum]

@QuirkyTechie If PROTON_LOG=1 doesn't create a log, but the game is running, then you are not using the Proton 3.7 Beta channel. Can you recheck your Steam Play settings and restart the Steam client?

As for checking the output of Steam itself, be sure you launch Steam itself from a terminal. The protocol handler just forwards the info on to the main Steam process and exits, so you won't get useful terminal output from just that.

[aeikum]

@bumbar1 I understand your issue. Your system SDL installation is too old. If you can upgrade SDL to at least version 2.0.6, that should fix this issue. I also hope to have a workaround in the next Beta of Proton.

[bumbar1]

@aeikum upgrading SDL to 2.0.6 fixed the issue; game works now, thanks!

[ghost]

@aeikum I am on the beta 3.7-4, at-least it is selected in the SteamPlay settings. The logs are created when Python 2.7 is installed and working because the game will launch. Once Python 2.7 is removed the games fail to launch and therefor no logs are created even though Python 3.6.5 is installed which should be enough according to the beta changelog.

The game immediately exits (goes to sync cloud saves then nothing) right after you press play. Tried with 3 supported SteamPlay games.

Just running steam from the terminal and trying to play also doesn't produce much either, no Python errors are shown. I ran steam and clicked play on the games and then exited and this is all it shown:

Running Steam on ubuntu 18.04 64-bit
STEAM_RUNTIME is enabled automatically
Pins up-to-date!
[2018-08-28 14:21:16] Startup - updater built Aug 27 2018 19:01:45
[2018-08-28 14:21:16] Opted in to client beta 'publicbeta' via beta file
You are in the 'publicbeta' client beta.
[2018-08-28 14:21:17] Verifying installation...
[2018-08-28 14:21:17] Verification complete
html_chrome.cpp (394) : Assertion Failed: CefCrashReportingEnabled()
[2018-08-28 14:24:22] Shutdown

Again if I install Python 2.7 and then play the logs are created fine (because the game starts) and are stored in $HOME/steam-$APPID.log which would signify I am on the 3.7-4 beta.

So based on my testing, 3.7-4 is still depended on Python 2. My issue isn't exactly the same as @dragonfly-lover but without Python 2 my games crash which is similar.

[Elkasitu]

@QuirkyTechie can you try manually replacing your proton script with the one here on github?

[aeikum]

@QuirkyTechie Hmmmmm I'm really puzzled by this. Do you get output similar to this if you just try to run the proton script directly?

[aeikum@aeikum ~]$ python ./.steam/steam/steamapps/common/Proton\ 3.7\ Beta/proton
Proton: No compat data path?

[ghost]

@aeikum If I run:

$ python3 ./.steam/steam/steamapps/common/Proton\ 3.7\ Beta/proton

Yes I get the same as you do: Proton: No compat data path?. Using just python doesn't work because of no python 2 installed.

[ghost]

@Elkasitu Thanks for trying to help. I tried that but Steam re-downloads... something after I replace it, not sure exactly what it's downloading but it still does the same regardless.

If you or @aeikum or anyone wants to test this. I'm using Ubuntu 18.04 (with a minimal install) which should only include Python3 by default. Install Steam and switch to the beta 3.7-4 beta, then install Fallout Shelter or something. See if you get the same.

[Elkasitu]

@aeikum I think I figured out what the issue is from @QuirkyTechie's latest post: for some distros, /usr/bin/env python is reserved for python 2, therefore if your only python installed is python 3 then/usr/bin/env python doesn't exist, and python cannot be found when parsing the shebang line.

I think this is more of a distro issue, /usr/bin/env python should always exist as per PEP 394 [1]

Whether it should point to python2 or python3 is debatable

[1] https://legacy.python.org/dev/peps/pep-0394/

[ghost]

@aeikum Yep that is it. @Elkasitu is correct.

I changed #!/usr/bin/env python to #!/usr/bin/env python3 in proton.py and it worked without issue. I probably should have noticed that but I'm far more familiar with Windows and haven't wrote anything with Python in years. But yes I can confirm that is the issue.

Thanks by the way.

[aeikum]

Siiigh. Paging @MayeulC and @nanonyme as they lead this effort and may have some suggestions here.

[nanonyme]

That breaks compat with macOS as mentioned earlier
Edit: was talking about python3 hashbang

[nanonyme]

@aeikum I don't see big chances of having a single hashbang that works everywhere at this point based on conversation. I suggest for now prioritizing breakages and making code and documentation reflect that for now.
I think currently
1) It should work on macOS
2) It works on any system with Python 2 and has /usr/bin/python point to it
3) It works on any system that calls /usr/bin/python Python 3
4) It does not work on systems that unexpectedly just removed /usr/bin/python link without doing anything else
5) It does not work on systems that removed Python 2 and kept calling /usr/bin/python Python 2
Note that in this category you can always install Python 2 to fix the problem

Different prefixes allowed, /usr/bin is example for clarity

[nanonyme]

Given the thid-party deps a shim would probably not be practical so it's probably best to keep declaring Python 2 requirement even though it might work on some Python 3 setups

[pchome]

FYI: in e.g. Gentoo https://github.com/mgorny/python-exec wrapper used as /usr/bin/python. Preferred versions controlled by https://github.com/mgorny/python-exec/blob/master/config/python-exec.conf.example

_I have 2.7 and 3.6 installed, and 3.6 manually set as default. No issues with proton._

[nanonyme]

@pchome can you confirm that Gentoo hit category 4?

[nanonyme]

@btegs Probably because in general that would work poorly. Python has importable modules and you might end up importing a long chain of modules and compiling bytecode until you found out you have the wrong interpreter. Worst case is the code is syntax compatible and you only hit the issues during runtime in some conditional branch

[pchome]

can you confirm that Gentoo hit category 4?

Emm... It sounds to me like "you unexpectedly destroys your system and Steam don't work then", or I misunderstand something.

In gentoo you should remove all packages which depends on an python version before you can remove python version itself. And you can't completely remove python (all python versions) because system tools require python, and therefore it will stop functioning.

I think broken systems is not Steam/Proton responsibility.

[nanonyme]

Reworded. @pchome

[pchome]

As for system behaviour -- there will always be at least one python version. And they should fallback as described in mentioned config example.

But to be sure -- I just manually removed /usr/bin/python and a game don't launched from steam this time (that was expected).

[dragonfly-lover]

Forgive me, but i'm not skilled in python,

So, to summarize, please correct me if i'm wrong:

1) the problem affects (at least) ubuntu based distributions, while for gentoo it's different

2) As @Elkasitu reported, the problem is that _ "for some distros, /usr/bin/env python is reserved for python 2, therefore if your only python installed is python 3 then/usr/bin/env python doesn't exist, and python cannot be found when parsing the shebang line."_

3) according to @QuirkyTechie _"I changed #!/usr/bin/env python to #!/usr/bin/env python3 in proton.py and it worked without issue. "_

I have to try by myself but would like to know which is the correct path of proton.py

thanks

[pchome]

You probably could use

#!/usr/bin/env -S ${python_version}

by previously testing python and setting python_version= variable.

[Elkasitu]

Workaround proposed at #898, if anyone experiencing these issues could test it that'd be great.

[aeikum]

I'm more tempted to just ship a different proton script on macOS vs Linux. On mac call python2, on linux call python3. We can depend on python3 being installed on any modern distro...... right?

[Elkasitu]

You can maybe depend on a python3 package being available on most if not all linux distributions, but I don't think you can depend on it being installed by default, especially for LTS / minimal releases.

[aeikum]

@Elkasitu I just checked on a fresh installation, and Ubuntu 18.04 LTS ships python3 by default, but no python2 or python binary.

[nanonyme]

Note that there were separate changes done to your deps (ffmpeg and so) that you need to check if you do that

[nanonyme]

It might make sense to make Steam distro integrations pull in Python 3

[aeikum]

@nanonyme Those are just used at build-time, so I'm not too broken up if they require a specific python setup.

[ghost]

@Elkasitu Just an FYI, the change made in #898 does appear to work, based on a quick test I did in a VM.

[aeikum]

Pushed 5286f43fda79c732c3fe979dd2058fdae445b990 to fix this.

[MayeulC]

Sorry for being late to the party. I really don't have much time right now, but I think I would have relied on making the proton executable a shell script instead of a python one. This shell script could then determine at runtime which python version to use, and launch the proton payload. I'll be able to explain the rationale and put up a proof-of concept at a later date (~1 day), if you think that it would be an acceptable solution.

[nanonyme]

@MayeulC that's essentially what @aeikum did except this is a build-time differentiation that results in Python2 only on macOS

[Elkasitu]

@MayeulC what you propose has been done at #898

I think @aeikum's solution is good nonetheless, if a big enough number of people complain about incompatibility with python2 on linux, then the solution could be revised

[nanonyme]

@aeikum what's your opinion as a maintainer of this? Is the behaviour in the new PR more desirable than current one?

[aeikum]

@nanonyme I'm willing to defer to more experienced python programmers, but I feel like our explicit use of python3 is fine. Adding a bash script to handle a couple of annoying distro decisions seems like an unnecessary complication, especially given how common python3 is these days. We should document this dependency on the PREREQs page, but I'm waiting for the Beta to be moved to the Stable branch to do that.

[aeikum]

Long since fixed, and now documented on the Wiki pages.