Activitywatch: Compiling on Windows

[the-architech]

Hey Guys,

I am attempting to compile from source on a Windows machine, however, running into some issues. At the moment I am compiling using the following commands:

cd .\aw-watcher-afk\
poetry shell
poetry install
pyinstaller aw-watcher-afk.spec --clean --noconfirm

No errors are prompted and everything appears to build correctly. However, when I try to launch the executable, I get the following error.

Traceback (most recent call last):
  File "aw_watcher_afk\__main__.py", line 3, in <module>
ModuleNotFoundError: No module named 'aw_core'
[21392] Failed to execute script __main__

I have not used MinGW yet, however, my thought behind this was that the make file is now using poetry so I should not need to do make build and make package?

Please note, this is the raw source downloaded from github, no changes made have been made to code.

How does the Activitywatch team build the windows packages? From what I can find in posts, it looks like you compile on Linux but use Wine to emulate a Windows environment?

[johan-bjareholt]

The hint what is going wrong is right there in the log, aw-core can't be found. Not sure though why "poetry install" didn't install the dependencies itself (or simply can't find them?).

You are correct that using the Makefiles should not be necessary, they were actually not necessary before either with setuptools.

We are not using wine to build the windows executable, the releases are built in a windows VM in AppVeyor.

[xylix]

Did you run poetry install in the project root? The sub-application pyinstaller executables arent really configured or tested to work alone, the normal way to package them can be read from https://github.com/ActivityWatch/activitywatch/blob/master/scripts/package/package-all.sh

[johan-bjareholt]

The sub-application pyinstaller executables arent really configured or tested to work alone

They were before, I believe that they still should be. Would be nice if we could fix this if it's broken.

[xylix]

Oh yeah I agree that it should be possible to run them. I was just throwing ideas around to try to fix the original problem in this issue.

As another possibility, @the-architech are you trying to run the executable in the shell with the poetry virtualenv active or in a new shell / through gui?

If the dependencies (including aw-core) are installed in the virtualenv then they cant be found outside it. I don't think we pack our internal dependencies into every pyinstaller executable since there would be quite a lot of overlap then in the application executable. You could try to confirm this by looking into the pyinstaller created directory and checking which files beginning with "aw" exist there.

[the-architech]

When the build finishes I have a /dist directory. When navigating inside there is the aw-watcher-afk executable. I execute the watcher from the Powershell and it throws that error. When I launch it from the GUI, it shows a console for one seconds and closes.

I tried to copy my compiled executable to the prepackaged windows release for 0.9.2 that was published yesterday and I get the same error.

Also, there is a size difference between the files 1981 vs 1579, my compiled version being smaller.

I did not run the poetry install in the directory root, just for aw-watcher-afk for now.

[ErikBjare]

@the-architech If you run poetry add git+https://github.com/ActivityWatch/aw-core.git then rerun the pyinstaller command that should fix it.

Not sure why pip/poetry doesn't pick up the aw-core dependency from aw-client, I guess it should be added to the pyproject.toml regardless.

[the-architech]

I may be doing something wrong here. But after running suggested command and running the pyinsatller command I still get the same error.

[xylix]

Did you try to execute the watcher with the virtualenv (so poetry shell) activated?

[the-architech]

I never left the poetry shell, but I tried to pull a fresh copy of the source in a new directory.

Commands I used:
git clone --recursive https://github.com/ActivityWatch/activitywatch.git
cd aw-watcher-afk
poetry update
(I am using python 3.7 so I need to use pyhook37). This was changed in the toml but it should make no difference from my understanding.
poetry shell
poetry env use python
Using virtualenv: C:\Users\the-architech\AppData\Local\pypoetry\Cache\virtualenvs\aw-watcher-afk-32LmHvY6-py3.7
poetry install
Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

• Updating pyHook (1.5.1 -> 1.5.1 https://github.com/the-architech/aw-packages/raw/master/pyHook-1.5.1-cp37-cp37m-win_amd64.whl)
• Installing aw-watcher-afk (0.2.0)

poetry add git+https://github.com/ActivityWatch/aw-core.git
pyinstaller aw-watcher-afk.spec --clean --noconfirm

This time I get some errors:

2875 ERROR: Hidden import 'Xlib.keysymdef.miscellany' not found
2876 INFO: Analyzing hidden import 'Xlib.keysymdef.latin1'
2879 ERROR: Hidden import 'Xlib.keysymdef.latin1' not found
2879 INFO: Analyzing hidden import 'Xlib.keysymdef.latin2'
2880 ERROR: Hidden import 'Xlib.keysymdef.latin2' not found
2881 INFO: Analyzing hidden import 'Xlib.keysymdef.latin3'
2882 ERROR: Hidden import 'Xlib.keysymdef.latin3' not found
2882 INFO: Analyzing hidden import 'Xlib.keysymdef.latin4'
2883 ERROR: Hidden import 'Xlib.keysymdef.latin4' not found
2883 INFO: Analyzing hidden import 'Xlib.keysymdef.greek'
2884 ERROR: Hidden import 'Xlib.keysymdef.greek' not found
2885 INFO: Analyzing hidden import 'Xlib.support.unix_connect'
2886 ERROR: Hidden import 'Xlib.support.unix_connect' not found
2886 INFO: Analyzing hidden import 'Xlib.ext.shape'
2889 ERROR: Hidden import 'Xlib.ext.shape' not found
2889 INFO: Analyzing hidden import 'Xlib.ext.xinerama'
2890 ERROR: Hidden import 'Xlib.ext.xinerama' not found
2890 INFO: Analyzing hidden import 'Xlib.ext.composite'
2891 ERROR: Hidden import 'Xlib.ext.composite' not found
2891 INFO: Analyzing hidden import 'Xlib.ext.randr'
2892 ERROR: Hidden import 'Xlib.ext.randr' not found
2892 INFO: Analyzing hidden import 'Xlib.ext.xfixes'
2893 ERROR: Hidden import 'Xlib.ext.xfixes' not found
2893 INFO: Analyzing hidden import 'Xlib.ext.security'
2894 ERROR: Hidden import 'Xlib.ext.security' not found
2894 INFO: Analyzing hidden import 'Xlib.ext.xinput'
2895 ERROR: Hidden import 'Xlib.ext.xinput' not found

Still googling those errors. Will provide an update once I find anything.

[xylix]

Those errors look to me like "normal ones" on our windows builds https://ci.appveyor.com/project/ErikBjare/activitywatch/builds/31659221#L3107 . Altough we should probably do something to suppress them since they take attention away from the actually breaking errors and warnings. Xlib is the library we use for window watching on linux, and it shouldn't be necessary on windows since on windows we use wmi, win32gui and win32process for window watching.

On this latest build did it also error afterwards?

[the-architech]

It did error with the same output after I try to launch it from powershell still within the venv and from the GUI.

One major difference I see between my build and the packaged build you guys provide is that there are files missing from the /dist directory. I will have to account for those and share the names of the missing files.

[the-architech]

So I tried using cygwin, used the same process that I had used on my Linux box, which had worked. I installed all required packages, used Python3.6 this time, and performed the following commands after I cloned the repo:

poetry shell
poetry install
poetry use python #redundant I know but just making sure
make build
make package

No errors during build process, when I tried to execute aw-watcher-afk still got the same error.

Off to try WSL2 now on Windows.

[the-architech]

Well should have probably tried this from the start but here is what I get when I try to execute __main__.py directly from source.

.\__main__.py
Traceback (most recent call last):
  File "C:\Users\the_architech\Documents\activitywatch\aw-watcher-afk\aw_watcher_afk\__main__.py", line 3, in <module>
    from aw_core.log import setup_logging
ModuleNotFoundError: No module named 'aw_core'

PS. WSL2 did not work either

[ErikBjare]

Weird. Can you go into activitywatch/aw-core (while in the same poetry shell), run poetry install there (after having run poetry install in aw-watcher-afk), and then run python3 -m aw_watcher_afk? (or however the entrypoint is set up)

[the-architech]

I got it to work guys. Looks like spawning a shell within aw-watcher-afk does not work. However, spawning a shell within activitywatch, running poetry install, then going into aw-watcher-afk, running a poetry install there too, and finally running pyinstaller worked for me.

I was able to execute the aw-watcher-afk.exe without issues.