Activitywatch: Terminal watcher - bucket and event design

[Otto-AA]

I've been working on a terminal watcher in the last days and need some inspiration on the bucket and event design. (you can view the WIP repo here)

The reason why I made a terminal watcher was mainly, because VS Code has an integrated terminal, but doesn't seem to provide any API for tracking activity in it. Hence, tracking via the aw-watcher-vscode extension as well as using the aw-watcher-window data both won't work and I have a rather significant loss of activity data (I guess about 1/3 of VS Code time).

---

The current implementation of the terminal tracker sends an event to the server every time a command is being executed in a (bash) terminal. This way following data could be tracked:

• command + args
• path of the terminal
• execution time of command
• any environment variable
• anything else you can get with bash
• (with different implementation) outputs of the command

Using this data, I can think of a few ways to use them:

1. Track my personal interaction with the terminal
2. Track the frequency and duration of commands I've used (in particular backup and compiling commands would interest me)

While for the first use case, working with heartbeats seem the better solution, the second one would seems suited for events. That's why I would consider splitting these two into separate buckets.

---

Independent from that, I think it could be useful to create a general terminal watcher with several platform specific watchers attached to it. So for example aw-watcher-terminal-bash only writes some things to a named pipe and aw-watcher-terminal forwards this to the server. But this could be changed later and is not necessary to discuss now from my point of view.

[johan-bjareholt]

Awesome, have beent thinking about something similar but never started working on it. @ErikBjare made a WIP tmux watcher long ago which is completely broken now but aimed at tracking terminal activity aswell https://github.com/ActivityWatch/aw-watcher-tmux

The current implementation of the terminal tracker sends an event to the server every time a c command is being executed in a (bash) terminal. This way following data could be tracked:

• command + args
• path of the terminal
• execution time of command
• any environment variable
• anything else you can get with bash
• (with different implementation) outputs of the command

Let's just make sure to not track data which is rarely useful and can take up a lot of space (I am primarily thinking about "any environment variable").

Using this data, I can think of a few ways to use them:

1. Track my personal interaction with the terminal
2. Track the frequency and duration of commands I've used (in particular backup and compiling commands would interest me)

While for the first use case, working with heartbeats seem the better solution, the second one would seems suited for events. That's why I would consider splitting these two into separate buckets.

How would the actual interaction be different from the commands themselves? To me personally it isn't important "how" I interacted with the terminal, only what command I decided to actually run (but maybe you have another use-case?). In my case at least it would only make sense to track the durations of commands and send them as events, heartbeats wouldn't work since I can have multiple terminals open running different commands in parallel.

Independent from that, I think it could be useful to create a general terminal watcher with several platform specific watchers attached to it. So for example aw-watcher-terminal-bash only writes some things to a named pipe and aw-watcher-terminal forwards this to the server. But this could be changed later and is not necessary to discuss now from my point of view.

This sounds like a great idea to me! :)

I have a couple of questions though:

• Is the bash watcher compatible with zsh? I see that you use bash-preexec which is pretty much a clone of a built-in feature in zsh but I don't know if they are compatible.
• What happens if a bash script is being run, will the watcher still send each command in the script as if it was entered interactively?

[Otto-AA]

To me personally it isn't important "how" I interacted with the terminal, only what command I decided to actually run (but maybe you have another use-case?)

I meant to track how long I've been using the terminal, the time I've spent writing commands in the terminal (contrary to the time the commands needed to execute). Nearly the same could be achieved via the window watcher, yet this one does not track integrated terminals (e.g. in VS Code).
For me both seem interesting, so I'd be in favor of implementing both :)

What happens if a bash script is being run, will the watcher still send each command in the script as if it was entered interactively?

No, bash-preexec seems to only support interactive mode.
Don't yet know of an easy way to record in-script commands (using bash --verbose myscript.sh or script -c myscript.sh _could somehow_ work)

Is the bash watcher compatible with zsh? I see that you use bash-preexec which is pretty much a clone of a built-in feature in zsh but I don't know if they are compatible.

From looking at this explanation I am not sure if the preexec function is interactive-mode only. I guess so, but I'm not entirely sure.
Apart from that, I've tried zsh <<< 'aw-watcher-bash "echo \"Greeting from zsh\""' which works (except for also outputting 'bash' as the shell which could easily be fixed). So only the part in .bashrc would need to be changed.

Let's just make sure to not track data which is rarely useful and can take up a lot of space

Currently I am tracking following:

• command + args
• path
• shell
• shell version (actually dunno if that is helpful, maybe if some are using multiple versions of the same shell)

Regarding the shell-independent aw-watcher-terminal, I'd be glad if someone could help me with the (cross platform) installation process. The current setup is definitely not user-friendly :confused:
I will split aw-watcher-terminal and aw-watcher-bash shortly.

[Otto-AA]

Is the bash watcher compatible with zsh? I see that you use bash-preexec which is pretty much a clone of a built-in feature in zsh but I don't know if they are compatible.

Yes, I've tried it out :smiley:

Just add the same without the bash-preexec part to .zshrc

# Send data to local ActivityWatch server
preexec () {
  # Call aw-watcher-bash in a background process to 
  # prevent blocking and in a subshell to prevent logging
  (aw-watcher-bash "$1" "zsh" "$ZSH_VERSION" &)
}

[johan-bjareholt]

I meant to track how long I've been using the terminal, the time I've spent writing commands in the terminal (contrary to the time the commands needed to execute). Nearly the same could be achieved via the window watcher, yet this one does not track integrated terminals (e.g. in VS Code).
For me both seem interesting, so I'd be in favor of implementing both :)

Aha, yeah that's annoying.

No, bash-preexec seems to only support interactive mode.

Well that's the behaviour we probably want anyway, so that's good :)

Currently I am tracking following:

• command + args
• path
• shell
• shell version (actually dunno if that is helpful, maybe if some are using multiple versions of the same shell)

Seems like a sane place to start to me!
Might be a good idea to replace shell version with the name of the terminal emulator in use though, that should be more interesting IMO.

Regarding the shell-independent aw-watcher-terminal, I'd be glad if someone could help me with the (cross platform) installation process. The current setup is definitely not user-friendly confused
I will split aw-watcher-terminal and aw-watcher-bash shortly.

Since aw-watcher-terminal also depends on aw-core and aw-client we should definitely set up a normal setup.py install script for it. After that you should only need:

• one "pip install" command for aw-watcher-terminal
• install preexec
• move the .sh file somewhere
• Add preexec to your .profile script of choice

Since people who use the terminal are usually a bit more experienced that shouldn't be too bad.

[Otto-AA]

What do you think about leaving the event design (at least a bit) open to the user?

I am currently thinking about parsing all unknown given arguments into the event data. So everyone could add custom data like the terminal emulator name, the shell version, the current battery charge or whatsoever.

In practice it should easily be possible by separating the required and custom args with --

echo '--event preexec --pid 1234 [...other required args] -- --battery-charge 5 --terminal-emulator gnome-terminal > $fifo_path

I'd also consider adding a --bucket bucket_id option and using a different event handler which has no required args at all:

echo '--event custom-event --bucket my-very-own-bucket_{host} [...custom args]' > $fifo_path

With these two options, it would be rather easy for developers to quickly make trackers on their own. It also has the advantage of being language independent, as it uses a named pipe as input. This probably exceeds the purpose of aw-watcher-terminal but should be easy to implement there.

[johan-bjareholt]

With these two options, it would be rather easy for developers to quickly make trackers on their own. It also has the advantage of being language independent, as it uses a named pipe as input. This probably exceeds the purpose of aw-watcher-terminal but should be easy to implement there.

True, but some downsides/TODOs would be:

• No windows support for pipes (I believe atleast, not sure)
• We need to create a event which can create the bucket before passing --bucket
• We need to pass eventtype aswell when creating the bucket
• No support for heartbeats which is more reliable (but not possible in aw-watcher-bash, so it's fine here)

echo '--event preexec --pid 1234 [...other required args] -- --battery-charge 5 --terminal-emulator gnome-terminal > $fifo_path

Notes on this example:

• Each argument should be quoted and contents escaped for safety
• Sending custom data (such as --battery-charge) with a preexec command to a bucket with terminal activity doesn't make sense, better to make a seperate watcher and bucket for that and then make a query afterwards when visualizing if you want to merge the data somehow.

IMO I could see the benefit of this being language agnostic, on the other hand it will be cumbersome to make it a high-quality general-usecase client. HTTP requests are also language agnostic and the HTTP API is rather simple so I believe it would be better to just utilize that instead of having yet another layer.

[Otto-AA]

Yeah, this definitely has some downsites. I also wouldn't put it on the same level as aw-client or aw-client-js. From my point of view this would function as client for quickly developed watchers (in languages other than python and js) which don't necessarily need to be stable (and definitely not published to the activitywatch repo).

The downsides I see are:

• no direct success/error response
• no get functions (get_events, get_buckets, ...)
• maybe a bit work to adopt to windows
• hard to implement objects / lists for event data

The advantages I see:

• quick creation of custom watchers
• no need to dive into documentation, as this is pretty much a one-liner
• as far as I know, in most languages writing to a file is easier than making a http request. Probably making the writing asynchrounous negates this advantage

Regarding the disatvantages ones you mentioned:

No windows support for pipes (I believe atleast, not sure)

Windows at least has named pipes. But I don't know if they are created the same way in python as on linux

We need to create a event which can create the bucket before passing --bucket

We could make a dict of already created buckets. Then we can check each time an event occurs and if it's not in this dict, then create the bucket

We need to pass eventtype aswell when creating the bucket

Either that or using something like event.custom

No support for heartbeats which is more reliable (but not possible in aw-watcher-bash, so it's fine here)

Not yet, but should be possible to add.

We could also create several fifos for the normal terminal watcher, custom-events and custom-heartbeats.

So the updated message sent to the fifo could look like this:

--event "custom.my.event.type" --bucket "some-name_{host}" -- --custom-arg "val" --custom-arg-two "val 2"

---

Each argument should be quoted and contents escaped for safety

I know, I just thought I'd drop the quotes to keep it shorter (and everything except potentially $fifo-path works without quotes)

Sending custom data (such as --battery-charge) with a preexec command to a bucket with terminal activity doesn't make sense

I agree, that wasn't a good idea -_-

[johan-bjareholt]

The advantages I see:

• quick creation of custom watchers
• no need to dive into documentation, as this is pretty much a one-liner

This client is not any simpler than the normal clients and does not seem easier to learn IMO. You still need to understand the concept of a bucket, the format of a event, heartbeats etc.. In a normal client lib you have to do 2 essential steps, create a bucket and send events/heartbeats. The only difference this client does is that it does these two things in one step, but the input is still the same which is why it is conceptually not much easier to understand. You still need to dive into documentation to understand how buckets and events works. Back before the 0.7 activitywatch release our heartbeat API actually contained the bucketname similar to how you do it here (but in aw-server instead of in the client), but decided against it after:

• We realized that we needed a buckettype and possibly more backwards-compatible fields in the future
• it was confusing to have two ways to create a bucket through the HTTP API
• When we started adding HTTP error handling there would be a lot of possible errors in the heartbeat API, it became simpler when splitting the functionality and make the heartbeat API only do a single thing

• as far as I know, in most languages writing to a file is easier than making a http request. Probably making the writing asynchrounous negates this advantage

This is the only advantage IMO but it comes with the cost of no error handling which will make the debugging very annoying if you don't get the format exactly right from the start.

--event "custom.my.event.type" --bucket "some-name_{host}" -- --custom-arg "val" --custom-arg-two "val 2"

Also, why not use a format people are used to? In aw-watcher-vim we are doing a similar thing as here but do it through a stdout pipe rather than a named pipe, but there we simply just send JSON since python has a built-in json parser so we don't need to implement the parsing ourselves. A not thought through example would be:

{"bucket": {"name": "bucketname", "type": "eventtype"}, "data": {"key": "val"}}

This would also allow for not only integer and string values, but also arrays and objects.

[Otto-AA]

Yeah ok, seems like it's not as good as my imagination promised -_-

[Otto-AA]

So, if we don't make a custom event design, we gonna need to define a standard.

I'd suggest following. Feel free to add/scrap something :)

For events:

• shell_name
• terminal_emulator_name
• path
• command
• exit_code
• [duration equal to command execution duration]

For heartbeats:

• shell_name
• terminal_emulator_name
• path

Bucket Ids:

• aw-watcher-terminal-commands_{host}
• aw-watcher-terminal-activity_{host}

Event types:

• app.terminal.command
• app.terminal.activity

[johan-bjareholt]

The format looks good IMO. Personally I don't care about terminal_emulator_name and it will just take up DB space but I'm sure there are people who run different terminal emulators at the same time for some reason and might find this valuable so let's keep it.

It might be a good idea to be able to distinguish sessions with PIDs though and use then as unique identifiers. Here's a usecase where it could become a bug without it:

• I open terminal1 and start htop in my home folder and keep it running
• I have another workspace and open terminal2 and keep it running at the same time
• I close htop in terminal2 which will send that htop in the homefolder has finished executed
• I close htop in terminal1 and according to aw-watcher-terminal htop has not even started and can therefore not know the duration

By identifying how each process starts/exits with PIDs this bug can be fixed. We probably shouldn't send the PID to aw-server though since it's just an arbitrary number.

EDIT: Is it even possible to get the PID in preexec though?

[Otto-AA]

By identifying how each process starts/exits with PIDs this bug can be fixed. We probably shouldn't send the PID to aw-server though since it's just an arbitrary number.

Exactly my intention and the way it is implemented :smile:

I only didn't add it to the format, as this will not be stored on the server.

The event format for sending to the pipe is (in the current implementation):

Required for all following events

• --event
• --pid
• --time
• --path

Preopen:

• [base params]

Preexec:

• [base params]
• --command
• --shell

Precmd:

• [base params]
• --exit-code

Preclose:

• [base_params]

I will add documentation regarding this format as soon as it's finished (and I have time to make it)

[johan-bjareholt]

Awesome! :)

[Otto-AA]

What do you think about restoring sessions? Like writing all commands executed at a specific time in a specific terminal into a script file?

If you think this is useful, we could store the pid as well as terminal-created and terminal-closed events on the server to make this possible.

I believe it would be risky to execute the whole script without looking through it, but for copying parts out of it, it _could_ be useful...? It would be kinda like bash_history with some metadata to search for.

[johan-bjareholt]

Well, if you could get a unique id of the session (the process id of the bash/zsh process maybe?) that would be trivial to fetch afterwards. Could be pretty cool.

[Otto-AA]

I think I'll leave terminal_emulator_name as it doesn't seem that easy to get this information and I definitely don't know if it is possible for many shells

[ErikBjare]

Can I close this as I assume the questions have been answered? :slightly_smiling_face: