Jrnl: Analytics on jrnl.sh documentation site?

[micahellison]

I wanted to open a couple questions up for discussion:

• Should we have analytics on the documentation site at jrnl.sh?
• If so, what analytics provider should we use?

Should we have analytics on the documentation site at jrnl.sh?

Pros:

• Depending on the analytics platform, we might get some information regarding issues with the site on certain devices
• It would be interesting to see where users are coming from and what platforms they use
• Depending on the richness of the data, it may give us some clues to triage important issues (e.g. what everyone is searching for when they run into a problem with jrnl)

Cons:

• jrnl is a very privacy-focused project, and tracking users feels antithetical to that
• Even if we trust a particular analytics provider to anonymize visitor data, that doesn't mean all the users do
• It's yet another thing to manage and deal with as a project with multiple maintainers
• There may even be legal implications with certain solutions? (GDPR, etc.)

If so, what analytics provider should we use?

I'm not super familiar with everything that's out there besides Google Analytics but I am tempted to lean away from them and towards a more open solution. Maybe Open Web Analytics?

This conversation was started by @maebert in PR #762 but I wanted to get the conversation out in the issues section.

[ParaplegicRacehorse]

If it queries anything from my browser, or sets a cookie, then I dislike the idea.

That said, I am totally okay with server-log based analytics. You can get much of what you're asking for, particularly the "what people are searching for," from those.

(
and, just because it's a real thing: I haven't been able to install or update jrnl using pip(2 or 3) or pipsi for at least the last two years; up to and including yesterday. I forget the actual error... some compile thing for a dependency.

Analyzing the documents site would not reveal my struggles to manually install.
)

[maebert]

I would like to know what operating systems visitors use, and which parts of the docs are read most - that is data that helps us improve jrnl and the documentation.

I would even like to consider some opt-in in-app analytics; giving the users the option to send anonymous crash reports that include OS, python version and terminal emulator and send it to Sentry (which is open source and has a very high rep in the OSS community). Crash reports would greatly improve the development process - thoughts?

[ParaplegicRacehorse]

I would like to know what operating systems visitors use, and which parts of the docs are read most - that is data that helps us improve jrnl and the documentation.

Most of this should already be available in the server-logs.

Crash reports would greatly improve the development process - thoughts?

Why not just dump the crashlog to local storage with a convenient link to send to Sentry? See, one of the problems with automatic-internet stuffs is ... you have to be online. It also means users don't get to examine and sanitize the output prior to submission.

I may be an exception, but I am rarely online simply because I am rarely in range of a connection-point. Coredump+link means I can choose -- for each individual event -- whether to send the report. It also allows me to know what is being reported, which may help me troubleshoot locally without needing developer help. Auto-filling an email ( mailto:blablabla subj:crashblabla ) with attached dump could be okay, too, because I can still view and edit the report but the pop-up interruption would be horrid. Quietly failing with a coredump is better.

Yes, the extra effort required means you'll get fewer reports. It also means you'll get the reports which users feel most urgently require attention.

I can prevent cookie-setting and most fingerprinting for the website logs; I just don't want to have to. And, if the in-app analytics are truly opt-in, then I guess I'm okay with that in a stomach-wrenching and worrying sort of way. If such a thing is implemented, I'll probably switch to emacs+org-mode for journaling even if it is a little less convenient when emacs isn't already running.

Just my two cents. Maybe this removes me from the target demographic; and that's okay, too. Just consider that most people who are going to journal from CLI are going to be people who are technically competent enough to file their own reports and will most likely have a reasonable idea of what to include in it.

Now that you have mentioned any in-app telemetry stuff, even opt-in, I am now considering dumping it into a firejail or docker container without network access ....

[maebert]

Most of this should already be available in the server-logs.

jrnl.sh is hosted on github pages, so we can't access any server logs unfortunately.

Why not just dump the crashlog to local storage with a convenient link to send to Sentry?

I like this approach - I think we could make it even easier by wrapping everything in jrnl.cli top level with a catch-all, saving it to a log file, and then just asking the user whether they want to send it to sentry so they just have to confirm it if they're online (which I think the majority of folks are), with the option to just open the file later tool

[micahellison]

It sounds like Google Analytics isn't the best direction. I do like the idea of dumping the crashlog to a local file (and perhaps that setting itself should be a config option), however. I've spun that off to a new issue (#771).

[maebert]

While #771 is a great idea, it solves an entirely different issue than GA.

May I suggest we look at https://usefathom.com instead of GA? They have three huge advantages:

• It's privacy first, cookie-free analytics
• It allows you to make the dashboard public, which is great for community projects
• It's not google.

Con is that it's $14/month. I reached out to them but they don't offer discounts for OSS projects :-/ If that looks promising though I'm happy to pay for that (and we can set up a community through https://opencollective.com/ to crowd-fund ongoing costs in the future).

[wren]

I agree that public info, and not Google are definitely the way to go.

I also want to make sure to consider the very private nature of our project whenever discussing any sort of analytics. Any actions we take on the website to gather user info is going to reflect on the project as a whole and its attitude towards user data. I'm not against gathering data, but I want to be very sure that we only collect data that we have a good reason to collect.

For the proposal to collect OS versions, and which docs pages are read the most often:

I agree that the most read pages would be useful in improving the docs. I do want to point out that we have some tickets in the backlog about improving the --help output and adding man screens to jrnl, though. I think the usefulness of the website numbers is probably going to drop a bit once we have that all in place. Either way, though, it would definitely be useful to know which parts of the docs people read the most and improve on those.

The OS info, though, I don't think would be very helpful. The reason is that I don't think the visitors of the site would provide a representative sample of the users of the app. We already get some pretty useful statistics about the app directly from PyPI. And we can go even more in-depth accessing their public dataset* (python versions, country codes, etc). We can also look at Homebrew download stats, and maybe Choco someday but I don't know if their numbers are public (I suspect that the Homebrew numbers will climb once we automate the v2 distribution like we have for PyPI, so that'll give us a better idea of Mac usage).

A thing I do think could be useful for to collect from the site would be browser versions, although that might just be my background. The web dev in me wants a browser matrix. That being said, gone are the days of Internet Explorer; modern day browsers are overwhelming webkit-based (plus Firefox), so maybe a matrix isn't as important anymore. 🤷‍♀️

So, in short, I just don't see the website stats ever being as relevant as the PyPI stats for improving jrnl. Although I do see them being useful for improving the docs, and the docs website itself, which isn't nothing. The question to me is how much are those things worth to jrnl? I'm not sure of the answer to that yet, but $14/month does seem a bit steep to me. For less ongoing cost, we could move the entire site to an S3 bucket and collect those same stats from the logs (it'd be a bit of up-front work to get it moved, but could easily be maintained by our current setup of mkdocs and travis).

While we're on the topic, though, I do find the crowdfunding option potentially useful. There's still the question of how the jrnl.sh domain itself will get paid for year after year (I know that's currently coming out of @maebert's pocket). I've also kicked around some ideas (like doing a user survey to try some of better user metrics in a non-invasive way), that would cost some money. Not a lot, but some. I could--and would be happy to--pay for these, but it feels a bit against the open-source ethos to do it that way. I like the idea of being more open, and transparent, and allowing users to participate in funding the project. And I'd be happy to discuss using something like Open Collective, or Flattr, or whatever to accomplish that goal.

Anyway, that was more long-winded than I initially intended. Thanks for reading, and everyone please let me know what you think!

• Be careful if you use the BigQuery interface to their public dataset! I accidentally got myself an $850 invoice while working on a chart for OS version distribution of jrnl downloads (turns out that charts re-run their queries every time you change an option).

[maebert]

Thanks for the thoughtful response!

I agree that analytics on the web don't really add a lot of marginal value, I'm okay with shelving this for the time being.

As for setting up an Open Collective or Flattr: I'm happy to keep paying for the domain, but in the spirit of transparency it makes sense to at least report these expenses, even without reimbursing them. Happy to set this up if we want to move forward with that!

[wren]

Sounds good. I'll get this issue closed out for now.

For the domain, thank you for continuing to pay for it. I know we don't have a lot of expenses, but I still appreciate the monetary donation to the project. Does that mean that you intend to keep ownership of the domain, too, or are you open to transferring to a shared account for jrnl?

[micahellison]

Hey @maebert, any thoughts on this? It would be nice for something as important as the domain not to be bound to any one person.

[maebert]

Heya, of course open to transfer it to an account shared by the org - or simply create a shared login, too. What's easier?

On Sat, Jan 04, 2020 at 10:47 AM, micahellison < [email protected] > wrote:

Hey @ maebert ( https://github.com/maebert ) , any thoughts on this? It
would be nice for something as important as the domain not to be bound to
any one person.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/764?email_source=notifications&email_token=AAH7U7PSADCIIML6625MG5LQ4DKVHA5CNFSM4JTIDBAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIC5X2A#issuecomment-570809320
) , or unsubscribe (
https://github.com/notifications/unsubscribe-auth/AAH7U7JFGE74OOLKYRVFBHTQ4DKVHANCNFSM4JTIDBAA
).

[wren]

@maebert I think shared login is good for now. I'll email you at the address on your website, if that works for you.

[MinchinWeb]

I came across this recently --> https://github.com/milesmcc/shynet

The project seems to be brand new, but I wonder if it would work for the type of site statistics that were hoped for here.

[wren]

@MinchinWeb seems neat, but also seems like we would need control over the server (as opposed to continuing to use GitHub pages). Worth keeping an eye on, though.

[fnune]

I use Goatcounter for my personal website. It's admittedly limited in functionality but it's private.