Jrnl: The future of jrnl

[conor-byrne]

Just wondering where we are in terms of the 2.0 release? The latest mention I can see is #339.

Thanks!

[AdrienLemaire]

@maebert seems more busy with other projects... I wonder if he's still journaling :stuck_out_tongue:

Looking at the progress of https://github.com/maebert/jrnl/compare/2.0-rc1, I see:

• 4 commits in 2017 from @radomirbosak
• 3 commits in 2018 from @imjennyli, @emceeaich and @hoclun-rigsep
• 1 commit in 2019 from @wren

I was thinking of working on jrnl features that I care about, like daily streak report (maybe heatmap), and journal name bash autocompletion, but the lack of progress on this project is demotivating. The v2 was waiting for more tests in 2015, and ... still waiting?

[wren]

It does seem that @maebert has somewhat moved on. He seems to still occasionally check this repo, though, as evidenced by my PR he merged in recently. So, I'm not completely sure what to think.

[wren]

@maebert Are you open to discussing adding more maintainers to this project? I've reached out on Twitter, but I'm not sure the best way to contact you.

[zorab47]

As a tool I use daily with a simplicity I love, I second adding more members to help maintain the project.
My desire to journal more frequently was rejuvenated by jrnl years ago. Thank you to @maebert for creating it!

❤️ 🙏 ❤️

[maebert]

Hey everyone, thanks for your support and enthusiasm! I'm super happy to add more maintainers to this project, as realistically I can't give it nearly the time and attention it deserves.

In my experience, BDFL-style open source projects (with one person taking point) often produced the best software, but also suffer from the bus factor. If nobody wants to commit to taking full ownership right now, I'm open to exploring a fully democratic model with a set of core maintainers who can make decisions on features and PRs together.

@wren, sounds like you're up for it, so I already added you :) @zorab47 and @Fandekasp, are you in too?

Let's bring this project back to life together!

[maebert]

PS. @wren, do you think we can merge https://github.com/jrnl-plus/jrnl-plus back upstream?

[wren]

@maebert Hi! Thank you for the add to the repo! I was just starting to get attached to my nascent fork, but I think working with the original project will have the biggest impact for users.

I'm happy to commit to full ownership of the project, and merge my fork back upstream. I'm very excited to work on this! The only thing I would ask is about preventing the "bus factor" in the future. I would strongly prefer to have the repo owned by an org instead of a single user (like how jrnl-plus is set up). In case anything happens to me in the future, it would make it much easier for a small set of admins to keep the project moving forward. Would you be open to transferring the repo to an org (happy to keep you as admin, if you like)?

[maebert]

@wren Done, done and done. We're now at jrnl-org/jrnl and you are an owner too.

[wren]

Awesome. Thank you, @maebert! 🎉

I've been working with @micahellison, who has been invaluable through this, so I'm going to add him as an owner, too.

[maebert]

Please do, and welcome onboard @micahellison 

On Wed, Jul 03, 2019 at 11:07 AM, Jonathan Wren < [email protected] > wrote:

Awesome. Thank you, @ maebert ( https://github.com/maebert ) ! 🎉

I've been working with @ micahellison ( https://github.com/micahellison ) ,
who has been invaluable through this, so I'm going to add him as an owner,
too.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/591?email_source=notifications&email_token=AAH7U7PJFMZVUWXL7FWYLB3P5TTG7A5CNFSM4HMHOWV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFIDBY#issuecomment-508199303
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/AAH7U7IGC6WTNZVZSHM5UYTP5TTG7ANCNFSM4HMHOWVQ
).

[wren]

@micahellison and I have a hackathon scheduled for this Saturday. We were going to focus on developing the fork more, but we can focus on getting it merged back upstream instead. With the org set up on Github, we can work through most of the issues to get this project back up and running.

There is one issue I see, though, and that's the jrnl.sh domain. It's currently down (#597), and we don't have any way to troubleshoot it. There are various options on how we can proceed (keep jrnl.sh, get new domain, use a github.io subdomain, etc), but I wanted to check in with you, @maebert, to see how you feel about it since you own the domain. How would you want to proceed in relation to the jrnl.sh domain?

[maebert]

Had an issue with nic.sh ( http://nic.sh/ ) and talked to them this morning , domain should be up again shortly! 

I'll be available on Saturday too (but primarily working on other stuff)  if you want to shoot over questions :) 

On Wed, Jul 03, 2019 at 11:28 AM, Jonathan Wren < [email protected] > wrote:

@ micahellison ( https://github.com/micahellison ) and I have a hackathon
scheduled for this Saturday. We were going to focus on developing the fork
more, but we can focus on getting it merged back upstream instead. With
the org set up on Github, we can work through most of the issues to get
this project back up and running.

There is one issue I see, though, and that's the jrnl. sh ( http://jrnl.sh/
) domain. It's currently down ( #597 (
https://github.com/jrnl-org/jrnl/issues/597 ) ), and we don't have any way
to troubleshoot it. There are various options on how we can proceed (keep jrnl.
sh ( http://jrnl.sh/ ) , get new domain, use a github. io (
http://github.io/ ) subdomain, etc), but I wanted to check in with you, @ maebert
( https://github.com/maebert ) , to see how you feel about it since you own
the domain. How would you want to proceed in relation to the jrnl. sh (
http://jrnl.sh/ ) domain?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/591?email_source=notifications&email_token=AAH7U7KEWDDADLE2H3HICVTP5TVV7A5CNFSM4HMHOWV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFJ6CI#issuecomment-508206857
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/AAH7U7LR7OKB44B22UUYTZLP5TVV7ANCNFSM4HMHOWVQ
).

[zorab47]

This is exciting news! Thanks for opening up the project. I can help with jrnl, but cannot dedicate a lot of time to it.

I'm so glad to see @wren, @micahellison, and @Fandekasp stepping up!

[MinchinWeb]

I found my way here after receiving an email that I was being "removed as a collaborator on jrnl's repositories". If this is a sign that jrnl is going to see new life, that that's wonderful!

I've been maintaining my own development branches under master-wm and 2.0-wm. The master-wm branch is/was intended as fixes needed in the 1.9.8 version, and the 2.0-wm branch is intented to be a release-worthy "version 2.0". This second branch is basically what I've been running locally and using daily for the last 2 years.

I've been busy with a number of other projects, but I've also been thinking about how to move jrnl forward. The big change in version 2.0 was/is changing the date/title header on an entry so that the timestamp format could be changed without losing your whole journal file; another big change was updating the cryptography backend. The next step (which can be done post-2.0 release), in my thought process, is to consider moving the importers, backends, and exporters to a plugin system. My stumbling block is I haven't been able to find a simple, lean system to set this up.

For backends, we currently support three: DayOne Classic (a folder of XML files), plain text, and encrypted plain text; I'm not sure we should have these as "public" plugins as much of the internal logic (and the importers and exporters) work best when we know the bounds of the pieces of information that are in play for each entry. Two possible expansion ideas are to do a nested folder structure (i.e. put today's entry at 2019\07\05.txt) or a database. Nested folders may be handy as the number of entries grow (I'm currently up to just under 1,000) and you want to archive pieces of it at a time. A database moves away from current "plain text works forever" ethos jrnl started with, but is a feature that has been requested in the past; perhaps the DayOne 2 format is an obvious choice. Or maybe we're better leaving the database option as an exporter.

Currently with backends, the DayOne option is the only one that stores metadata (beyond the timestamp and the entry title) as such; should we look into something like YAML front matter to include extra metadata in the plaintext options?

For importers, I think there is great value is allowing users to write their own (without having to directly modify jrnl), and this relieves jrnl maintainers from having to write something to support every format under the sun. The importer framework might also be useful to leverage to create templated entries, or entries generated from a series of questions posed to the user.

For exporter plugins, this is something I would use today. My current workflow is to maintain entries with jrnl, but I export them to create a website with Pelican to read and review them. To this end, I use a slightly modified version of the current markdown exporter to work with Pelican's quirks. (This is my prjct branch). I think there are many similar situations where people would like to extend their usage of jrnl, and using an exporter would provide a relatively simple way to do so.

In summary, I'm glad to see new life in jrnl, and I would be happy to help. Hopefully some of my ideas prove useful, but I'm happy to support another way to reach these goals.

[maebert]

Thanks @MinchinWeb - I just added you to the org as well!

There is one big strategic decision here. Do we want to get all of the new features (new encryption backend, importers / exporters, new configuration style, more backends, ...) in in a big new release, or focus on what's done and what works, wrap this up, call it a 2.0, and go back to incremental improvements after that?

After the 2.0 push has been dormant for 5 years I personally prefer the latter approach, as long as we focus on making upgrading really easy.

[wren]

@MinchinWeb Hi! I was cleaning up old permissions on the repo, which is why you got that notice. I'm very happy for this project to keep getting help from as many collaborators as are willing to give it, so thank you for taking the time to chime in with your guidance.

I think your ideas sound great (particularly the plugins idea), and would love to discuss implementation of new features. But first, our current focus is getting a viable 2.0 release ready, and it sounds like that's the same focus of your 2.0-wm branch. Do you think you have time to prep a PR for the fixes from your branch into the 2.0-rc1 branch? I would love to get them into a proper release where others can benefit from them. What do you think?

[wren]

@maebert Hey, we're working on stuff today and have few quick questions. What's the best way to get ahold of you? Twitter? Email?

[wren]

@maebert To clarify, there are questions with some sensitive info (like getting added on pypi), so github isn't the best place.

[wren]

Hi! So, here's where we're at:

• the jrnl-plus fork is merged back into this project
• got the CI pipeline back up and running
• went through all open PRs and closed/commented/etc on each one (they're all now labeled)
• some PRs we will wait a bit on, since we want to give contributors a chance to come back before closing useful PRs
• various administrative things

And the big news:

🎉 We have a new release candidate (v2.0-rc2)! 🎉

Once we get rc2 tested a bit more, and have a better sense of bugs, we'll be ready to call v2.0 an official release. Fixes for critical bugs will go into v2.0, and lower priority fixes will go into a later release (v2.0.1). So, to be clear: v2.0 is now locked for anything but critical bug fixes, and any new features will go into subsequent releases (once we tame the backlog). So, if you have a feature or fix you'd like to see, please get an issue filed (if there isn't already one).

In the meantime, if everyone can please install and use the v2.0-rc2 release and report your experience (either report bugs, or report that you didn't experience any bugs), it would be very, very helpful. The more people we can get to run rc2, the better off we'll be when it comes to official release time.

Thanks, everyone!

[wren]

@maebert We still need to talk about getting me and @micahellison added as maintainers on PyPI, otherwise, we'll be unable to distribute v2.0 using pip when the time comes.

[maebert]

That, and Travis. For now I put the PyPi password into environment variables in Travis ( see #162 ) so we can deploy from there, I’m looking into what the best way to set up shares pypi credentials is too.

On Sun, Jul 7 2019 at 20:51, < [email protected] > wrote:

@maebert ( https://github.com/maebert ) We still need to talk about getting
me and @micahellison ( https://github.com/micahellison ) added as
maintainers on PyPI, otherwise, we'll be unable to distribute v2.0 using pip
when the time comes.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/591?email_source=notifications&email_token=AAH7U7IRLS5OFMBC7YMMXS3P6K2TZA5CNFSM4HMHOWV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZL4P5Y#issuecomment-509069303
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/AAH7U7POBB4DSETOLH3GGF3P6K2TZANCNFSM4HMHOWVQ
).

[wren]

@maebert Travis uses the Github permissions now, so any owner of this repo can manage Travis (that's how I got the pipeline back up and running without your credentials). For PyPI, you just need to add me as a maintainer, and then either of our accounts can manage the project there without sharing any credentials.

[maebert]

@wren What's your PyPI username?

[wren]

@maebert nowandwren

[maebert]

Clever. Added.

On Mon, Jul 08, 2019 at 9:42 AM, Jonathan Wren < [email protected] > wrote:

@ maebert ( https://github.com/maebert ) nowandwren

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/591?email_source=notifications&email_token=AAH7U7N5RNYWUH4YQFGLJ4DP6NVAZA5CNFSM4HMHOWV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNVJEY#issuecomment-509301907
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/AAH7U7IRF5RMAOJHZ2UBCWDP6NVAZANCNFSM4HMHOWVQ
).

[MinchinWeb]

In reviewing some open pull requests, there is Pull Request #458 that would support the nested folder structure I'd suggested as a new backend. Another suggested backend is to add Git support.

@maebert Thanks!

Further to you comment, I think something that would be worthwhile prioritizing is making the release process as simple as possible, especially since we all seem to rather time-limited. I've created a module that I use for my own projects that provides for single command releases (bump version, run tests, build docs, package and push to PyPI, plus a few more checks). But I'm happy to support any solution to that end.

@wren Mostly I just wanted to make sure it wasn't a sketchy take-over, but I'm glad I found this issue that explained everything! Also, I fear I might have missed the revitalization if it weren't for the notification.

---

I've updated #416 that deals with some issues certain editors (particularly on Windows) were running into. It's a 2-line bugfix, and probably a good candidatefor the 2.0 release.

I see that support for DayOne-based journals was dumped when merging in the jrnl-plus fork. This is a feature I use all the time and this change keeps me from being able to test 2.0-rc2 locally; I maintain a daily work journal using a Windows clients that can read the DayOne format (Journely). A major release is probably required (or at least recommended) for such a change, but it would be nice if we had another format that supported metadata and attached pictures, and ideally a Windows client, before we drop support for DayOne Classic. On the plus side for keeping it, the format should be stable and most of the bugs long worked out. For replacements, RedNotebook was proposed as one option.

[wren]

@minchinweb Yup, I agree about prioritizing the release process. After getting v2.0 out, and the backlog tamed, I want to prioritize updating the documentation, and project tooling (dependency management, release tools, etc).

Also, about the Day One support. We didn't drop the support, just the tests. They were sorely outdated, and jrnl was never updated for the Day One 2. So, the same functionality should still be there. I agree that we need to have a larger discussion about Day One (and support for third party formats). I ultimately see this as a schema discussion, but I'm getting ahead of myself. I agree that this needs more discussion, and needs to be a major release when it does happen.

So, if you want to try v2.0-rc2, you should still have the same Day One support you've been enjoying. But make a backup, please, just in case!

[AdrienLemaire]

Hi guys. I'm in the middle of a trip, so won't be able to do much in the short term.
I was just wondering where/how did the members get added to the project, because I can only see @wren in the organization member list so far: https://github.com/orgs/jrnl-org/people.

Can we also add a Slack or equivalent to easy communication?

[wren]

@Fandekasp Displaying membership in an org is up to each member of the org. I set mine to public, which is why I'm displayed. The other members that I see listed are all set to private (which is a personal preference).

[maebert]

To ease with testing, you can install RC2 with pip install jrnl==2.0.0rc2.

This is a pre-release, so pip install jrnl will still install 1.9.8.

[MinchinWeb]

@wren Glad to hear that DayOne support is still in the tin. Sorry if it was much ado about nothing. I will get set testing rc2!

I would also list myself as a public organization member, but I think I just got added to the repo, rather than the org...? This is what it shows on my settings under:

[emceeaich]

Will there be an RC3 soon that fixes the issue with the pinned version of cryptography? I've been using pipx to install and run jrnl.

[wren]

Yup, rc3 is scheduled for this Saturday, July 20th. One of the two critical
bugs is already fixed, and I'll work on the other one that day (unless
someone else gets to it first).

>

[micahellison]

Hello there! We pushed out rc3 the other day to fix the pinned cryptography issue and some parsing issues with markdown and dates.

You can install it w/ pip like so:
pip install jrnl==2.0.0rc3.post2

[MinchinWeb]

The two issues I needed to work on should be ready to push to master:

• #350 -- a bunch of clean up when dealing with DayOne (classic) journals
• #639 -- change over the Markdown headers style

Both have been rebased to the current master, and have all tests passing.

Also, I came across autopud, which appears to be a package that will automatically push a release to PyPI when code is successfully pushed to master. I leave it hear mostly as a bookmark for post-2.0 when we revist the release procedure.

[maebert]

Automatic deploys are included in #612 - configured to publish to Pypi on tagged commits to master :) 

On Thu, Aug 01, 2019 at 9:27 PM, MinchinWeb < [email protected] > wrote:

The two issues I needed to work on should be ready to push to master:

• #350 ( https://github.com/jrnl-org/jrnl/pull/350 ) -- a bunch of clean up
  when dealing with DayOne (classic) journals
• #639 ( https://github.com/jrnl-org/jrnl/pull/639 ) -- change over the
  Markdown headers style

Both have been rebased to the current master, and have all tests passing.

Also, I came across autopud ( https://github.com/autopub/autopub ) , which
appears to be a package that will automatically push a release to PyPI
when code is successfully pushed to master. I leave it hear mostly as a
bookmark for post-2.0 when we revist the release procedure.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/jrnl-org/jrnl/issues/591?email_source=notifications&email_token=AAH7U7KNFYA6NFFUOHQBYHDQCOZTLA5CNFSM4HMHOWV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MRLQA#issuecomment-517543360
) , or mute the thread (
https://github.com/notifications/unsubscribe-auth/AAH7U7MIP5VWCIFJ6OXFWUTQCOZTLANCNFSM4HMHOWVQ
).

[defulmere]

Hey, I was just browsing my GitHub stars and happened to stumble across this conversation.

It’s a great example of open source collaboration and volunteerism, people stepping up to keep the projects they love alive.

Kudos to all of you :-)

[maebert]

☝️This shoutout made my day. I'm incredibly proud and grateful of everybody who put in their time and skill to keep this project going. This is OSS at its best.

[jordanrinke]

I just found this project recently, and was sad when it initially looked dead but this revival is fantastic to see. Super stoked and appreciate all of the work that everyone has done.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alichtman]

@wren Would it be possible to add me as a maintainer? I've now written some substantial features for the project and have been actively involved for ~7 months. It seems like jrnl could benefit from some more maintainers.

[wren]

@alichtman I want to make sure great contributors like yourself are happy and involved in jrnl as much as possible. Can you please clarify what you think additional maintainers could help jrnl with? I just want to be sure we understand what you're looking for when you ask to become a maintainer.

[alichtman]

The thing that stuck out to me the most was the lag in getting completed PRs merged in. For example, https://github.com/jrnl-org/jrnl/pull/842/commits/47e0305aa002143e7a3aadc7e1bc0ba85d966479 was committed and working in November, however, just merged in last week.

With another maintainer, I think iteration speed will increase. This leads to better features for jrnl, faster.

[wren]

@alichtman Ah, okay, I thought that might be the case. I'm sorry that your feature didn't make it into jrnl as fast as you wanted. I do want to point out that our PRs are open to the public, and anyone can review and add comments that can help speed up the review process.

For the specific commit you mention: you'll notice that your original PR was merged in a while ago (in November). I merged it to a separate branch named v2.2. This was done while scheduling features so that no one version is too big (to assure quality), while also allowing bugfixes to keep flowing.

After that, the branch piled up some merge conflicts, and other fixes took priority, so it actually ended up being v2.3 instead of v2.2. Again, I'm sorry your fix didn't make it in as quickly as you would have liked.

You might be interested to know that we've actually been changing our merge strategy to speed up the PR-to-release cycle. We've moved to just using the develop and master branches (which is why you no longer see branches like v2.2). This does mean that some PRs will still take a bit longer while we have a beta out (like we do right now), but overall it should speed things up quite a bit.

All of that being said, I do agree with you that our merge strategy hasn't as clear as it could be to contributors. We'll be updating docs soon to clarify what everyone should expect, so please look forward to that. In the meantime, the best thing anyone can do to help PRs along is add comments to, and manually test, the PRs that come our way.

[dbxnr]

I kind of agree with @alichtman here, the cadence of development is quite slow and there doesn't seem to be any overarching plan to adhere to. It's a great project and very useful piece of software, however development is effectively dead...

[wren]

Hi, @dbxnr! If you're wondering what we're up to, please check out the changelog. We also do some more behind the scenes, but that can give you a general idea of what's going on.

As I briefly mentioned above, our biggest bottleneck right now is testers. We purposefully don't merge in too many big changes into each version so that we can properly test each version before releasing it. It does slow us down a bit, but ensures we maintain a high level of quality in jrnl.

If you'd like to see the development cycle go faster, I would encourage you to load up some of our beta builds and/or PRs, run them locally, and report back if you find any issues or if the build looks good. This go a very long way to increase the amount of code we can put out into the world.

[wren]

It sounds like we need to clarify expectations around merge strategy,
and roadmap. We've heard you and are working it. If you have specific
suggestions on how best you'd like this communicated, we'd love to hear them.
We're also working on a plan for this ourselves, and will lay something out
soon.

I also want to take this moment to say that we rely on the community to help
jrnl for many things. Both of you--and the entire community--already have what
you need to help jrnl move faster. Please see below for a list of things that
currently take up a lot of our time. Help with these things would speed up the
development cycle (and I would personally appreciate anyone who's willing to
jump in to help with any of these things).

• Test prereleases - When we have a new version ready, we put out a
  prerelease. These prereleases need to be manually tested. The nature of jrnl
  means we deal with extremely sensitive data, and cannot risk data-loss. The
  more people test these and report back, the shorter the time a prerelease
  needs to exist before becoming a full release.
• Test PRs - These also need to be manually tested. Our testing suite isn't
  fully there to cover everything we'd like, so manual testing fills the gap
  for now. Bonus points if you can add a screencast of how the new feature
  works.
• Speaking of tests, add tests. Just add tons of tests. Test anything and
  everything. The more reliable our testing framework becomes, the less
  necessary manual testing becomes.
• Confirm bug reports. By their very nature, bug reports are issues that got
  through our testing. A common reason for this is either hardware or software
  that we don't have. We test most things on 4 different computers (plus the CI
  stack), but that doesn't come close to covering all the possible use cases. A
  simple comment like "Confirmed on jrnl v2.2, MacOS 10.15, Python 3.8.1" would
  be extremely helpful in tracking down bugs.

[dbxnr]

We purposefully don't merge in too many big changes into each version so that we can properly test each version before releasing it. It does slow us down a bit, but ensures we maintain a high level of quality in jrnl.

This it literally the role of CI.. Based on my experience with the codebase, it would make a lot of sense to write more thorough tests in a framework that doesn't employ a DSL. If testing is a bottleneck and you can't rely on your own implementation, it sounds like you need to change approaches.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. You can link back here from your new issue to continue the conversation.