minchin.jrnl v7 “Phoenix” released

Today, I do something that I should have done 5 years ago, and something that I’ve been putting off for the last 2 years¹: I’m releasing a personal fork of jrnl²! I’ve given this release the codename Phoenix, after the mythical bird of rebirth, that springs forth renewed from the ashes of the past.

You can install it today:

pip install minchin.jrnl

And then to run it from the command line:

minchin.jrnl

(or

jrnl

)

Features Today

Today, the codebase is that of jrnl v2.6³ in a new namespace. In particular, that gives us a working yaml exporter; now you can build your Pelican sites again (or maybe only I was doing that…).

The version number (7) was picked to be larger than the current jrnl-org/jrnl release (currently at 4.0.1). (Plus I thought it would look cool!)

I’ve moved the configuration to a new location on disk⁴, as to not stomp on your existing jrnl (i.e. jrnl-org/jrnl or “legacy”) installation.

Limited documentation, to match the current codebase, has been uploaded to my personal site at minchin.ca/minchin.jrnl. (Although it remains incomplete and very much a work in progress.)

And finally, I extend an invitation to all those current or former users of jrnl to move here. I welcome your contributions and support. If there are features missing, please feel free to let me know.

Short Term Update Plans

I’ve pushed this release out with very few changes from the base codebase in a effort to get it live. But I have some updates that I’m planning to do very shortly. There updates will maintain the Phoenix codename, even if the major version number increments.

The biggest of these is to launch my much anticipated plugin system. The code has been already written (for several years now⁵, actually), can it just needs to be double checked that it still works as expected.

The other immediate update is to make sure the code works with Python 3.11 (the current version of Python), which seems to already be the case.

Medium to Long Term Project Goals, or My Vision

These are features I’d like to add, although I realize this will take more than tonight. Also this section lays out my visions for the project and some anti-features I want to avoid.

The Plugin System

The plugin system I think will be huge movement forward to make minchin.jrnl more useful. In particular, it allows minchin.jrnl to import and export to and from new formats, including allowing you to write one-off export formats (which I intend to use personally right away!). Displying the journal entries on the commandline is also handled by exporters, so you’d be able to tweak that output as well. I also intend to extend the plugin system to the storage backends.

My hope is that this will futureproof minchin.jrnl, allowing new formats to quickly and easily be added, retiring deprecated formats to external plugins, and being able to quickly test and integrate new formats by seemlessly bring external plugins “in-house”.

In particular, I’m planning to have separate plugins for “true” yaml + markdown exports and Pelican-formated markdown, to add an export format for Obsidian⁶ formatted markdown, and add backend format plugins to support jrnl v1 and whatever format they’re dreaming up for jrnl v4⁷.

In short, I hope plugins will allow you to make minchin.jrnl more useful, without me being the bottleneck.

Plain Text Forever

One of the things that drew to the original jrnl implementation was that is was based on plain text, and using plain text to store journal entries. Plain text has a number of advantages⁸, but the near universal backwards and forewards compatibility in high on that list. Yes, plain text has it’s limitations⁹, but I think the advantages far outweight the disadvantages, particularly when it comes to a journal that you might hope will be readable years or generations from now. Also, plain text just makes it so much easier to develop minchin.jrnl.

The included, default option for minchin.jrnl will always be plain text.

If you’re looking to upgrade your plain text, you might consider Markdown¹⁰ or ReStructured Text (ReST)¹¹.

If you’re looking for either a database backend or more multimedia functionality (or both), you’re welcome to write something as a backend plugin for minchin.jrnl; that ability is a featured reason for providing the (to be added shortly!) plugin system in the first place!

MIT Licensed

The original jrnl was initially released under the MIT license, and that only changed with the v2.4 release to GPLv3¹². My hope and goal is to remove all GPL-licensed code and release future versions of minchin.jrnl under the MIT license²³.

My opposition to the change¹³ was because I’ve come to feel that Open Source work is both given and received as a gift, and I feel the GPL license moves away from that ethos.

I suspect the least fun part of this partial re-write will be getting the testing system up and running again, as the original library jrnl v1 had been using has gone many years without updates.

To this end, I’m requesting that any code contributions to the project be dual-licensed under both MIT and GPLv3.

Documentation in Sphinx

Documentation will eventually be moved over to Sphinx (from the current MkDocs), a process I’ve began but not finished. Driving this is the expectation that I’ll have more technical documentation (than is included currently) as I layout how to work with the plugin API, and Sphinx makes it easy to keep code and documentation side by side in the same (code) file.

Furthermore, I want to document how to use minchin.jrnl as a Python API generally; this would allow you to interact with your journal from other Python programs.

Easy to Push Releases

Knowing my own schedule, I want to be able to sit down for an evening, make (several) small improvements, and then push out a new release before I go to bed. To that end, I want to make the streamlined to push out new releases. Expect lots of small releases. :)

Drop poetry

poetry is a powerful Python project management tool, but is one I’ve never really liked¹⁴. Particular issues include a lack of first class Windows support¹⁵ and very conservative upper bounds for dependencies and supported Python versions. Plus I have refind a system elsewhere using pip-tools¹⁶ and setup.py to manage these same issues that I find works very well for me.

This has been accomplished with the current release!

Windows Support

jnrl, to date, has always had decent Windows support. As I personally work on Windows, Windows will continue to have first class support.

Where this may show is tools beyond Python will need to be readily available on Windows before they’re used³³, and the Windows Terminal is fairly limited in what in can do, at least compared with some Linux terminals.

Replace the Current Code of Conduct

I don’t much care for the current Code of Conduct¹⁷: it seems to be overly focused on the horrible things people might do to each other, and I’m not sure I want that to be the introduction people get to the project. I hope to find a Code of Conduct that focuses more on the positive things I hope people will do as they interact with each other and the project.

My replaced/current Code of Conduct is here (although this may be updated again in the future).

Open Stewardship Discussion

If the time comes when someone else is assuming stewardship for the project, I intend for those discussions to be help publicly¹⁸.

My History with the Project, and Why the Fork

This section is different: it is much less about the codebase and more focused on myself and my relationship to it. I warn you it is likely to be somewhat long and winding.

My Introduction to jrnl

Many years ago now, I was new in Python. At that time³⁴ when I came across a problem that I thought programming might solve, I first went looking from a Python program that might solve it.

In looking for a way to manage the regular text notes I was taking at work, I found jrnl, which I eventually augmented with DayOne (Classic) for in-field note entry (on a work-supplied iPad) and Pelican for display.

Jrnl was more though: it was the object of my first Pull Request³⁵, my first contribution to Open Source. My meagre help was appreciated and welcomed warmly, and so I returned often. I found jrnl to be incredibly useful in learning about Python best practices and code organization; here was a program that was more than a toy but simple enough (or logically separated) that I could attempt to understand it, to gork it, as a whole. I contributed in many places, but particularly around Windows support, DayOne backend support, and exports to Markdown (to be fed to Pelican).

In short jrnl became part of the code I used everyday.

jrnl Goes Into Hibernation

I have heard it said that software rewrites are a great way to kill a project. The reasons for this are multiple, but in short it (typically) saps the energy to update the legacy version even as bugs pile up, but the new thing can’t go into everyday use until it is feature-compatible with the legacy version, and the re-write always takes way more effort than initial estimates.

For reasons now forgotten³⁶, a “v2” semi-rewrite was undertaken. And then it stalled. And then the project maintainer got a life¹⁹ and the re-write stalled even moreso.

The (Near) Endless Beta of v2, or the First Time I Should Have Forked

For me, initially, this wasn’t a big deal: I was often running a development build locally as I tinkered away with jrnl, so I just kept doing that. Also, I had just started working on my plugin system (for exporters first, but expecting it could easily be extended to importers and backends).

As the months of inactivity on the part of the maintainer stretched on, and pull requests grew staler, at some point I should have forked the project and “officially” released my development version. But I never did, because it seemed like a scary new thing to do²⁰.

Invitation to Become a Maintainer

And then²¹ one day out of the blue, I got an email from the maintainer asking if I wanted to be a co-maintainer for jrnl! I was delighted, and promptly said yes. I was given commit access to the repo on GitHub (but, as far as I knew, no ability to push releases to PyPI), and then…not much happened. I reached out to the maintainer to suggest some plans, as it still felt like “his” project, but I never heard much back. And I was too timid to move forward without at least something from him. And I was busy with the rest of life too. After a few months, I realized my first plan wasn’t working and started thinking about how to try again to move the project forward, more on my own. In front of me was the push to v2.0, and a question of how to integrate my in-development plug-in system.

The Coup

And then on a one another day, again out of the blue, I got an unexpected email that I no longer had commit access to the jrnl repo. I searched the project for updates, including the issue tracker and came up with #591 where a transition to new maintainers was brought up; I don’t know why I wasn’t pinged on it. At the time²², I said I was happy to see new life in the project and to see it move forward. But it was unsettling that I’d missed the early discussions.

It also seemed odd to me that the two maintainer that stepped forward hadn’t seemed to be involved with the project at all before that point.

For a while, things were ok: a “version 2” was released that was very close to the v2 branch I was using at home, bugs started getting fixed regularly, and new releases continued to come out. But my plugins never made it into a release.

Things Fall Apart (aka I Get Frustrated)

But things under new management didn’t stay rosy.

One of the things they did was completely rewrite the Git history, and thus change all the commit hashes. This was a small but continueing annoyance (until I got a new computer), because everytime I would go to push changes to GitHub, my git client would complain about the “new” (old) tags it was trying to push, because it couldn’t find a commit with the matching hash.

But my two big annoyances were a re-write of the YAML exporter and the continual roadblocks to getting my plugin system merged in.

My plugin system has the longest history, having been started before the change in stewardship. Many times (after the change in stewardship), I created a pull request²⁴ and the response would be to make various changes or to split it into smaller pieces; I would make the changes, and the cycle would continue. But there was never a plan presented that I felt I could successful complete, nor was I ever told the plugin system was unaligned with the vision they had for the project. I lost considerable enthusiasm for trying to get the plugins merged after rewriting the tests for the third time (as the underlying testing framework was changed).

The YAML exporter changes are what ultimately left me feeling frozen out of the project. Without much fanfare, the YAML exporter was changed, because someone²⁵ felt that the resulting output wasn’t “true” or “pure” YAML. This is true, in a sense, because when I had originally written the exporter, it was designed to output files for Pelican with an assumed Markdown body and YAML front matter for metadata. At the request of the (then) maintainer, I called it the “YAML exporter”, partly because there was already a “Markdown exporter”. I didn’t realize it had been broken until I went to upgrade jrnl and render my Pelican site (which I use to search and read my notes) and it had just stopped working²⁶. The change wasn’t noted (in a meaningful way) in the release notes, and the version numbers didn’t give an indication of when this change had happened³⁰. I eventually figured out where the change had happened, explained the history of the exporter (that again, I had written years earlier) and proposed three solutions, each with a corresponding Pull Request: 1) return the YAML exporter to it’s previous output²⁷, 2) duplicate the old exporter under a new name²⁸, or 3) merge my plugin system, which would allow me to write my own plugin, and solve the problem myself. I was denied on all three, and told that I ‘didn’t understand the purpose or function of the YAML exporter’³¹ (yes, of the plugin I’d written³⁷). The best I got was that they would reconsider what rose to the level of a “breaking change” when dealing with versioning³².

I Walk Away

The combined experience left me feeling very frustrated: jrnl was broken (to me) and all my efforts to fix it were forably rebuffed.

When I tried to express my frustrations at my inability to get a “working” version of jrnl, I was encouraged to take a mantal health break. While I appreciate the awareness of mental health, stepping away wouldn’t be helpful in this particlar case because the nothing would happen to fix my broken tooling (the cause of my frustrations). It seemed like the “right words”(tm) someone had picked up at a workshop, but the same workshop figured that the “right words”(tm) would solve everything without requiring a deeper look or deeper changes.

So I took my leave. I’ve been running an outdated version (v2.6) ever since, and because of the strictness of the Poetry metadata²⁹, I can’t run it on anything newer than Python 3.9 (even as I’ve upgraded my “daily” Python to 3.11).

I Return (Sort Of); The Future and My Fears

So this marks my return. My “mental health break” is over. As I realize I can only change myself (and not others), I will do the work to fix the deeper issues (e.g. broken Pelican exports, lack of “modern” Python support) by managing my own fork. And so that is the work I’ll do.

Looking forward, if I’m the only one that uses my fork, that would be somewhat disappointing, but also fine. After all, I write software, first and foremost, for my own usecase and offer it to others as a gift. On the other hand, if a large part of the community moves here, I worry about being able to shepherd that community any better than the one I am leaving.

I worry too that either due to there being conflict at all, or that all of my writings are publically displayed, others will think less of my work or myself because of the failings they see there. It is indeed very hard to get through a disagreement like this without failing in some degree.

But it seems better to act, than to suffer in silence.

A Thank You

Thank you to all those who have worked to make jrnl as successful as it has been to date.

If you’ve gotten this far, thank you for reading this all. I hope you will join me, and I hope your experiences with minchin.jrnl are positive!

The header image was generated locally by Standard Diffusion XL.