Monday, January 4, 2021

keeping a log: 9 months / ~1k entries in

Previously: org mode, vimwiki, timeslice.

Mechanisms inspired directly by: A demo & talk from Lars Wirzenius on his ikiwiki-based external brain and journal; fediverse discussion of the Org mode agenda; and possibly too much reading about the Zettelkasten.

Back in March, in the throes of a bunch of rabbitholing about note-taking, I roughed out a system for keeping short, granular log entries in my VimWiki. I agonized for quite a while about how to do this before deciding to start with the stupidest thing that could possibly work.

The short version is that I have a hotkey to create datestamped files in a log/ directory, like these:

./vimwiki/log/2021-01-04-2033-33.wiki
./vimwiki/log/2021-01-04-1719-51.wiki
./vimwiki/log/2021-01-04-1516-18.wiki
./vimwiki/log/2021-01-04-0914-03.wiki
./vimwiki/log/2021-01-04-0142-59.wiki

A new entry opens with a template like the following:

%date 2021-01-04 21:46:40.056011313-07:00
%title

I then give the entry a human-readable title, links to relevant topics, and as much text description as seems useful. A typical entry looks something like:

%date 2020-12-11 16:49:51.356943342-07:00
%title Configuring digiKam again

[[/configuration]] [[/photos]] [[/digikam]]

Digging around in the guts of an old `digikam4.db`.  Changed the album root to
point to the new path in `~/workspace/photos`.

Then, when I’m viewing a topic page like digikam or photos, I can press another hotkey to pull up a window with any linked log entries. When I’m viewing the diary page for a given day, a bit of shell boilerplate shows me all the log entries for that date.

I’ve elaborated on this all a bit since March, but the underpinnings are still just a few hundred lines of hacky scripting and Vim configuration. Before I put any work into cleaning it up, I thought I’d try to outline some stuff I’ve learned.

I’ll use the time-honored form of “answers to questions no one has actually asked me”:

Why a log? Because in taking notes, I’m worried about two dimensions: Subject matter and time. A single flat wiki namespace can be workable for navigating the who/what/where, but it’s lousy for navigating the when.

I’ve also spent a lot of my life keeping logbooks, looking at logfiles on computers, writing a journal, and publishing a datestamped blog. At Wikimedia, I’ve been particularly impressed by how useful the server admin logs are, and I pretty much live and die by command-line history and bookmarks. It’s a notion with an overwhelming amount of precedent in my life.

What distinguishes a log entry from any other wiki page? Its placement in the log/ namespace and a handful of formatting conventions.

Was this actually a good way to approach the problem? Yeah, I think so, with caveats.

Is the implementation sound? Not by miles, but it holds up better than I expected. Eventually the flat directory structure will get cumbersome in the shell, and grepping through files like I’m doing some places might get less practical.

How are the ergonomics? Not that bad, but there should be as few keystrokes as possible involved in writing a new entry, and this doesn’t quite cut it.

What’s a good fit for this kind of log entry? Finding a new piece of software, writing a letter, taking notes on a meeting, setting up or decommissioning a piece of gear, finishing a book, garden/yard work, house and vehicle maintenance, phone calls, general life events, sysadmin work, etc.

What’s not? The single thing I’ve done the most of that probably makes the least sense in this format is logging individual expenses and financial transactions. This has been useful enough to convince me that tracking what I’m doing with money is a good idea, but clunky enough that I’ve learned stuff like “paid the mortgage” and “bought groceries” should be structured, query-able data. The most that I have to bash out with a keyboard in that context should be an annotation on a specific record or group of records. That’s not to say I’m thrilled at the prospect of keeping a rigorous double-entry ledger that balances out for every transaction in my life, but I can see the appeal in a way I couldn’t really before.

This generalizes I guess: A lot of the history I care about lives in structured, formal-ish systems like version control, banking, various databases — and other parts of it should. Like sometimes I log specific weather events, but usually when I want to know about weather in the past, what I’d really like is a way to quickly aggregate a bunch of data points.

That points at two categories of “log entry”: The loosely-typed human-readable kind that make sense as wiki pages, and the granular, highly-structured and repetitive kind that make more sense in something like a database table. Then there’s a third that doesn’t quite fit in either box. Sometimes I paste a lengthy shell transcript into a log entry, for example, and while that’s more or less fine, it points at a gap in the tools I use. It would be way nicer just to push a button when I’m doing something in the terminal that it’s important to remember exactly, and then it can record until I tell it to stop and let me add some tags and a summary to the session.

So what next? Well, I’ve arrived at something I’m going to keep using. I’d miss it if I quit, and it’s easy to accumulate a useful record this way. I might clean up the mess a bit and package its components as a VimWiki addon. After that, I’m going to spackle more stupidest-things-that-could-possibly-work on top to augment it, and think about more ways to surface and integrate other parts of the meta-log that are scattered all over the systems I use.

p1k3 / 2021 / 1 / 4
tags: topics/data, topics/logging, topics/notes, topics/technical, topics/vimwiki