Sunday, February 8

systemd & fsck

I just hit my first real frustration with systemd, which is running on the Novena. The default storage here is a microSD card, and I’ve had to force-reboot this thing enough times that I’d like to run fsck on the root filesystem.

It used to be that you could call shutdown -F to force an fsck on boot. The old aliases still exist, but I think the thing I’m supposed to do here is systemctl reboot, and there doesn’t seem to be an analogous pattern any more.

On the other hand, some of the choices immediately evident in the design of systemctl and journalctl seem interesting and not without merit.

Friday, February 6

A little visual poetry, courtesy of the clipboard history for the text editor on my artisanal computer:


Monday, February 2

kernel-o-matic & pi finder

Published Friday:


Published a week or so before:


These have been a lot of my working time these last couple weeks, an overlapping set of projects aimed at making the Pi (and eventually other single-board computers) more usable and Adafruit’s hardware and tutorials both more accessible. This has been frustrating and rewarding by turns. I’m trying to reduce the complexity of a domain I just barely understand, in a lot of ways, which may be a good summary of software development in general.

Vagrant is something I should have paid attention to sooner. The interfaces to virtualization are finally starting to overcome my innate laziness on the whole question.

I just booted a Windows XP box for some reason. It made that noise. You know the one.

raspberry pi 2

Announced today:


Expect this to prove interesting. I’ve been having a lot of conversations about the relative merits of small computing systems, and while this can hardly be said to address all the complaints you might have about the Pi, boosting processor and RAM will do a lot for practical usability.

telling composer to ignore php version requirements

Using Composer to set up a little project, I run into the problem that the locally-installed PHP is a bit behind the times.

brennen@exuberance 0:09:32 /home/brennen/code/project $  ./composer.phar install
Loading composer repositories with package information
Installing dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - sparkfun/sparklib 1.1.9 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.8 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.7 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.6 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.5 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.4 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.3 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.2 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.11 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.10 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.1 requires php >=5.5.17 -> no matching package found.
    - sparkfun/sparklib 1.1.0 requires php >=5.5.17 -> no matching package found.
    - Installation request for sparkfun/sparklib ~1.1 -> satisfiable by sparkfun/sparklib[1.1.0, 1.1.1, 1.1.10, 1.1.11, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9].

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <> for more details.

Read <> for further common problems.

Well, ok. I wrote a lot of that code, and I’m pretty sure nothing I want out of it will break under a slightly stale PHP. I check ./composer.phar help install, and sure enough, there’s an option to ignore this requirement:

brennen@exuberance 0:13:21 /home/brennen/code/project $  ./composer.phar install --ignore-platform-reqs  
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing sparkfun/sparklib (1.1.11)
    Downloading: 100%         

Writing lock file
Generating autoload files

I never used to quite get the “install an executable utility script in the root directory of your project” thing, but the whole paradigm is growing on me a little as my projects accumulate little Makefiles and shell scripts to render HTML, publish revisions, or deploy packages.

Sunday, February 1

why aren't relational databases everywhere, like filesystems?

Background reading for this post:

A couple of months ago, I went from working on one big software project to working on a series of smaller, limited-scope projects. Of course a lot changes when you do that. One of the things I’ve really been noticing is that I’m not using relational databases for much right now, and probably haven’t written any SQL since November or thereabouts.

I should note here that I think filesystems are a really powerful abstraction, frequently underrated by people who ought to know better, and I usually argue for simple tools. I am often that guy who wonders why you didn’t just use grep, and even when I have a robust database at my disposal, I often explore data by hacking together silly ad hoc pipelines in the shell.

With that out of the way, I just spent years treating first MySQL/MariaDB and then PostgreSQL as default storage for most problems, and I’ve arrived at a healthy respect for the tools, particularly PostgreSQL. It is nice to accumulate collections of structured data. It’s nice to have defined schema with a rich set of types, and to expect type constraints to be enforced and the semantics of conversion between types made explicit. It’s really nice to have a regular language for querying these collections.

Thinking about all of that, I thought “I should really just stand up psql on my primary machine and populate it with interesting things”. SQL is, to be sure, imperfect, but as a nerd working in the guts of a money-making enterprise, it quickly becomes one of your basic tools for turning a store of facts into answers to the questions that you didn’t know you were going to have to answer. It’s one of the technologies that help render empirical questions answerable, or at least help define the boundaries of which ones are answerable.

And thinking about that, I started to wonder why I hadn’t set up a personal database years ago — why stores of relational data weren’t as much a part of my personal kit as the hierarchical filesystems I’ve been accumulating for 20+ years. And it seems to me like there’s something important in the answers to that question.

In no particular order, some thoughts:

  • No operating system in wide use offers a relational DB to the user out of the box. There’s a concerted effort underway to obscure them, but filesystems have been user-level interfaces for most of the history of computing and the idea of a “file” is pretty well embedded in the general awareness by now.

  • Moving files between machines and environments is (relatively) easy: The protocols, hardware, and services are widely available. Moving relational data is almost universally unpleasant, despite piggybacking on the file stuff. It’s unforgiving in interface, slow, error prone, and implementation specific.

  • There are plenty of tolerable GUI file managers, and desktop systems offer them out of the box. The same can’t really be said for databases. GUI clients exist (even some fairly good ones), but they’re mostly erratic, require installation, demand configuration, and/or cost money.

  • text : vim / emacs :: sql : ? — there is not, so far as I am aware, any good answer to this question. The statement generalizes quite a bit, and yet.

  • While most programming languages offer library support for the widely-used relational DBs, making them accessible for programmers who care to invest a little time, there’s no comparable level of access at the level of the shell, the windowing system, the web browser, etc.

  • People accomplish a lot with applications like Access, but while this speaks to the utility of databases, forms, and scripting, it is not anything we should feel especially good about.

  • A relational data store requires configuration to install, and configuration (or at least another set of credentials) to access from a client.

  • Like programming languages and text editors, a full-featured database is generally designed as a robust environment for tool use, offering lots of abstractions and utility functions, and quite extensible within its own space. What it’s not is much of a first-class citizen in a user-level tool environment. Databases are for people who build infrastructure, but they aren’t part of the infrastructure from the perspective of an end-user in the same way that things like filesystems, HTTP clients, and simple utility software are. Databases are earth-moving equipment, not highways and on-ramps.

  • Writing queries is fairly cheap, once you learn a little conceptual machinery, but defining or altering schema is usually slow and painful. (I’m pretty sure that the friction here generates a lot of the incentive to prefer NoSQL tools, and avoid rigorous schema definitions generally, which leads to a lot of probably-unnecessary pain.)

  • It should be trivial to pipe things in and out of queries/resultsets, but I don’t think it really is.

Of course, there’s a whole history here, and though I don’t know it well I do have a rough sense of the conditions and accidents that brought us to the status quo. It’d be a mistake to believe that the dominance of hierarchical filesystems as we know them was inevitable, but filesystems as a base layer appear to satisfy a lot of evolutionary constraints.

Which is to say that I’m not really interested in arguments that the filesystem is a dead-end or whatever. On a long enough timescale, I suppose it might be, but life is short and I have problems to solve. I’m just interested in whether you can make something approximately as good as all the good parts of PostgreSQL legible and accessible in the day-to-day nerd environment, transparent to the network and to the broader ecosystem of tools. What would it actually take?