Saturday, October 24

userlands, 1 of ?

I want to start sketching out this idea I keep having before I lose it again to distraction.

I wrote userland while experiencing, I now think, the end stages of a particular relationship to an existing toolset and intellectual apparatus. Namely, that of the Unix ecosystem, broadly considered.

That is to say: I had reached a certain level of competence, if not exactly mastery, in the most generally useful command-line environment. I had passed through the amateur enthusiast phase into the working professional1 phase, while retaining a sense that the system we had was grounded in sound principles, to the extent that in its broad outlines it was historically inevitable. Ragged around the edges, sure, and complicated by all sorts of counterproductive bullshit, but still under it all a sort of workingperson’s rugged theoretical coherence.

Last fall, Paul Ford wrote this, in some thoughts on (emphasis mine):

Looking at the cabal it turns out the nerds best known to me are realllly a lot of white dudes. And looking at the signup sheet, this community def. is going to be okay on white dudes. We’re nailing it, white-dudewise. But Unix is for everyone. It’s like the Sistine Chapel or Angkor Wat. It’s bigger than the culture that made it. It’s for our servers and our phones and it’s for all the genders and all the races and most of all Unix is for the children.

That last part really resonated with me, like a lot of ideas did. I figured ok, we have this thing (these many interlocking things) that are a deep part of our cultural heritage. If they were accessible to more people, people would have greater recourse in coping with the evils of computing and the power imbalances of the modern network. The tech is here, I might have said, it is just not very equitably distributed.2

I retain a lot of that impulse, but implicit in it was a judgment of the Unix things as also being more or less the right ground for the efforts of the nerd class. Or at least one such ground: A probability cloud of useful abstractions and scaffolding within which sensible people built tooling, and in ignorance of which much absurd wheel-reinventing was constantly enacted.

I have for a long time now been a conservative, of sorts, about actually existing software and innovation in user-facing systems. Harshly critical of the last decade’s inventions and revolutions in UX and system architecture.

Two aphorisms, as they sound in my head, without looking them up:

“Those who do not understand Unix are condemned to reinvent it, poorly.”

“Unix is the worst OS except for all of the others.”

I switched jobs this year. Where I used to focus on a handful of big, application-layer software projects and a pile of business logic, instead I did customer-facing stuff for Adafruit: I wrote little introduction-to-Linux tutorials and documented simple hardware projects and helped support the Raspberry Pi. I read too much of debian-devel. I wrote a lot of shell scripts. I fought with version control. I did everything wrong.

I also listened to my friends talk about intrinsically hard problems of their work: Deploying and configuring and coordinating big systems. Making little systems behave consistently for many users. Asking questions about system state and getting consistent answers.

Here is a very partial list of things I think I know right now:

— The Bourne Shell and its descendants are genuinely terrible programming languages.

— Systems tasks are of sufficient complexity to benefit from a first-class high level language where fundamental abstractions at the level of “string”, “variable”, “loop”, “conditional”, etc. are not debilitating to use.

— Regexps are a total fucking horrorshow. Because of the Turing Tarpit, or something very like it. Because semantic density only contributes to legibility up to a certain point. Because you can always hurt yourself real bad with a chainsaw, even if you know what you are doing.

— The packaging of software for Debian is an unremitting nightmare. Packaging software is just an unremitting nightmare. Also configuration management.

— Configuration management and deployment tools (Puppet / Chef / Ansible / Salt / your bespoke artisanal Python script) have in recent years adopted the language of declarative state and idempotence. This is, in practice, somewhere between a polite fiction and a dirty lie.

— Determinism in configuration of an entire system is a noble goal, it’s just totally fucking impossible for mere mortals to achieve in the current scheme of things.

— Most humans aren’t smart enough to write safe or secure programs in C. Most useful systems-level programs and services are written in C.

— One way to think of a filesystem is as a powerful baseline abstraction for most things in a computer system. Another is as a massive collection of inconsistently represented and needlessly hierarchical state.

— Statefulness is the Fruit of the Tree of Knowledge of Good and Evil.

Torturing the implementor on behalf of the user may be a grim necessity. Then again, like most forms of torture, it may harm its alleged beneficiaries along with its victims.

— Arbitrarily structured text files as baselines for configuration and source code have nearly intractable advantages. You only need to deal with a system where Every Last Thing is in XML, think hard on the Windows Registry, or maintain somebody’s Microsoft Access business logic to understand this much. On the other hand, it renders every system configuration task a nightmare of ad hoc text manipulation.

— Ad hoc text manipulation is powerful. Ad hoc text manipulation is hell.

— The officially-sanctioned desktop environment situation on nearly every Unix derivative is deeply unpleasant, and has been in a state of decreasing quality since the first decade of this century.3

— The terminal as a primary working environment is a jungle of sharp edges, and has become ever more difficult to treat as a refuge in a world where a) people want to communicate in pictures, and b) device manufacturers are doing everything in their power to kill the physical keyboard no matter what you have to say about it.

I’ll stop.

I guess my point is this: Unix sucks. I say this with love, but still, sucks. And not just “the Unix we actually got”, but also the theoretical, idealized, ur-Unix. Its basic ideas, for all of the brilliance and pragmatic down-in-the-dirt utility threaded through them, are not really sufficient to building humane systems.

More broadly, operating systems as they are currently built are bad environments for normal human beings, and nightmare environments for the specialists who work on them directly.

I’m inclined to think this matters, because I’m still a strong partisan of the view that regular users should have ownership and control over their machines. That a large share of computational utility can (and should) be kept local and distributed, if we are to stand any chance of avoiding the worst outcomes of the architecture of centralized control that the internet has made so practical, so necessary-seeming to all the people with the power to make tech decisions.

I also have a selfish desire, as a specialist, for a world in which the working conditions of programmers and sysadmins are less full of broken crap. Because for all that technical gigs are some of the better work you can get these days, as far as money and respect and not breaking your back go, they are also kind of shit in ways that both grind people down and filter out to the rest of society through countless little channels.

I’m eliding so many things here. The history of this kind of criticism. The other could-have-been worlds of OS structure. Ted Nelson and all that. So on and so forth. Maybe I’ll get to those things. I’m not exactly sure.

To be less negative, though, it’d be interesting to spend time thinking about not just what it is that’s wrong with the current dispensation, but what it is that could be fixed and outright replaced in the light of how our existing systems fail—but also how they succeed, or nearly do. Because it seems like right now we’re in the middle of replacing everything (necessary) with stuff that is built in resolute ignorance of a huge tradition of qualified success and tragicomic failure (doomed).

1 Quiet, you.

2 Uh huh.

3 Yes, I also and especially feel this way about Apple.

p1k3 / 2015 / 10 / 24
tags: topics/cli, topics/debian, topics/linux, topics/technical, topics/userland, topics/warelogging

thursday, october 22

filing cabinets

people ask me sometimes about becoming a programmer
it's a fair enough question
i'm at least nominally a professional in the field
and it can be good work if you can get it

i think my answers have varied in quality,
but it seems like almost no one ever finds them very satisfying

by definition
being in the condition of wanting to be a programmer
in line with many ambitions that merely require the investment
of many thousands of hours
is being unable to discern
bad answers from good — not knowing how to know

next time maybe i will take a different tack and ask
well, are you interested in being a career bureaucrat?

p1k3 / 2015 / 10 / 22
tags: topics/poem

Tuesday, October 20

Reading: Ancillary Justice and Ancillary Sword, by Ann Leckie. Almost done with the second, and one more book to go in the series, I think.

I’d been half-aware of these for a while, but I didn’t really think much about reading them until I hit this comment over in a MetaFilter thread about some ridiculous assholes:

So what are some good, recent , literary works of Conservative SF?

If by “Conservative SF” one means “military SF with space battles and laser rifles, where the good guys are an expansionist, conquering force, where the protagonist is a strong-jawed and clear-headed hero who outfights and outwits those who would confound the plot, and where tactics, strategy, and technology are both visionary and critical story elements,” then may I recommend the Hugo Award-winning Ancillary Justice?

…which somehow kind of struck a nerve, because I’m not sure I did want to read about the good guys as expansionist, conquering force. But then again I have read a decent amount of MilSF at one time or another, and I am still a sucker for space battles, and anyway I was curious what it was about Leckie’s stuff that would so upset the GamerGate-slash-John-Birch-Society-slash-Christofascist wing of marginally-published SF.

With respect to KathrynT (and thanks for nudging me toward the read), I don’t think that comment’s a very accurate description of the work. So far, it feels like a complicated meditation on the mechanics of empire, oppression, political power, surveillance, and distributed identity. It’s ambivalent about empire, about the situation of its protagonist—a piece of literal machinery in a ruthless, exploitative colonialist project—but it often feels like an essay on empire’s dysfunctions and cruel hypocrisy.

Then there’s the gender thing. Which I think works, mostly, and is interesting, and reads for the most part far more as an element of the worldbuilding than it does as polemic. Which, oddly, despite the deeply sexist and gender-essentialist past of so much of the genre, is what I remember about all the SF that used to shake my younger brain’s half-formed notion of what gender was, exactly.

p1k3 / 2015 / 10 / 20
tags: topics/ann-leckie, topics/metafilter, topics/read, topics/reading, topics/sfnal

Sunday, October 18

New York always feels something like a foreign country. Not like one so foreign that its modes and routines are completely illegible, but one far enough from usual life that I’m often unsure what the rhythm is, the protocol, the whatness-of-what. I speak the language (one of them, part of it) but I don’t exactly speak the place.

Is it as far from Colorado, on some hypothetical graph of civilization, as Colorado is from London, Budapest, Tirana, Christchurch, Prague? Maybe not. There’s no mistaking the American superstructure of both places for anything else. Still, it’s a trip.

p1k3 / 2015 / 10 / 18
tags: topics/colorado, topics/new-york

Thursday, October 15

Circling back to thoughts I had a while ago, it sure is depressing how much the actual technology of the database is invisible to the general population.

Everybody talks about scale. (Me included.) You know what? Fuck scale. Scale the way it gets meant in software and networks might be as bad as the Empire State Plaza as an architectural ideal. Nobody wants to live in a landscape of fascist concrete spaceships. Normal humans mostly don’t want to spend their days smack in the middle of Times Square either. These places might be a living for a few, but they’re not exactly a solid material and spiritual foundation for a thriving and humane civilization. You need fields and neighborhoods and places where things are quietly growing.

Here’s what I’d like in a database technology: The kind of scale it should primarily worry about isn’t the scale of running a single web app or social network silo with a massive userbase. Not corporate scale. Instead, it should “scale” to the level of widespread use by individuals and small organizations, at about the level of a tool like spreadsheets, except with the available guarantees and structural rigor that are so specifically lacking from passing around files full of arbitrarily input row-based data.

And by this I don’t mean that the thing in question should be a single-provider closed-source service running on other people’s computers. That’s 90% of the problem in the first place. Not that I’m claiming I know how to make it available otherwise. I think what I’m asking for is a modality for exchanging and accessing data that affords users a bunch of strong abstractions without enmeshing them in all the hard details of service configuration and messing around with files. And most of the ways we know how to do that kind of thing involve corporate scale and other people’s computers.

As an example, the technology of files is still available to most users, even though the BigCorp tech world is trying to hide it from non-developers, and seems likely to succeed once everyone uses iOS/Android-style clients where the app silo is the whole paradigm. Still, it’s taking a long time to get there.

So why are files resistant to total erasure from the user experience, despite so much effort to supplant them and so much disdain from the Ted Nelson wing of computer intellectualism? It’s not just that they’re a fundamental abstraction for building systems at this point, is it?

Maybe a big part of it is that filesystems with roughly equivalent semantics got built into all of the now-major operating system families so early in the history. No really equivalent thing happened for databases. (People with experience in the mainframe world might quibble, but that world was always about corporate scale.) And it’s kind of like you might need that level of support (at the baseline of the system in every common end-user OS) for the tech to be both user-accessible and not instantly susceptible to centralization of control.

Which is sort of a problem, since it would run directly counter to the perceived interests of most corporations with the resources to make decisions about end-user operating systems these days.

I’m probably thinking about this all wrong.

p1k3 / 2015 / 10 / 15
tags: topics/scale, topics/technical

Wednesday, October 7

I don’t really care about Jonathan Franzen, in a way that probably functions as a microcosm of all the ways I usually fail to care about literary fiction as a genre. Life is short, the available writing is well-nigh infinite, and there is so much else to be interested in one way or another. It’s even possible that, once or twice, I have defended some notion Franzen has expressed in prose as perhaps not entirely worthy of the overheated loathing his stylings tend to generate in some quarters of the internet, but that’s the sum total of my involvement to date, and in general I’m happy to keep it that way.

Still, sometimes an author or work you don’t have the energy to care about at all directly affords the pleasure of reading a really good bad review.

To wit, Barrett Brown on Purity:

There’s an old joke, which holds that in heaven, the cooks are French, the cops are English, and the engineers are German; whereas in hell, the cooks are English, the cops are German, and the engineers are French. We live in a sort of silly cultural hell where the columns are composed by Thomas Friedman, the novels are written by Jonathan Franzen, the debate is framed by CNN, and the fact-checking is done by no one.

p1k3 / 2015 / 10 / 7
tags: topics/reading

Monday, October 5

It’s the first Monday in October:

      October 2015
  Su Mo Tu We Th Fr Sa
               1  2  3
   4  5  6  7  8  9 10
  11 12 13 14 15 16 17
  18 19 20 21 22 23 24
  25 26 27 28 29 30 31

Somewhere around nine o'clock at night, I get out of the shower and develop a craving for bad junk food. I wind up driving to the Diamond Shamrock with the radio on. It’s classical I don’t exactly recognize, but it seems familiar. Jeremy’s getting out of his truck as I come around the corner, but I don’t stop to say hey. There’s a version of tomorrow where I have elected to stop by for a beer tonight, and it doesn’t start with me waking up sober, let alone at a decent hour.

The gas station has Gardettos Original Recipe Snack Mix, so I get a bag of that, the big package of Twizzlers, and a bottle of iced tea.

It’s shameful how much of the tea I consume these days comes out of a plastic bottle from a gas station. I was raised on two kinds: Made in bulk with powder, and sun tea made with Lipton teabags in a big glass jar. The important thing wasn’t so much the origin, though: It was that you made the stuff at home, or at least mixed it there, and always had a pitcher in the fridge. At least through the hot months.

Of course the hot months are just about done. There’ll be more hot days — maybe even weeks — but it’s been in the 60s, hazy gray, the light weirdly muted, the ground wet with occasional rain. I’ve been wearing long pants and shoes, digging out sweatshirts and hats. This morning I turned the heat on in my car as I rattled over to the office.

Somebody died on Long’s Peak a couple of days ago. Reading the forums now confirms a gut feeling that it’s awfully late in the year for that shit, if you aren’t a lot more experienced than me. Maybe even if you are.

p1k3 / 2015 / 10 / 5