Monday, January 21
#CircuitPython2019
I’ve been spending a decent chunk of my paid time in recent months on the
CircuitPython ecosystem, doing things like testing libraries
against Adafruit-Blinka, a shim that mimics CircuitPython’s core modules on
Linux systems.
(For background: CircuitPython is a fork of MicroPython aimed
at educational users, beginner hobbyists, etc., with full-time development
staff sponsored by Adafruit.)
The community’s been asked to write some stuff about hopes for the language in
2019:
As 2018 comes to a close we like to reflect on how the year went and set goals
for 2019. In the last two years (2017, 2018) this has been a blog post by me
(Scott aka tannewt). For 2019, we’d like to do it a bit differently. This time
we’d like everyone in the CircuitPython community to contribute by posting
their thoughts to some public place on the Internet.
I guess I’ll start with some reflections on my experience in 2018 and then list
some thoughts about where things are headed. None of this should be taken too
seriously — my role on this one is that of a low-level software bureaucrat,
and it’s an extremely safe bet I don’t know what I’m talking about.
reflection 0: python could be worse
Python and I have always been sort of an uncomfortable fit. I’ve used it here
and there — Ansible stuff for configuring web servers, one-offs like a
static HTML image gallery generator — but I’ve never
warmed up to it much. I never got over the whitespace thing, and the claims of
“executable pseudocode” that hover around the language strike me as more than a
little exaggerated. Anyhow, I have shell and Perl for small-time utility
scripting, and the large projects I’ve worked on have mostly been PHP. So for
one reason and another, until recently I’d probably written more lines of, say,
QBasic or vimscript than I had Python.
Still and all, I’m well past the point in my working life where I’m capable of
feeling anything like love for most software. Grudging toleration is about the
best I manage these days. And it’s clear that Python has a set of virtues in
the space it occupies.
The syntax is for the most part easy to grasp, modulo the brittleness of
indentation and a few other infelicities. The built-in collection types
(lists, dicts, tuples, sets) aren’t as easy for a beginner to use as something
like PHP’s swiss-army-knife of an array type, but they’re also much less
conceptually muddled. The language seems to encourage a straightforward
procedural style with some light object orientation on top. Even if this isn’t
exactly considered cool, there are far worse ways to program.
In fact, Python’s aesthetics are often relentlessly un-hip, which is probably a
useful bulwark against the violently unmaintainable cleverness that infests so
many language cultures in 2019.
Packaging isn’t a lot of fun to figure out, but it basically works once you get
the right boilerplate in place. PyPI is a mess, but as other language-specific
package managers & repositories have shown us repeatedly, it could be
substantially worse.
I think most of my substantive ongoing concerns with Python are around the
way bugs seem to proliferate on type boundaries, the tendency of code to spew
exceptions all over the landscape in conditions the developer didn’t quite
anticipate, and the swampy nightmare that is the ongoing 2-to-3 version
transition. (Some of these are mitigated to an extent by the smallness of most
CircuitPython codebases, and the 2-to-3 problem is sidestepped by way of aiming
only for Python 3 compatibility.)
Very broadly speaking, Python’s problems are mostly in classes of problems
shared by that family of dynamic, garbage-collected interpreted languages that
came to prominence in the 90s and early 2000s: Perl, Python, PHP, Ruby, etc.
By the same token, Python shares many of the positive qualities of its
relatives. There’s a vast set of undertakings that are essentially painless in
these languages and excruciating in an environment like C / C++ / Arduino.
All of this to say: I was initially skeptical, but it seems like a Python is a
reasonable thing to build a hardware development ecosystem around.
reflection 0a: the workflow is pretty good
Plug in a board, get a USB drive you can drop code onto and a REPL you can
access over serial.
Ten years ago this would have blown my mind, and in 2019 it still feels like
kind of a revelation. In most ways, it’s a vast improvement over the
ergonomics of the Arduino-style compile-upload-debug cycle. If you mess around
with hardware and haven’t tried this, it’s worth playing with.
reflection 1: the hardware isn’t quite there yet
…but it’s real close. At this point, I’ve written CircuitPython code for:
- ATSAMD21E18 Cortex M0 boards (48 MHz, 32KB RAM):
- ATSAMD21G18 Cortex M0 boards (48 MHz, 32KB RAM):
- ATSAMD51J19 Cortex M4 boards (120 MHz, 192KB RAM):
- Various and sundry iterations of the Raspberry Pi
Of these, it feels like the SAMD21 boards just don’t have enough RAM to do much
that would require more than one driver library. For instance, I recently
offered to build my dad a remote thermometer for his pump house. The Feather
M0 LoRa boards seem like they should be ideal for this, since all I need to do
is read a temperature sensor on one board, transmit it, and display the
temperature on the other end. I just couldn’t seem to get the headroom to
import, let alone use, both Adafruit_CircuitPython_RFM9x and
a driver for any of the several displays I tried. Ultimately I gave up on
CircuitPython for the project, and plan to complete it in Arduino / C++ with
the RadioHead library instead.
I had similar experiences when working up the Glitter Positioning
System and an as-yet undocumented USB pedal-input
project. In those cases, I wound up switching to the Feather
M4 Express, and the contrast is dramatic. The SAMD51 feels like a Real
Computer for this use case, with enough memory to easily handle a LoRa radio,
GPS unit, LSM9DS1 9-DOF, and an RGB LED ring for display output all
at once.
At the low end of the available hardware, you can glimpse the platform’s
possibilities and do some basic projects, but there’s a good chance you’ll be
frustrated trying to go anyhere beyond that. At the higher end, it’s eminently
possible to do real work.
reflection 2: this could be pretty important on linux systems
This brings me to the Raspberry Pi, where (strictly speaking) CircuitPython
doesn’t run, but the project as a whole has quite a bit to offer.
Adafruit-Blinka is a shim that aims to provide most of the core
CircuitPython modules on some common small-board computers with GPIO pins,
which means that CircuitPython drivers and helper libraries can be used from
regular CPython 3.x. At heart, it’s a collection of hacks, but this
nevertheless feels like a game changer for a certain class of hardware projects
on small Linux systems. It’s trivial to use a lot of this stuff, and that in
turn opens up a ton of connections to the rest of the stuff you can do with a
high-level language in a Linux environment.
At a time when it’s nearly impossible for most users to exercise any control
over most of the computing hardware in their lives, this sort of thing feels
like at least a small step in the right direction.
reflection 3: the community stuff is well-handled, except…
Having a code of conduct, doing most development work in the open, welcoming
beginners, involving the user community in code review, etc.: In general I
think this is a commendably well-managed software project, and plenty of FOSS
efforts could learn something from the overall tone it takes.
On the negative side, I actively dislike Discord with its CPU-eating
Slack-but-for-gamers vibe, and I continue wishing that GitHub were not eating
all public-facing software development, but I recognize the accessibility these
platforms afford and acknowledge that my side of these questions is presently
in a state of utter defeat, so I won’t spill any more virtual ink on the
subject in this context.
thoughts / hopes for 2019 and points beyond
Ok, so some notes on stuff I hope to see improve on the scale of about a year:
Stability: Things are much better than when I first tried
CircuitPython, but there are still glitches to reckon with.
Occasionally a board will have to be power-cycled to get code to run
correctly. Connecting to the REPL over serial sometimes hangs mysteriously
or prints a bunch of garbage. Things (unsurprisingly) get downright wacky
when you run out of memory. The USB mass storage stuff exhibits some
unpredictable behavior—writes sometimes fail without much observable
rhyme or reason. I’m sure all of this will get better; this is in many
ways still a young project. Still, there’s a lot of usability to be gained
just by making the whole thing a couple of notches less flaky.
Asynchronous processing / multitasking / parallelism / event handling:
I have no idea what shape it should take, but people keep wanting some
kind of paradigm for handling asynchronous events in something closer to
realtime, and I don’t think they’re wrong to want it. It’s remarkable what
you can accomplish just by polling inputs in a tight loop, but sooner or
later you want something better. Smarter people than me hash some of this
out in this GitHub issue.
CPython compatibility: This is an explicit goal of the project, so I
don’t think there’s much danger of it getting lost in the shuffle. I do
think it’ll improve things the closer it gets to 100%.
Linux platforms: Right now, the Raspberry Pi is still more or less the
only first-class citizen in terms of Adafruit-Blinka support. It’d be great to
see a couple of other stable, widely available platforms added to that
list.
Library management: The experience of installing modules for use on the
Raspberry Pi is, perhaps oddly, a lot less painful than installing them on
CircuitPython boards, since you can just use pip3
. It’d be interesting
if there was a nice clean way to use PyPI for dropping libraries into place
on the CIRCUITPY drive or in a project directory. (Maybe this is handled
by tools like Mu? I should know, but I don’t because I keep just
using Vim.)
Security and privacy: The way things are going, there are eventually
going to be a lot of network-exposed microcontrollers running some Python
variant. Some of the people responsible for those systems will be
experienced software professionals, which means that they’ll do as well on
this front as software professionals usually do (i.e., completely
terrible, but many of them will know that security is a consideration and
when they use the tech to violate your privacy they’ll usually be doing it
on purpose).
I’m not, however, convinced that the rest of CircuitPython’s target
demographic is equipped to reason about these problems at all. As embedded
systems continue getting smaller, more powerful, and more ubiquitous, this
is going to matter more and more. The organizations and projects teaching
people how to put computers in everything need to start thinking a lot
harder about how dangerous it is to put computers in everything, and
communicating that danger to the people they teach. I’m not sure what form
that takes with CircuitPython, but I know that it should be on people’s
radar as the platform expands to more devices that handle wifi, bluetooth,
LoRa, etc. Particularly given the security track record of web projects
built on similar languages.
Community involvement: Right now, this is an Adafruit joint, through
and through. That’s fine as far as it goes, but it’ll be interesting to
see whether it ever truly makes the transition to something that a bunch of
different orgs and individuals outside of the Adafruit bubble are involved
in. I think that’d be a good thing, but there’s an inherent tension
between the kind of tight control that’s built this project according to a
well-defined plan and the diversified set of inputs that give projects like
this a life of their own well beyond the interests of a particular company.
Anyhow. That’s probably enough rambling for one post. To reiterate, I
probably don’t know what I’m talking about.
adafruit, circuitpython, glitter-positioning-system, hardware, python, technical :: p1k3 /
2019 /
1 /
21