Citizen Engineer

Battle of the Blobs

by ladyada@alum.mit.edu and fill@2600.com

In a previous column entitled "Patently Hacking" (34:2), we rejoiced with the upcoming (now present!) opportunity to create a patent-free, licensing-free, open-source MP3 device.

For two decades, if you played (decoded) MP3s on a device, you needed to buy a licensed chip or pay mp3licensing.com.

That is over.  Now it's time to build.

Lucky for us, despite the patent constraints on distribution, open-source MP3 co/decoding stacks have been available for years.

We used the open-source Helix (www.helixcommunity.org) CODEC, which was written by RealNetworks in 2000 (ironically, much of our work in porting was working out the data buffering code...) and wrapped it up nicely to be used with any Arduino-compatible ARM board: github.com/adafruit/Adafruit_MP3

So of course, now we're ready to design our own MP3 player boards!

But, while we were working on the wrapper, we started thinking about microcontroller/computers, blobs, and open stacks.

Over the last ten years, we've seen a few properties of the electronics market collide.

First, the Moore's law-esque rapid increase in processor capability tied with plummeting costs to where a 120 MHz 32-bit microcontroller with 1 MB of Flash and 256 kB of RAM is maybe $3.

Second, the ascendancy of ARM as the core of choice (don't get me wrong, there's other awesome cores, but 70 percent of the 32-bit market share is ARM!  1 MB of Flash... what on Earth would you need so much space for?

The answer is software stacks - but not just any software - specifically, software-to-replace-hardware.

Rather than hand-code all logic on a microcontroller and rely on assistive chips to manage things like, say, a USB stack, MP3 decoding, or low-power radios, the extra processing power in small computers is being used to reduce materials cost.  Much of this is possible because we've sort of all agreed to use 32-bit ARM processors - the Helix MP3 CODEC is optimized to use the FPU on ARM chips.

This is good, but has a catch.

When all functionality is frozen in hardware, there's no way to interface to it other than the predefined interface.  For example, the STA013, an $8 MP3 decoding chip designed in 2004, had a few pins that you would clock MP3 data to.  MP3 data in, audio data out.  That's it.  While it's opaque, it's also, in a sense, complete.

With closed software stacks, the hardware interface is often hidden, replaced with a software API you are forced to use.  You can't see anything beyond the outer-surface of the API, so we call it a blob.

Sometimes the API is good, but as all good hackers know, the best and juiciest parts of an API are what is not documented or exposed.  That's where hacking and coding come in: if we can pull apart or reverse engineer the blob, we can do more with the hardware because we're not limited to whatever the blob-writer envisioned.

All it takes is one person with concerted effort to create an open stack to release a ton of innovation.  And we're seeing more open stacks that are well written, documented, and supported, to replace the vendor-specific closed-source stacks and blobs.

For example, Nordic Semiconductor is a manufacturer of Bluetooth Low Energy (BLE) chips.

These chips contain both an ARM core and a 2.4 GHz radio.  The radio is just a radio - the BLE protocol stuff is all managed by a "soft device," a blob that your compiler can link to.  The soft device works well, but it could be better and it could be open.

So along comes the Apache Foundation and the Mynewt group (mynewt.apache.org).  They have written a lovely open-source, Real-Time Operating System (RTOS) that contains open stacks for Bluetooth Low Energy, replacing the proprietary and closed soft device blob.

Their stack is faster, and is more flexible, giving the coder more control over her application.  Not to be outdone, the Linux Foundation has their own open wireless stack, Zephyr Project (www.zephyrproject.org), which has wide processor support.

Another example we bumped into is interfacing with the capacitive touch peripheral on the ATSAMD21, a chip we use in a bunch of our microcontroller boards.

Capacitive touch lets you make a pin turn into a person sensor, which is great for adding a non-mechanical or non-standard interface - say if you want to make a banana into a touch sensor.  But, Atmel, the maker of the chip, has decided not to document the registers of the captouch controller.  Instead, they provide you with... a blob!

Due to the way our project was structured, we couldn't dynamically link to their blob, and besides, it was forcing us to use the hardware in a clumsy way.  So, we reverse-engineered the blob using a disassembler, to break their API function calls down to individual register reads/writes.  Our new API (github.com/adafruit/Adafruit_FreeTouch) is lighter and, while not as fully-featured, is fully-open for others to build upon.

We've seen some really wonderful new and open interfaces to existing hardware.  Here's some of our favorite open stacks!

How to get started?

There are a lot of different ways to attack a closed blob.

If you have a software blob like a .so or a firmware binary, check out radare (rada.re), an open-source disassembler, or IDA (www.hex-rays.com), a commercial decompiler.  (As decompilers and disassemblers go, ARM is a well-supported target.)

If there's a hardware interface, use a logic analyzer to grab data traces and look for patterns.

USB is really easy to attack, with a hardware "man-in-the-middle" device or by hooking into your operating system's USB host stack to see the commands fly by, then rewrite them in libusb!

Citizen, there is no better (or more fun!) way to use your curiosity and hacking skills than to create new open stacks and interfaces.  You may not get rich and famous, but you will get to show off your keen skills and see some really cool projects.  And best of all, you'll join a vibrant hacking community that offers a future free of blobs.

Good night and good luck.
Return to $2600 Index