Announcing libperipha and cortex-uni-startup projects

More and more ARM Cortex based boards appear for use by OpenSource/OpenHardware/Maker communities, and
frequently asked question in response to a new announcement is often "Is this supported by open-source toolchain?". Many people know that gcc supports ARM, including Cortex-M, very well, and many projects now lean towards using https://launchpad.net/gcc-arm-embedded as the package of their choice.

But compiler is only part of the toolchain - there should be also at
least basic hardware definition headers which would allow to access peripherals of particular chip. There was big step in that direction done by ARM Ltd, itself, by releasing CMSIS library for the Cortex-M CPU cores under open-source license, though that didn't go without controversy, see comments to that post and followup.

That's great step, but again, it's only part of the story - while it allows to access Cortex-M core registers which are common across all MCUs, there're more to MCU than just CPU - it's all the peripheral blocks. Even basic things as GPIO and timers are vendor- and model-specific, and periphery is what differentiates one vendor's  offering from another's.

And other vendors don't rush to follow ARM's example, so few vendor-specific CMSIS packages are under OpenSource license (one great example of the contrary is Energy Micro EFM32 CMSIS - it's great because they open-sourced it even before ARM). One  good approach to use OpenSource-friendly MCUs for your projects, and everyone is encouraged to do so. Unfortunately, that may not be practical or even possible (I wish I could by EFM32 chips in my part of the world).

So, the idea is to provide open-source MCU headers for use in OpenSource projects - until all vendors get smart to provide them. That's how libperipha project was born. Note that it's not limited to ARM Cortex-M MCUs, though support for them was a primary motivation.

There's final, small, but just as important piece which construes MCU toolchain - startup code. Without it, your application simply won't run. Another project, cortex-uni-startup (unified Cortex startup) was created to cover that. As excellent article by Ivan Sergeev explains, Cortex-M was specifically re-engineered to allow startup code to be written without any single line of assembly! cortex-uni-startup explands on this to provide easily reusable Cortex-M core startup code (usable on any Cortex-M device, but without support for peripheral-specific interrupts), then extends it to cover interrupts of particular MCU model.

These two projects are separate to cover differences in scope (cortex-uni-startup is Cortex-M specific, libperipha is unbound), licensing (cortex-uni-startup core is public domain, libperipha is BSD and others), and usage (libperipha is gross database which you probably will just want to use, then hack, cortex-uni-startup is nice learning tool of beauty of Cortex-M architecture, which you welcome to glance over before using - you won't be overwhelmed, as it's small).

Last final note - while these projects are conceived to be comprehensive, I don't have an aim to sit and add each and every possible bit of every MCU there. Instead, it is intended to be easy do add missing pieces to it on demand - right while you're working on some applied project and find such missing pieces. Then, if people find this to be indeed so, and project overall useful and contribute, then they indeed will acquire comprehensive coverage of MCUs. So, have a look, give it a try, shared feedback, spread the word, contribute patches, and overall, enjoy ;-).

Growing GPL/FOSS Loyalty Among Chinese ARM SoC Vendors

There are visible changes in how Chinese ARM SoC vendors deal with Open Source license compliance (words) and Open Source software at all (spirit) during this year. Not only western resellers of devices based on Chinese SoCs release GPL compliance packages, but many popular SoC companies themselves established GPL compliance/loyalty/support pages. Few go beyond just adhering to word of GPL by releasing kernel source blobs (by blobs I means large tarballs devoid of any development history and support files), but also establish git repositories, release Android trees, set up public bugtrackers.

Of course, they are still not exactly GPL compliant - scrutiny of any releases would likely show missing parts, binary objects for drivers, etc. But there's definitely a difference between plucking out WiFi driver out of flash image you have on you device and downloading "official binary" from a SoC vendors. It's also a good start, and there's a hope the situation with complete source availability will improve. And as everyone watching the area knows, the industry at the whole needs to improve - Western companies may comply with GPL by providing source for kernel shims, but that doesn't help users to support vendors' hardware if they just move binary blobs to userspace.

The changes discussed didn't come by themselves and at once. Lot of people paved the road to this via gpl-violations.org, via various leaks, etc. But critical change came with emergence of Open Hardware startups. They are led by people who strongly believe in Open paradigm, but also to do their ODM-like business, they need to provide sufficient documentation/sources to the customers. So they have little choice but request that from upstream SoC vendors, in redistributable form. So, thanks Rhombus-Tech, Olimex, Cubitech!

Anyway, here're the links to Chinese companies' GPL loyalty pages I know so far. Additions and corrections are welcome.

• Telechips 
• Rockhip
• Allwinner
• Amlogic
• VIA/Wondermedia

And as extra, community projects to support some of vendor SoCs from above list:

• Rockchip 
• Allwinner 
• VIA/Wondermedia 

Braindead X11 Clipboard Exposed

I've been using Linux with X11 as GUI system as my primary and only work system for like 6 years now. It's great, much better than another world - asymptotically.  But there're few issues which sustain - an  being hated - for years. One of them is how clipboard in X11 works. It works illogically, unpredictably, and non-reliably. Based on many years' background black-box analysis, there appears to be more than one clipboard buffer, and some apps apparently take clipboard with themselves when they quit.

It finally came a right moment to both be hit by it and have time to investigate. This blog post shares the same sentiment as I above, and confirms suspected design braindeadness.

To quote:

Three selections are defined in the ICCCM: CLIPBOARD, PRIMARY, and SECONDARY, each of which behaves like a clipboard in its own right:

• CLIPBOARD: traditionally used when text is copied and pasted from the edit menu, or via the Ctrl+C and Ctrl+V shortcuts in applications that support them.
• PRIMARY: traditionally used when a mouse selection is made, and pasted with middle-click or Shift-Insert.
• SECONDARY: ill-defined secondary selection. Most applications don't use it.

Be gone soon, X11, be replaced by something sane.

Ubuntu Spyware

On few occasions I noticed that some "zeitgeist" daemon running on my Ubuntu 12.04 system takes a lot of CPU cycles, so it had to be killed. Today, clearing off my disk, I also noticed that it eats noticeable chunk of it (100MB+). I proceeded to investigate what Zeitgeist is, finally. Here some links and comments:

1. Tutorial on GNOME Activity Journal, Zeitgeist and Sezen describes it as a universal user's interaction logging system.
2. I was never asked if I want to install or enable such system and/or logging during Ubuntu install/upgrade.
3. It turned out that daemon was installed and running automatically, but frontend app (gnome-activity-journal) wasn't installed.

So, this thing collected my activity without my consent and behind my back, and clearly not for the purpose of letting me use that data. This all pretty matches the definition of trojan spyware. In addition to that it also eats system resources and makes system running slow.

Few people share the concern that Zeitgeist is exactly spyware and not some productivity tool, future of Linux user interface and other marketing bullshit from Gnome3 and Canonical:

• http://martesmartes.blogspot.com/2011/12/zeitgeist-spyware-delivered-with-linux.html
• http://askubuntu.com/questions/227111/can-i-trust-ubuntu-even-though-stallman-calls-it-spyware-what-about-zeitgeist
• http://forum.prisonplanet.com/index.php?topic=191577.0

How to remove Zeitgeist: http://martesmartes.blogspot.com/2012/06/even-nicer-little-improvement-in-linux.html

Overall situation with spyware on Ubuntu, from Richard Stallman: http://www.fsf.org/blogs/rms/ubuntu-spyware-what-to-do .

Links for June 2013

• PCB design for high pin count SMD components: ever wondered why they "stupidly" don't put via in BGA pads? Here's why, vividly explained: http://blog.screamingcircuits.com/via_in_pad.html (other blog posts are insightful too).

Buffalo SkyStation CWR-GN150S Teardown

Walking last few hours thru Hong Kong streets and molls during last visit, I was looking for some impulsive shopping, and bought micro-router having following labels: Buffalo SkyStation CWR-GN150S, Hornington NE-GN150S, etc. What lured me was its really small size (5.5x4.5x1.7cm) and 2 Ethernet ports. It was also couple of bucks cheaper than TL-WR703N. I was thinking where's the catch, and later got it - it doesn't have USB host port (micro USB it has is only for power).

All manual for the router is in Japanese/Chinese, same for firmware, no English firmware on Internets, actually, barely any English pages at all. Teardown was even more disappointing: it has only 16MB of RAM (bye, OpenWRT), and turned out it uses RTL8196C SoC, infamous for being of MIPS-Lexra architecture, which is MIPS subarch crippled due to patent trolling between MIPS vendors. There's barely any support for this subarch from gcc and OpenSource projects, I found just one brave guy who tries to change the situation:

• http://ncrmnt.org/wp/2011/08/30/rtl8196-vs-openwrt-2/
• https://github.com/nekromant/linux-rlx-upstream

 Well, I added hacking on all this stuff to my (very long) queue. In the meantime, just posting this to let other folks to google it up.

Peripheral Template Library - attempt to efficiently capture common microcontroller algorithms in C++

There were not many posts in this year, partly because I decided to concentrate on writing code and hacking things instead. One of the project I find myself hacking regularly during last half-year is a library modestly called Peripheral Template Library.

Ever since initial amazement with Arduino warn off, it became clear that while it provides concise and easy to use (to learn, to understand) API, it is to schematic, implemented too inefficiently, libraries have reuse, compatibility and configurability problems, and all those things are not portable across really wide MCU range and otherwise not production ready (because of all those efficiency, reuse, etc. issue), but rather a toybox.

Trying to implement things without Arduino overhead, there was growing dissatisfaction with the need to slash code when moving from one MCU to another (due to economical and availability issues, or plain for creative reasons). It became clear the need to abstract things away, but without losing efficiency. The only way to achieve that is metaprogramming. As nowadays nobody writes MCU software in LISP, I turned to C++ - after almost 10 years break I guess.

Many embedded programmers shrug at C++, and well, I did that for any C++ at all for quite some time. But modern C++ is not that "object-oriented syntactic sugar" which your old Turbo C++ 1.0 was (yeah, last one which could run on your old XT, because 2.0 required 286 and protected mode memory extender). Modern C++ is multi-paradigm language, of which "runtime" "object orientation" is just small part. The core part is however compile-time type-based metaprogramming template engine, that's something which allows to achieve aforementioned abstraction/efficiency ratio (and we don't talk here about "C" efficiency here, but something closer to Assembler level).

That's how PTL was born. The idea was to wrap the most core part of any MCU - GPIO (General Purpose Input/Output), and on top of it implement support for standard communication protocols - UART, SPI, I2C, 1-Wire, etc. Everything implemented as templated classes, so can be easy specialized to use hardware blocks of particular MCU. Generalized cross-MCU was one of top goals from the start, i.e. more attention was given to design and implementation of classes which capture basic MCU functionality which is available in any MCU, rather than adhoc peculiarities of particular MCU. The latter can be wrapped too of course - on demand.

Past GPIO and communication protocols, handling timers and delays was second goal. Support for CPU-based delay - both (perfect-cycle) static and dynamic was implemented, then basic timer interface was added, with delay on top of it, with all 3 kinds of delays providing the same interface and thus fully replaceable and combinable.

What comes out of it is available at https://github.com/pfalcon/PeripheralTemplateLibrary . MSP430, AVR, STM32 and EFM32 are supported (all to the different level though, but the basic functionality as described above is available for all by now). The grand aim is to design framework to support all possible (and impossible) MCUs ;-).

There're still log of refactors to do and functional block of individual MCUs to wrap. Some aspects, like clock tree management needs more though and design. Then, there's a need to add bindings/drivers for specific devices, I did that for few already, but of course that's area where I won't be able to compete with existing code, so I'm looking into finding good codebases to port.

License-wise, Peripheral Template Library is released under LGPL3 license, but I'm already thinking about adding linking exception and/or switch to BSD-like license altogether.

Here's a usual LED blinker example:

#include <gpio.hpp>
#include <cpu.hpp>
#include <board.hpp>
#include <delay_static.hpp>
#include <delay_time.hpp>

using namespace PTL;

typedef TimeDelay<board::freq, StaticDelay> delayer;

int main()
{
    cpu::init(cpu::DEFAULT);
    board::LED::port::enable();
    board::LED::output();
    while (true) {
        board::LED::high();
        delayer::delay_ms(500);
        board::LED::low();
        delayer::delay_ms(500);
    }
}

This will compile and work out of the box on any of supported boards - specific LED to use (board::LED) is configured in board config header.

Let me know what you think about PTL. For me, it's definitely exciting, innovative area - not many libs provide such efficient support with such wide heterogeneous coverage, and you really do architecturing and design here, not just dumb boring code. (I admit that partly PTL effort is driven by dissatisfaction of technical inefficiencies, imperfection and warts in "one level bigger" Linux world.)

Links for May 2013

I didn't post "Links" topics for a while - that's of course not because there're no interesting projects around, but because there're too many and you can't keep up with them ;-). Anyway, not collecting links to them doesn't help either, so:

• openFrameworks - C++ framework to glue together ~20 well-known technologies and libraries, mostly for visualization, UI and multimedia (all collectively "creative arts coding")
• Cinder - Another C++ contender from the similar area
• Emscripten - "An LLVM-to-JavaScript Compiler" (C, C++ compilation to JavaScript)
• asm.js - "An extraordinarily optimizable, low-level subset of JavaScript", from Emscripten authors.
• "List of languages that compile to JS"- from CoffeeScript authors.

On animation in user interfaces

I always considered animation in GUI a whimsical feature wasting resources (in particular, power in mobile devices) and oftentimes just covering triviality of contemporary software for mobile systems (IPhone and Android, hello), and inconveniences user with slow, repetitive, boring effects.

Live and learn, I found usecase where lack of animation makes user interaction unobvious and complicated. It's not about going from one square page to another though, but about applying transformation to graphs.

It all started with my desire to study dependencies among Android modules (which is task towards my "Packaged Android" project). I found adepends.py tool (my fork), which is very nice and dumps dependency info in .dot format for rendering with Graphviz. Looking at few graphs it became obvious that for comfortable browsing there should be interactive graph viewer with few capabilities which I summarized in Stackoverflow question.

My searching didn't turn suitable viewer, but fortunately I found xdot.py, which is extremely cool graph viewer app written in Python, which was suitable base for easy implementation of needed features.

And here story comes to the topic of the post - once I implemented prototypes of  those features, they indeed proved to be really helpful, but didn't give aesthetic satisfaction - that's because when hiding a node for example, Graphviz's dot completely overhauls rendering of a graph, so user loses track of context he used with. Having shape of a graph morph from old to new state would help a lot.

Actually, animation can help even with simpler graph operation, for example going from one node to another in a complex graph - seeing a ride over an edge which connects them helps intuitive understanding a lot. See ZGRViewer's "link sliding" feature in a video on Youtube:

Simplified form of going from node to node is implemented in xdot.py, except that it moves between nodes via shortest path, which doesn't capture context of curved edges. (Btw, ZGRViewer may be full of cool ideas, but its UI grossly sucks comparing with XDot, literally, it's outbreak from last century). I seriously consider implementing non-linear animation approximating Bezier curves for XDot.

So that's it - going from hating animations to wanting implement non-trivial animations myself in couple of days ;-). (That's also the first time I embed video?)

Overriding install prefix in build systems

Following up with reviewing capabilities of various build systems in previous post, recently I looked how to build all packages with custom --prefix and friends values. This is continuation of Optware for Android project, where I'd like to achieve ability to install on non-rooted device.

Here are the results:

• I was surprised at first, but OpenWRT has --prefix hardcoded as /usr when calling configure. Fortunately, that call is in common file, so it would be easy to introduce configurable variable instead. However, it turns out that install routines, which are individual for each package, have /usr hardcoded either, so each recipe needs to be patched still, which is problem for maintainability and/or upstreaming.
• Looking at Optware, situation is even worse - there, there's even less common infrastructure, and even configure call is hardcoded in each script with --prefix=/opt.
• Buildroot, being prototype for OpenWRT, has the same hardcode at /usr.
• I obviously turned to OpenEmbedded at that point (well, I tried Yocto this time) - it for sure always have had nice configurable variables for prefix and all other layout variables (--sysconfdir, etc.). Turned out, while variables are there, OE/Yocto is not ready out of the box to full configurability of both prefix and base_prefix. There were few issues seen:
• Compiler search path issues. Some aspects of search paths (mostly related to shlib linking) appear to be not stored by gcc when configured during cross-compiler, and require command-line overrides. OE passes --sysroot on each invocation, but I had to add few -rpath-link directives to get it right. While it took some time to figure the need for -rpath-link, I'm still not sure I figured where to put them (well, per upstreamability criteria).
• Some packages still hardcode paths, especially related to base_prefix. Sometimes it's a bit harder than trivial to patch (like with busybox).
• Ordering issues. Some packages implicitly assume that base_prefix and prefix are different, and filter filesets during packaging by file prefix. This doesn't work if base_prefix==prefix, so requires reshuffling filtering patters and/or package order (fortunately, OE has well-defined package splitting order).