I finally found time to read Cliff Brake's excellent article summarizing his work on adding realy-working C#/.NET support to OE. Well written, it gives info why C# is useful choice for embedded systems, compares it to C and Python (great choices!) and details how to use it in OE. What of course would be *really* interesting to know is bug/feature (in the sense of performance, memory/disk footprint, compatibility with existing standards, etc.) comparison of current open-source C# and Java implementations, to be able to make choice for some new, independent project. Well, that's not going to be that easy so far.
One point where .NET has win over Java is "by default" portability/device coverage. It's pretty to write simple applications which run both on full-fledged .NET and CompactFramework, which means it's easy to develop small utils on desktop, and have them run without extensive debugging on PocketPC device. Back in the days I still bothered with trying to make PocketPC do what I need, that was a big win! Example of real-worldly usable app written in such way is here. Of course, back then, it was very hard to get full benefit of that opportunity, due to the way vendor provided toolset for .NET development - overloaded with GUIs and unimportant features, it possibly helped someone with a good favorite editor, but made integration work a great chore. Reading thru Cliff's article, one may once again rejoice that we have OpenEmbedded, which handles task of integrating 3rd-party dependencies into the project and complete deployable solution a breeze.
Returning back to Java, we are going to have fresh Java support too, thanks to the work of
Robert Schuster. Hope it will land soon in OE mainline, so the application developers/integrators for Angstrom, OpenMoko, etc. will have unprecedented choice of tools available at their fingertips!
Saturday, December 29, 2007
Sunday, December 16, 2007
Angstrom 2007RC1 is here!
Yesterday, Angstrom 2007 Release Candidate 1 was tagged in OpenEmbedded repository and corresponding images where built using autobuilder and uploaded to the autobuild area. They are waiting for machine mentors to test them and move to public download area. The latter was cleared from the older snapshots, and there even small FUD started among the users, so I've posted a small announcement to describe release-time order of things. That said, mentors are acting slow, with only your humble servant have moved h3900, h4000, and hx4700 images to there final location.
A word about versioning - as was announced, final release will be dubbed 2007.12, while RC go as 2007.11.
There're more exciting things to come, at least in PocketPC/WindowsMobile area - RFC was posted on very easy installation procedure for novice users, and that's in addition to LiveRamdisks, of course.
A word about versioning - as was announced, final release will be dubbed 2007.12, while RC go as 2007.11.
There're more exciting things to come, at least in PocketPC/WindowsMobile area - RFC was posted on very easy installation procedure for novice users, and that's in addition to LiveRamdisks, of course.
How to find out if USB mobile phone data cable is based on PL2303
Recently, I hunted for a PL2302-based USB-Serial cable to make a generic serial cable for NAS hacking. Of course, it's not easy to figure out on what based some cable sold in a small shop next door. During my searches, I even found that phone vendors have gone so bold that include PL2302 right into the phone, and sell passive cables ;-\. In the end, I bought a cable with transparent casing thru which I could clearly see what chip is there ;-). But based on the packaging of this one and other packages I saw, I could make following generalizations for future searches:
- If instructions on package mention Prolific (PL2302 vendor) site for future driver upgrades, that's good sign ;-)
- If it mentions using SER9PL.INF for Windows driver install, that's also likely it.
Saturday, September 29, 2007
More Angstrom LiveRamdisks
Last week, I spent some time to elaborate and automate LiveRamdisks production with the aim to mass-produce them from now on. Taking into account previous agreement with Angstrom maintainers to host them at the main Angstrom download area, this means that eventually there will be a user-friendly way to try a recent Angstrom snapshot on any supported PocketPC/WindowsMobile device.
With these new scripts, Angstrom x11 LiveRamdisks for the following devices were generated and uploaded:
Current problem with this is that uploading images in different formats (and LiveRamdisk is just another such format of course) means trafficking the same data over the net again and again, and I don't have such a big pipe. So, instead, I created a set of image conversion tools which would allow to upload master image format (apparently, tar.bz2) and convert it to other formats and make LiveRamdisk out of it right on the Angstrom hosting service
Right now, I'm building current Angstrom snapshosts, and going to try these tools on them...
With these new scripts, Angstrom x11 LiveRamdisks for the following devices were generated and uploaded:
- h2200
- h3900
- h4000
- h5000
- hx4700
Current problem with this is that uploading images in different formats (and LiveRamdisk is just another such format of course) means trafficking the same data over the net again and again, and I don't have such a big pipe. So, instead, I created a set of image conversion tools which would allow to upload master image format (apparently, tar.bz2) and convert it to other formats and make LiveRamdisk out of it right on the Angstrom hosting service
Right now, I'm building current Angstrom snapshosts, and going to try these tools on them...
Friday, September 7, 2007
Back from hiatus
Having spent August on vacation and later working with my GSoC student, Maria Zabolotnaya, I now gradually come back to speed on embedded stuff, even though I likely won't have as much time to work on this as before.
The first aim is to get OE build again, which is "of course" broken after being away for some time and then coming back ;-). Then, process all the backlog of HH.org kernel patches and cleanup tasks. Further aims will be to automate process as much as possible (for example, I finally want to setup local Continuous Integration for OE builds, like I did for HH.org kernel), and upgrade HH.org kernel to 2.6.22 (though to that time 2.6.23 likely will be out).
The first aim is to get OE build again, which is "of course" broken after being away for some time and then coming back ;-). Then, process all the backlog of HH.org kernel patches and cleanup tasks. Further aims will be to automate process as much as possible (for example, I finally want to setup local Continuous Integration for OE builds, like I did for HH.org kernel), and upgrade HH.org kernel to 2.6.22 (though to that time 2.6.23 likely will be out).
Wednesday, July 25, 2007
Finally out of the list of pressing OE issues
With the last changes to properly support modprobe 2.6 for kernel module configuration directive, I can't remember other pressing issues with OE which would prevent its proper and flexible usage for distro engineering. Among the other infrastructure changes done within the last half-year with I was involved are:
- Support for building efficient initrd/initramfs images
- Flexible and extensible initramfs to boot rootfs from different media
- Adding support for machine screen properties
- Image building speed improvements
- Alleviating maintenance effort for kernel porting projects with lots of supported devices (using defconfigman)
Thursday, July 19, 2007
Modular initramfs for OE
While all new initramfs-related features are yet need to be described, they are already put to work. Alex Osborne of Hack'n'Dev created a modular initramfs system to serve all the device booting needs. We discussed design principles and came to following tentative picture:
WANTED: modules to boot via WiFi, with WEP and WAP support, and to boot via GPRS ;-D
- initramfs in question handles just the task of mounting rootfs and booting from it.
- Doing so, it tries to follow both existing kernel behavior and interface, like command line parameters. For example, NFS boot uses standard root=/dev/nfs and nfsroot= parameters. Where extended feature is added, it tries to mimic existing parameters. For example, loop device boot uses root=/dev/loop and looproot=.
- Support for different boot methods is implemented by modules with more or less well defined interface. It's possible to construct initramfs images with the given feature by combining modules.
- Generally, new and extra features are expected to be added using modules, to keep design clean.
WANTED: modules to boot via WiFi, with WEP and WAP support, and to boot via GPRS ;-D
Initrd and initramfs with OE: snippet 1
One of the former weak areas of OpenEmbedded was creation of small, fine-tuned, (hand)optimized initrd and initramfs images. However, during few last months, the situation vastly improved, and by now OE offers all tools needed to create initrd/initramfs per your purposed and requirements. I would like to describe them, and as that would be quite a bit of contents, going to do it describing features one by one, in "snippets".
The first is:
Building initramfs with OE and everything you wanted to know about initramfs but didn't bother to figure out yourself.
IMAGE_FSTYPES = "cpio.gz"
Yes, that's all what you need to create an initramfs image, voila. After that, you can use resulting image instead of conventional initrd (in ext2 or some other format). For the kernel to be able to boot it, you need to have CONFIG_BLK_DEV_INITRD=y in its config (contrary to its name, this option actually means "support external initrd *or* initramfs images"). As initramfs support is unconditionally builtin into kernel, you don't need any filesystem enabled for initramfs support.
(Note that above is written for standalone initramfs image which is used in initrd-like manner. Support for initramfs which are linked into the kernel image so far is not provided OE, but would be pretty easy to add. Still, it's of more interest to vendors which work on specific products than OpenSource distros. By the way, mentioned config option is not required for builtin initramfs.)
Note that execution of initramfs starts with /init. If you want to rebuild initrd (or otherwise conventional rootfs image) as initramfs, you'll need to pass following parameter on kernel command line to have it started properly: "rdinit=/sbin/init".
Final note is that contrary to popular belief, initramfs does not completely covers and supersedes initrd functionally. Initramfs has relatively big memory requirements, which precludes usage of big initramfs images on low-memory systems. Specifically, initramfs is compressed, and needs to be uncompressed during startup, which already limits its size to at most half of the available RAM. But actually, ramfs, which is used as backing store for initramfs content, is pretty inefficient for such type of content (static), and that imposes even more limits. Also, my experiments show that kernel has poor error handling while uncompressing initramfs, and out of memory condition makes it just hang. So, when you need to fill memory up with lots of content, nothing would replace an initrd with builtin compression like cramfs or squashfs. Granted, that's rather adhoc need.
The first is:
Building initramfs with OE and everything you wanted to know about initramfs but didn't bother to figure out yourself.
IMAGE_FSTYPES = "cpio.gz"
Yes, that's all what you need to create an initramfs image, voila. After that, you can use resulting image instead of conventional initrd (in ext2 or some other format). For the kernel to be able to boot it, you need to have CONFIG_BLK_DEV_INITRD=y in its config (contrary to its name, this option actually means "support external initrd *or* initramfs images"). As initramfs support is unconditionally builtin into kernel, you don't need any filesystem enabled for initramfs support.
(Note that above is written for standalone initramfs image which is used in initrd-like manner. Support for initramfs which are linked into the kernel image so far is not provided OE, but would be pretty easy to add. Still, it's of more interest to vendors which work on specific products than OpenSource distros. By the way, mentioned config option is not required for builtin initramfs.)
Note that execution of initramfs starts with /init. If you want to rebuild initrd (or otherwise conventional rootfs image) as initramfs, you'll need to pass following parameter on kernel command line to have it started properly: "rdinit=/sbin/init".
Final note is that contrary to popular belief, initramfs does not completely covers and supersedes initrd functionally. Initramfs has relatively big memory requirements, which precludes usage of big initramfs images on low-memory systems. Specifically, initramfs is compressed, and needs to be uncompressed during startup, which already limits its size to at most half of the available RAM. But actually, ramfs, which is used as backing store for initramfs content, is pretty inefficient for such type of content (static), and that imposes even more limits. Also, my experiments show that kernel has poor error handling while uncompressing initramfs, and out of memory condition makes it just hang. So, when you need to fill memory up with lots of content, nothing would replace an initrd with builtin compression like cramfs or squashfs. Granted, that's rather adhoc need.
Wednesday, July 18, 2007
Finally a kernel hackers' initrd image in OE!
When I only started with Linux-on-PDA hacking, I, as people usually do, combed internet for any kernel/initrd images which were marked as being for the PDA model I had at that time. In such way I picked up few initrd's, and after practically comparing among themselves, selected one for standard hacking use, like kernel testing. A bit later I augmented it by installing some missing stuff (like mc), and seeing that many people are not so thorough and searching the net, put it up for others' use. That's basically what I've been using since them and suggesting to others who needed environment to test some kernel feature with minimal userspace involvement.
Of course, that situation was rather ridiculous - using some random binary-only image instead of OE-built is not perfect at all, and I was just hoping that someone else would go for that. Well, with the latest changes in OE to facilitate creating efficient initrd/initramfs images, I finally got to this myself. So, here's now such image, codenamed 'devimage'. Consequently, recipe to build it as a complete image is called 'devimage-image.bb'.
Its purpose defined as: provide environment which by default runs only minimal userspace (busybox sh and dropbear now, to support different connection methods), but offers all needed "kernel userspace" utilities from udev to specific utils for ALSA, IrDA, Bluetooth, WiFi, etc.
At the current stage it's not as complete as that, so help with filling it right is welcome (caveat: size should be kept in mind).
Of course, that situation was rather ridiculous - using some random binary-only image instead of OE-built is not perfect at all, and I was just hoping that someone else would go for that. Well, with the latest changes in OE to facilitate creating efficient initrd/initramfs images, I finally got to this myself. So, here's now such image, codenamed 'devimage'. Consequently, recipe to build it as a complete image is called 'devimage-image.bb'.
Its purpose defined as: provide environment which by default runs only minimal userspace (busybox sh and dropbear now, to support different connection methods), but offers all needed "kernel userspace" utilities from udev to specific utils for ALSA, IrDA, Bluetooth, WiFi, etc.
At the current stage it's not as complete as that, so help with filling it right is welcome (caveat: size should be kept in mind).
Friday, July 13, 2007
It's all firefighting, right?
Well, time for a nice "depressive" post ;-).
So, at the beginning of week I decided to port GParted. Was more or less easy, just a fix for an obvious thinko from the author regarding C++ std namespace ;-). However, it failed on linking stage with infamous rpl_malloc() and friends not resolved for libparted. Looking thru cofigure log, that's of course result of crosscompiling assumptions, but OE has defines for the corresponding tests in autoconf site files, so what was actually happening is shipped parted 1.8.7's configure not loading these site files.
It also turned out that autoreconf'ing of this parted version doesn't work, as it requires autoconf 2.61 and friends as minimal, while OE has 2.59. Trying autoreconfing 2.61 (which is DEFAULT_PREFERENCE=-1 in OE) lead to the same issue with site files loading. At this point I submitted bugs on this, and raised issue on #oe. Actually I didn't even expect immediate big attention, but it caught developers' eye, and with the help of XorA and hrw, to the end of the day we had a solution. (And the root of issue was bug in autoconf 2.61. They used "shell programming patterns" (cough) you really don't want to use in your scripts, and the result is proverbial - they outslied themselves).
Well, on the droop of day, GParted was built. I run in QEMU and was surprised that UI was pretty barely usable on QVGA. Run on a real device to try a real card brought a gore however. First of all, GParted didn't detected my SD card at all. I even went looking at GParted source for how it scans devices, but then run parted on it. "Overlapping partitions now supported" was abort message. Overlapping partitions? Me? No! Wrong call! Fortunately, running fdisk showed instead of the senseless bold accusation the fact that my camera indeed tried to dig weird holes in parttable, apparently confused with ext2 partition there ;-)
Well, finally GParted was run, bliss. But just going thru menus once made them no longer accessible. Already trained on such stuff with LiveRamdisks work, I knew it's out of memory. Running ps aux showed some unusual suspects. Killing them (btw, GParted's RSS was "merely" 15Mb comparing to those 6Mb's. Ah, it's all shared libraries, ok, ok, I shut up.), I finally had the thing live. Now closer look look showed it doesn't recognize EXT2 partition (at least FAT it did). The same in parted. Have no idea is it camera hosed it or it's just proverbial GNU software quality. It also turned out that the only operation I was able to do is to change boot flag for a partition ;-). I tried to format unrecognized partition as EXT2 for example, and it failed after shuffling a card for a bit. Getting to HTML report it laid in the home dir surfaced "sh: nice: Command not found". Oh, oops, dear app ;-).
Finally, I wanted to make a screenshot, but gpe-scap failed to write it, complaining that it can't do UTF-8 -> ISO-8859-1 conversion for PNG CREATOR chunk (soft laugh).
So, what are the conclusions? X is not an ultimate solution, but merely an option. Lure to run large body of existing, and updated, software is still a lure, as besides this opportunity X brings bloat, which on resource-constrained system vice-versa inhibits ability to run all that wealth of soft well.
Also, all those kernel-hacking games are really old. And I hope that everyone on the block understands by now that what they get once kernel is well and polished is havoc in userspace (this applies to new players like Neo1973 too). So, it's time (yes, finally now, if not long ago)
to leave machine-level feudalism behind, and bring all Linux PDA/phone communities together, to work on common aims. And OE is well established community crossroad project, so implementation-wise, that would mean making sure that any more or less developed device is available in OE, and that any newcomer for a questions "Is X supported?" or "Where can I find stuff for X?" could be answered "Use OpenEmbedded." Yes, right that - use OpenEmbedded, period. Not even "You can use OpenEmbedded, just copy one to another one, and tweak this and that".
So, at the beginning of week I decided to port GParted. Was more or less easy, just a fix for an obvious thinko from the author regarding C++ std namespace ;-). However, it failed on linking stage with infamous rpl_malloc() and friends not resolved for libparted. Looking thru cofigure log, that's of course result of crosscompiling assumptions, but OE has defines for the corresponding tests in autoconf site files, so what was actually happening is shipped parted 1.8.7's configure not loading these site files.
It also turned out that autoreconf'ing of this parted version doesn't work, as it requires autoconf 2.61 and friends as minimal, while OE has 2.59. Trying autoreconfing 2.61 (which is DEFAULT_PREFERENCE=-1 in OE) lead to the same issue with site files loading. At this point I submitted bugs on this, and raised issue on #oe. Actually I didn't even expect immediate big attention, but it caught developers' eye, and with the help of XorA and hrw, to the end of the day we had a solution. (And the root of issue was bug in autoconf 2.61. They used "shell programming patterns" (cough) you really don't want to use in your scripts, and the result is proverbial - they outslied themselves).
Well, on the droop of day, GParted was built. I run in QEMU and was surprised that UI was pretty barely usable on QVGA. Run on a real device to try a real card brought a gore however. First of all, GParted didn't detected my SD card at all. I even went looking at GParted source for how it scans devices, but then run parted on it. "Overlapping partitions now supported" was abort message. Overlapping partitions? Me? No! Wrong call! Fortunately, running fdisk showed instead of the senseless bold accusation the fact that my camera indeed tried to dig weird holes in parttable, apparently confused with ext2 partition there ;-)
Well, finally GParted was run, bliss. But just going thru menus once made them no longer accessible. Already trained on such stuff with LiveRamdisks work, I knew it's out of memory. Running ps aux showed some unusual suspects. Killing them (btw, GParted's RSS was "merely" 15Mb comparing to those 6Mb's. Ah, it's all shared libraries, ok, ok, I shut up.), I finally had the thing live. Now closer look look showed it doesn't recognize EXT2 partition (at least FAT it did). The same in parted. Have no idea is it camera hosed it or it's just proverbial GNU software quality. It also turned out that the only operation I was able to do is to change boot flag for a partition ;-). I tried to format unrecognized partition as EXT2 for example, and it failed after shuffling a card for a bit. Getting to HTML report it laid in the home dir surfaced "sh: nice: Command not found". Oh, oops, dear app ;-).
Finally, I wanted to make a screenshot, but gpe-scap failed to write it, complaining that it can't do UTF-8 -> ISO-8859-1 conversion for PNG CREATOR chunk (soft laugh).
So, what are the conclusions? X is not an ultimate solution, but merely an option. Lure to run large body of existing, and updated, software is still a lure, as besides this opportunity X brings bloat, which on resource-constrained system vice-versa inhibits ability to run all that wealth of soft well.
Also, all those kernel-hacking games are really old. And I hope that everyone on the block understands by now that what they get once kernel is well and polished is havoc in userspace (this applies to new players like Neo1973 too). So, it's time (yes, finally now, if not long ago)
to leave machine-level feudalism behind, and bring all Linux PDA/phone communities together, to work on common aims. And OE is well established community crossroad project, so implementation-wise, that would mean making sure that any more or less developed device is available in OE, and that any newcomer for a questions "Is X supported?" or "Where can I find stuff for X?" could be answered "Use OpenEmbedded." Yes, right that - use OpenEmbedded, period. Not even "You can use OpenEmbedded, just copy one to another one, and tweak this and that".
Wednesday, July 11, 2007
Linux installers bumbo-jumbo
On Monday, I decided, as a kind of change-activity rest, to review the situation on Linux installers market. I had closer look at what big players use, as well as googled and searched SF.net. Well, a generic, truly modular, lighweight and elaborated installer doesn't seem to exist. So, if we speak reuse, then considering an installer of specific distro is apparently the way to go. And big players are:
Well, ground-level solution is just present users with generic GUI partitioning app, and let them do partitioning themselves, right? So, I decided to quickly try how GParted runs on a common PDA, and that's all subject for another post.
- RedHat systems with Anaconda, written in Python (PyGTK) but with bunch of C code too. Seemed like nice design, I even checked out CVS repo, and tried to make it show me some dialog without complete build. Nah, written in Python doesn't automatically mean it's written nice. Building it also didn't work - it depends on bunch of RH-authored libs, which don't even packaged in Gentoo for example.
- Debian with Debian Installer. This is apparently largely written in C, as one of the design goals was to support all architecture set of Debian, in particular lowy ARMs. The default version is text-mode, but recently(?) they got also GUI version (GTK+, and they build it against DirectFB). It is also said to be modular and reusing other Debian components, like debconf, and indeed, looking at the SVN repo from bird's view in browser was scary enough that I didn't proceed to specifics.
- Ubuntu's Express-Espresso-Ubiquity. Reuse at run. Uses Debian Installer as the backend, with (apparently Python) frontend.
- Gentoo Installer. This one will be finished when it will be finished, plus also I managed to spot bunch of UML diagrams but not the language it's written in.
Well, ground-level solution is just present users with generic GUI partitioning app, and let them do partitioning themselves, right? So, I decided to quickly try how GParted runs on a common PDA, and that's all subject for another post.
Sunday, July 8, 2007
Driver for ACX100/PCMCIA
Thanks to Fabrice Crohas, htcblueangel maintainer, who have been tackling at acx_cs driver since we were able to achieve first WiFi ping on h4000 and htcbluenagle with the help of devmem2, the PCMCIA support for ACX100 WiFi driver now works! It took quite some time to figure out what issue with PCMCIA was and make it work after first slavemem-connected device (hx4700) had first ping, but finally we're there - ACX is support on the great majority of devices which have it (though it is still has something in store, as couple of newest device have it connected via SDIO).
Working WiFi also means that all hardware on h4000 is supported now, and it has A- support status per HH.org classificaion, with the only missing feature being a native flash bootloader for Linux. Of course, not everything works great yet, or even fully tested, like flash write at all (but it is known to work for h6300 which has the same DiskOnChip and uses the same driver). ACX itself for one needs lot of work to make it support suspend/resume properly and PM in general.
But all those are just things to test/bugs to fix, and not white spots in hardware information/support. We're past age of "Linux will never be able to support that device completely" for another bunch of devices, and the future is bright! ;-)
Working WiFi also means that all hardware on h4000 is supported now, and it has A- support status per HH.org classificaion, with the only missing feature being a native flash bootloader for Linux. Of course, not everything works great yet, or even fully tested, like flash write at all (but it is known to work for h6300 which has the same DiskOnChip and uses the same driver). ACX itself for one needs lot of work to make it support suspend/resume properly and PM in general.
But all those are just things to test/bugs to fix, and not white spots in hardware information/support. We're past age of "Linux will never be able to support that device completely" for another bunch of devices, and the future is bright! ;-)
Monday, July 2, 2007
First steps at making OPIE fit well with OE filesystem structure
One of the requirement set for OPIE to be supported for Angstrom was compliance with standard GNU/Linux installation location ($prefix and friends). There was bug in OE long enough for this, with few developers considering how to proceed with this.
I finally found time to test some ideas from it, and initial conversion from /opt/QtPalmtop to /usr went pretty smooth requiring besides rebuild few fixes to packages where the former path was harcoded in the literal form. Resulting opie-image booted in QEMU quite well, so I went with committing it, bumping snaphost date for OPIE 1.2.3-pre to 20070702 at the same time.
Hope other developers will help with testing it, and then the conversion needs to be fixed by moving data and plugin to appropriate locations (now they live at places like /usr/pics). This would take a bit more effort exactly because of this dichotomy - data vs plugins, with first should be living under /usr/share, and the latter (ideally) under /usr/lib. I wonder, if a solution like making QTDIR, etc. environment variables to accept list of paths is good enough (well, it is generic from system design point of view, but adds extra runtime overhead).
I finally found time to test some ideas from it, and initial conversion from /opt/QtPalmtop to /usr went pretty smooth requiring besides rebuild few fixes to packages where the former path was harcoded in the literal form. Resulting opie-image booted in QEMU quite well, so I went with committing it, bumping snaphost date for OPIE 1.2.3-pre to 20070702 at the same time.
Hope other developers will help with testing it, and then the conversion needs to be fixed by moving data and plugin to appropriate locations (now they live at places like /usr/pics). This would take a bit more effort exactly because of this dichotomy - data vs plugins, with first should be living under /usr/share, and the latter (ideally) under /usr/lib. I wonder, if a solution like making QTDIR, etc. environment variables to accept list of paths is good enough (well, it is generic from system design point of view, but adds extra runtime overhead).
Wednesday, June 27, 2007
Running "user" QEMU emulation on Fedora Core 5
I have few boxes around with different OSes/distros and try to use them in reasonable manner. For a some time I contemplated setting up continuous integration for OpenEmbedded, and the box suiting that purpose runs FC5 here.
Well, soon after running a test build on it, I hit issue that OE's binary locale generation, done by running target localedef under QEMU "user" emulation (qemu-arm) errors out there, and on closer inspection, just segfaults. Taking some time to run it thru gdb and strace, I found that it happens after mmap2() syscall, and in seemingly random places in code. More close look at the stuff showed that address parameter to faulty mmap() kind of correlates with VDSO location reported by gdb. Gotcha! So, somehow guest process manages to overwrite VDSO as used on FC5 system.
Now it took some time to figure out what's happening, but it turned out that FC5 applies whole bunch of patches (mostly labelled "execshield" and "xen") affecting VDSO handling to 2.6.15 kernel, when final set of VDSO randomization patches, etc. was not yet in mainline. In other words, FC developer took their own try on solving VDSO issues, with the outcome described above.
Well, it took only couple hours more to rebuild kernel without those patches and teaching it to pickup original initrd (maybe I should have rebuilt it instead, I just didn't do that before). And now I have pristine and full-feature OE builds on FC5 system, voila! ;-)
Well, soon after running a test build on it, I hit issue that OE's binary locale generation, done by running target localedef under QEMU "user" emulation (qemu-arm) errors out there, and on closer inspection, just segfaults. Taking some time to run it thru gdb and strace, I found that it happens after mmap2() syscall, and in seemingly random places in code. More close look at the stuff showed that address parameter to faulty mmap() kind of correlates with VDSO location reported by gdb. Gotcha! So, somehow guest process manages to overwrite VDSO as used on FC5 system.
Now it took some time to figure out what's happening, but it turned out that FC5 applies whole bunch of patches (mostly labelled "execshield" and "xen") affecting VDSO handling to 2.6.15 kernel, when final set of VDSO randomization patches, etc. was not yet in mainline. In other words, FC developer took their own try on solving VDSO issues, with the outcome described above.
Well, it took only couple hours more to rebuild kernel without those patches and teaching it to pickup original initrd (maybe I should have rebuilt it instead, I just didn't do that before). And now I have pristine and full-feature OE builds on FC5 system, voila! ;-)
Tuesday, June 26, 2007
Porting "Industrial" PDA?
On weekend on local second hand hardware resale board I spotted offer for an industrial PDA with a builtin barcode scanner. Second-hand PDA prices are generally crazy in this part of the world, but this was barely reasonably priced, and it included bare WinCE 4 OS (not PocketPC or Windows Mobile), which made it rather useless in hands of layman, and thus offered some room for bargain.
Being a big fun of "scientific organization of life", I contemplated getting a barcode scanner for quite some time, but anything besides pop supermarket "gun model" USB/RS232 is rather pricy/unavailable here. So, after all, I went with buying the piece. It was announced as PXA255, I checked that fact on vendor site, and didn't bother to check it during hand-off, but soon caught the fact that it's merely a StrongArm. Checking vendor's site closer, I understood that sloppy guys had a StrongArm device, which they later upgraded to PXA, added "-foo" to the name of model, and killed old one from the site. Lart such vendors, what can I say... Either way, the winner is PD-264 from Italian vendor EIA. I even wanted to call a guy to return it back, but then reconciled it as a good opportunity to revamp SA support in HH.org kernel. And unlike h3600 which I had before, PD-264 offers such opportunity in comfort - it has CF & SD slots, plus 64Mb RAM and 32Mb ROM. It lacks any networking interfaces though, and UART is taken by the scanner, so it's not that cool in the end, and debugging may be hard actually. Still, I got a barcode scanner with a whole PDA builtin cheaper than my previous favorites in shopping list, FLIC Bluetooth scanner, or Baracoda Pencil.
Now, this would be my first PDA I port from scratch (vs picking up existing port for maintenance), and I'm going to blog the process here, as it was raised many times that there should be a howto for newcomer porters. (Though I checked Michael Opdenacker's presentation, and it's [still] quite good, keeping in mind that it's presentation, not howto).
Being a big fun of "scientific organization of life", I contemplated getting a barcode scanner for quite some time, but anything besides pop supermarket "gun model" USB/RS232 is rather pricy/unavailable here. So, after all, I went with buying the piece. It was announced as PXA255, I checked that fact on vendor site, and didn't bother to check it during hand-off, but soon caught the fact that it's merely a StrongArm. Checking vendor's site closer, I understood that sloppy guys had a StrongArm device, which they later upgraded to PXA, added "-foo" to the name of model, and killed old one from the site. Lart such vendors, what can I say... Either way, the winner is PD-264 from Italian vendor EIA. I even wanted to call a guy to return it back, but then reconciled it as a good opportunity to revamp SA support in HH.org kernel. And unlike h3600 which I had before, PD-264 offers such opportunity in comfort - it has CF & SD slots, plus 64Mb RAM and 32Mb ROM. It lacks any networking interfaces though, and UART is taken by the scanner, so it's not that cool in the end, and debugging may be hard actually. Still, I got a barcode scanner with a whole PDA builtin cheaper than my previous favorites in shopping list, FLIC Bluetooth scanner, or Baracoda Pencil.
Now, this would be my first PDA I port from scratch (vs picking up existing port for maintenance), and I'm going to blog the process here, as it was raised many times that there should be a howto for newcomer porters. (Though I checked Michael Opdenacker's presentation, and it's [still] quite good, keeping in mind that it's presentation, not howto).
Monday, June 25, 2007
Power management for GPIOs
I just committed to HH.org kernel CVS Roman Moravcik's patch for improved power management for rx3000, and among other things, he took care to setup currently unused input GPIOs as outputs, which is known trick to cut unneeded current drain on them. That's something what all other ports should be reviewed for.
Sunday, June 24, 2007
Defconfigman support for S3C24xx ports
Thanks to Michal Panczyk, support for S3C24xx CPU and ports (Eten G500, iPaqs rx1950 & rx3000) added to Defconfigman. Corresponding static_defconfig's already in HH.org kernel CVS, will go to OpenEmbedded soon. (They need more review/cleanup though.)
First Angstrom LiveRamdisk
LiveRamdisk is a pun at LiveCD, hinting that it must be something which allows users to try Linux easily without much preparation and that entire distro will be loaded from RAM.
For PocketPC case, it is a single big .exe file which when being started from normal WinCE environment, loads initrd (or a something alike) and kernel into memory and boots the latter.
I wanted to do something like that for a long time, and rejoiced when Kevin O'Connor, another HaRET maintainer, added ability to create single-exe bundle to HaRET 0.4 this winter. Unfortunately, nobody picked up this possibility, so I decided to give it a try, finally. Alas, it turned out to be not so easy when came to loading realistic (~20Mb) images. So, bunch of hacking was needed both on HaRET and Linux side.
But finally, first LiveRamdisk, an Angstrom X11 snaphost for hx4700, was released two days ago. Besides announcing it on Angstrom mailing lists, I also did some cheap PR on local forums. Well, two days later, the reaction is rather mild - there were only 3 downloads. At the same time, I was pretty surprised that my unofficial Familair 0.8.4 release for h4000 was downloaded 900 times in total. So, time will tell ;-)
For PocketPC case, it is a single big .exe file which when being started from normal WinCE environment, loads initrd (or a something alike) and kernel into memory and boots the latter.
I wanted to do something like that for a long time, and rejoiced when Kevin O'Connor, another HaRET maintainer, added ability to create single-exe bundle to HaRET 0.4 this winter. Unfortunately, nobody picked up this possibility, so I decided to give it a try, finally. Alas, it turned out to be not so easy when came to loading realistic (~20Mb) images. So, bunch of hacking was needed both on HaRET and Linux side.
But finally, first LiveRamdisk, an Angstrom X11 snaphost for hx4700, was released two days ago. Besides announcing it on Angstrom mailing lists, I also did some cheap PR on local forums. Well, two days later, the reaction is rather mild - there were only 3 downloads. At the same time, I was pretty surprised that my unofficial Familair 0.8.4 release for h4000 was downloaded 900 times in total. So, time will tell ;-)
Blog start and older news
I always treated blogs (and generally diaries) skeptically, but well, all tools have their uses. So, I'm going to start blogging on my Embedded Linux (well, probably not only embedded) endeavors.
And first of all, older news, which I kept in my Handhelds.org wiki page.
2005-11-13: Subscribed to the h4100-port mailing list.
2005-11-29: Confirmed by bisecting that PXA suspend/resume broke in 2.6.13 comparing to 2.6.12. Turned out ot was fixed in 2.6.15
2005-11-29: Subscribed to the kernel-discuss mailing list.
2006-02-03: Proposal for asic3_mmc cross-machine refactor.
2006-02-05: Faced strange issue with ASIC3 suspend/resume, took few months to resolve ;-)
2006-05-30: Posted patch for h4000 ASIC3 suspend/resume problem, turned out to be as "simple" as initializing MSCx PXA registers after resume, what Linux kernel doesn't do (leaving that to bootloader). Anyway, this was second big issue I solved, and after that I felt "force" ;-).
2006-06-03: Started to work on final version asic3_mmc cross-machine refactor, based on the previous experiments.
2006-06-19: Final version of asic3_mmc cross-machine refactor posted.
2006-06-21: Granted write access to !HandHelds.org Kernel CVS repository.
2006-08-20: Posted RFC for common button map for all HH.org PDA-type machines.
2006-08-24: Granted write access to OpenEmbedded Monotone repository.
2006-09-20: Unofficial Familiar 0.8.4 release for h4000.
2006-10-03: Received donation of h3900 device to support elaboration of 2.6 kernel port for it and advance userspace/distrubutions support. Contributor: Mr. Alex Deucher. Thank you!
2006-10-11: Posted patch for common button map for all HH.org PDA-type machines.
2006-10-18: Received donation of hx4700 device to support elaboration of 2.6 kernel port for it and advance userspace/distrubutions support. Contributor: preferred to stay anonymous, but many thanks!
2006-11-09: Upgraded HH.org kernel CVS to 2.6.18.
2006-11-14: Finished implementation of asic3_keys driver and making sure it works well with gpio_keys.
2006-11-21: With having common linux-handhelds-2.6_*.bb and consistent names for all iPaq machines, first phase of cleaning up HH.orh-maintained machines in OE.dev is complete.
2006-12-02: Upgraded HH.org kernel CVS to 2.6.19.
2006-12-19: Angstrom upgraded to 2.6.16-hh8 kernel with common buttonmap support.
2006-12-22: h3900 reached full "C" status, with partial "B" status, in other words, basic PDA functionality works!
2006-12-28: h3900 2.6 port added to OE.dev, requested support by Angstrom.
2007-01-04: Initial release of [http://sourceforge.net/projects/defconfigman/ defconfigman]
2007-01-10: Started to revamp OPIE support in OE.dev, starting with cleaning up qte patchset.
2007-01-18: Received new contribution: h3600 and palmtt2 devices. Thanks! Now I have access to PXA250, PXA255, PXA270, StrongArm, OMAP devices.
2007-01-20: asus620 port updated in CVS. Thanks to Vincent Benony, asus620 maintainer.
2007-01-21: defconfigman kernel's sound tested to work with GPE. Both h3900 and h4000 in OE.dev now have defconfigs generated by defconfigman. h4000 updated to use modules for sound.
2007-02-16: Upgraded HH.org kernel CVS to 2.6.20.
2007-03-05: Continuous Integration system for kernel tree set up, http://handhelds.org/~pfalcon/busyb/buildlogs/
2007-03-17: Proposal to use Defconfigman-generated configs for kernel CVS, to improve consistency and CI coverage.
2007-04-11: GPIODEV API RFC posted to LKML/LAKML
2007-04-26: All 30 ports in HH.org CVS are buildable now (for 2.6.20)!
2007-05-01: SoC base drivers RFC posted to LKML.
2007-05-07: Upgraded HH.org kernel CVS to 2.6.21.
2007-05-11: ipkg speedup patches committed to OE.dev
2007-05-15: Mystery of PCMCIA ACX is solved, first WiFi ping on h4000!
And first of all, older news, which I kept in my Handhelds.org wiki page.