So, what it takes to upgrade Android on an arbitrary Android device (from user perspective)? What takes to run Linux in full-fledged mode? This boils down to few things:
Upgrade kernel. This means forward-porting hardware support modules to a new kernel version. That's, when source code for such modules is available. Sometimes, it's not violating GPL (or tainting kernel, but who wants that? ;-) ). In an ideal world, that would all that's needed, but vendors don't like GPL, and try to move stuff outside of kernel. So, the list continues.
Bitblt acceleration porting
Next step is making sure that basic hardware acceleration works - accelerated bitblt/compositing (defining compositing as bitblt with alpha). This is actually pretty important step - without accelerated bitblt, Android with more or less big screen will run pretty sluggishly. Well, X Windows won't run too zippy either. Bitblt code in Android lives in /system/lib/hw/copybit.*.so and depends on gralloc.*.so from the same dir (* there is vendor/implementation identifier - Android support multiple, pluggable implementations). Needless to say, for a random device, source for these modules are not available - vendors don't have to provide it, it's Apache2-licensed, and very few choose to uphold OpenSource spirit nonetheless. So, if new Android version have changes the ABI for those modules, then oops - upgrade is "not possible" is layman terms. Of course, real men will immediately think about writing adapters, etc...
What about other X and other "foreign" windowing systems? They would need drivers/adapters too, and base all their low-level FB access based on Android's gralloc/bitblt/etc. model.
OpenGL ES porting
Last, least, but nonetheless. Curse of the modern world - OpenGL. You didn't have it on your Apple 2 (I mean Apple 2, yes, not what you thought about!) and everything was great, wasn't it? Apart from games you don't have time to play, what it's useful for? Yes, as soon as we'll stream videosignal directly into the brain, it will be useful for augmented reality, but I mean, *useful now*. They keep talking they added some hardware acceleration using OpenGL to the normal UI, then immediately say it didn't work that great, and depends on many things, because... Because OpenGL simply doesn't work that great, yeah. For example, even "accelerated", it's pretty slow, buggy, inconsistent across devices, etc, etc.
Anyway, thanks for listening to the rant. Let's de-emphasize usefulness of OpenGL, let's just take it as a challenge - vendors hacked it, so why can't hackers hack it? The basic idea is the same as with bitblt - there's an interface between closed vendor OpenGL ES implementation and Android. If interface changed, you're hosed. I mean, you write adapters. You also write adapters to make it talk with your windowing system of choice, and not Android. The core interfacing part of OpenGL ES/Android junction is EGL. How to deal about it is worth a separate post.