So to save the time of others, i will add my conclusions here (august 2012 state this is)
* Casual X11 stuff (2D/3D) (any graphics toolkit that comes to your head) - currently this is not accelerated 2D or 3D, very slow, puts pressure on the tiny amount of cpu power the rpi has. even slightly animated stuff brings the rpi to it's knees, movie playback on x11 surfaces brings the user interface into a halt, depending on the movie player. does provide you apis for capturing input from keyboards and mice.
* OpenGLES2 (2D/3D) - works, nice, accelerated, great, but really difficult for developers that are new to the whole opengl concept (opengles2 is purely a shader disco). if you want to make 3D stuff - this is your current choice. the demos of quake show off how much it has under the hood. more than enough from such a device. By itself is just a drawing library, handling input you have to do on your own. But as a rendering library, it is probably the most powerful tool you will have on the raspberry, ever. A lot of examples on the web (you just have to make sure you have the screen initialization of raspberry pi at the start of your code).
* DirectFB (2D) - stock version shipped with raspbian is ancient, current new branches of 1.6 are starting to show egl/gles hardware rendering support, maybe in a not too distant future this will become a real option, but currently it feels hackerisch and unstable, some work is left to be done. I did not get all their demos to work, but the ones that did work were nicely fast.
* OpenVG (2D) - hardware accelerated vector graphics, similar to DirectFB, but does not provide input device handling. Much simpler than OpenGLES, if your target is 2D development. Examples and bootstrapping are hard to come by on the internet, but luckily we have a working sample linked here in the forums in openvg topic. If you want to draw lines and boxes, some circles and have it hardware accelerated, then this is your guy for now.
* Hacking directly on the framebuffer device (2D) - if you know what you're doing, then you know what you are doing. If you dont know what you'll be doing then i would advise against (you are likely to reinvent the wheel of directfb or openvg and just be wasting your own time). Perhaps more useful for "back to the roots" style of programming experiments.
Ideas for the foundation:
* if possible, perhaps you could assist to put up some fund to support the development of accelerated 2d/3d drivers for X11, or demo out the need for a driver as a competition in some university programming context like things. we really need this to be proud of those raspberries (and 2d acceleration is even more important than 3d, for now).
* lobby the directfb package building, we could seriously use a more recent version out of the box, the current one that ships with raspbian does not offer accelerated rendering yet.
* if there's a followup device coming for the raspberry pi, please make sure the same packages of graphics would be supported
ps. if any typos were left in here, i'm all to blame, all sorts of comments are welcome.
