I went with debian / jessie (armel) so I could have OpenGL support. As in glxgears and glxinfo does stuff beyond outputting text messages. But it's not accelerated, purely software rendered which is about 11 fps (windowed) for glxgears on a good day.
The fbturbo xorg driver gives framebuffer accelerated support. Formerly sunxi? I really haven't had a pi for very long. It basically allows you to drag windows around and not have them lag behind the mouse or skew graphically while they're being moved. The xserver-xorg-video-fbturbo package isn't in debian, so it's something that you have to compile from sources if you go that route. It should be in raspbian and other official pi distros.
Then there's the dispmanx option which replaces the SDL libs, but using it is what breaks the OpenGL compatibility, but may provide opengl es via the GPU aka accelerated? The dispmanx is supposed to give hardware scaling acceleration. So you can play games full screen and not take much of a hit performance wise on a larger screen. With a caveat of the game not trying to use the new resolution natively. I'm not yet using this option. But it's nice to know of it's existence.
I didn't set my pi up for gaming or media consumption, but rather for media content creation. So I can free up the primary desktop for other things. And not bear the brunt of a system that uses a lot of power and outputs a lot of heat under load. Running debian jessie gives me easier access to available video codecs for encoding. And it's familiar ground so compiling povray v3.7 from sources was less annoying. Although not as easy as it should have been. Yeah it takes 10x's to 30x's longer to render povray scenes or encode videos on a pi, but it does so quietly and without heating up my room.