Also, did you set up the VM image as an ARM computer, or as an X86 hardware? If it was x86, then, yeah, I can see 256MB being pretty limiting, unless you start dropping down to things like a simpler Window manager, pulling out unneeded stuff, etc.
But ARM code usually compiles to a LOT smaller memory footprint than x86. So I'm sure that between using and ARM environment, and going with lower memory footprint WM, and applications, you should be able to have a very capable system.
You just gotta learn how to effectively make use of what you have and not let it be bloated with a bunch of stuff you don't need. You strip out flashy backgrounds, animated window effects, having a ton of apps open at the same time, huge scrollback buffers, etc.
Heck, I've gotten Linux running on an old HP iPaq (not iPAD!) that had a whopping 32 (64? it's been like 8 years so forgive me) MB of RAM, and ran a pretty intensive Java SWING application on it!
I'd suggest grabbing one of the VM's that somebody here has put together for RPi dev and see if that works any better for you.