The 700 mA supply might be marginal (just from what I've read), but I've been having similar problems.
I'll start another thread for my problems, assuming I don't first find a thread that matches completely, but here's some info on my problems:
5VDC phone charger power supply, rated at 1000 mA with a decent (based on my voltage drop measurements) USB to micro USB cable. At full load with X started, 4.82 VDC between TP1 and TP2. 3.32V and 1.81V out of the regulators. At the CLI, (without X started), the LAN connection (100 Mb/s to home router) works fine. Pinged google for 2 hours (thanks google) with no problem. When I do "startx", pretty much immediately "smc95xx" error messages show up in dmesg output. Evidence that major blocking is occurring when any network ops are attempted and maybe just in general.
Tried increasing vm.min_free_kbytes to no avail.
Running the debian6-19-04-2012 image on a 2 GB SanDisk SD card.