Your site would not come up for me.
I am surprised. See the stats at http://stats.pingdom.com/cm97jhu1pp23/929603
. You can also take a look at http://www.webpagetest.org/result/130813_9N_RHW/
for some additional performance results.
Pingdom runs test from many parts of the world and they average about 670 ms (range from 1.5 sec to .2 secs). I see similar results from webpagetest.org site as well. You should try for your website as well to compare.
I am trying to be convinced about the ability to run a fast (under 1 second response time) web site hosted using RPI and WP.