swizzard
Posts: 28
Joined: Wed Jul 25, 2012 11:47 pm
Location: Brooklyn, NY

Mechanical Turk, timing, and the Pi as a server

Tue Sep 04, 2012 6:45 pm

Hey guys,
I'm thinking about using my Pi as the server for the project I'm doing for my masters' thesis, but I want to make sure it can handle it. Basically, I'm planning on getting users, through Amazon Mechanical Turk, to listen to some sentences and respond as to whether or not they understood them. Crucially, the time it takes the user to press the "I get it" button is going to be a major data point in my thesis, so I need to ensure that the timing is accurate, regardless of the server load. I'm not assuming it'll be the most popular thing on MTurk, but I need to plan for worst-case scenarios. Will the Pi be able to handle the load if, say, 10+ users are being served at the same time? Or would it be a better idea to host the project on a 'real' server elsewhere?

(sort of off-topic: do you guys think I'd be able to get more accurate times doing the timing client-side, with Javascript, or server-side, with Python?)

kodess
Posts: 9
Joined: Fri Sep 07, 2012 10:26 pm

Re: Mechanical Turk, timing, and the Pi as a server

Fri Sep 07, 2012 10:31 pm

I'm assuming you are making them listen to an audio stream, which the Pi can easily handle. I think your biggest cap will be your ISP's upload speed.

As for the timing, if you make them listen to a stream, just make the "Get it" button or what ever you have capture the time of click and send it back.

sjwright
Posts: 8
Joined: Sat Sep 08, 2012 11:05 am
Location: Cheshire
Contact: Website

Re: Mechanical Turk, timing, and the Pi as a server

Sat Sep 08, 2012 11:12 am

kodess wrote:I'm assuming you are making them listen to an audio stream, which the Pi can easily handle. I think your biggest cap will be your ISP's upload speed.

As for the timing, if you make them listen to a stream, just make the "Get it" button or what ever you have capture the time of click and send it back.
I agree with kodess, the Pi should be able to do this no problem. I've done some tests getting a few people to stream a video while another executes a relatively heavy php script which compresses a large file and it didn't seem to cause any problems.

Most fibre providers in the UK will provide enough upload bandwidth to do this, mine was tested over a plusnet fibre package with a max upload of 10 megabits.

swizzard
Posts: 28
Joined: Wed Jul 25, 2012 11:47 pm
Location: Brooklyn, NY

Re: Mechanical Turk, timing, and the Pi as a server

Sat Sep 08, 2012 9:17 pm

Thanks guys! I was thinking about calling the system time when the user clicks play and again when they click "I get it" or "I don't get it." I hope that'll be accurate enough--I'll probably need msec-length measurements, and I've heard...unkind things about timing in JavaScript.

Thanks again for the votes of confidence, though.

tyr
Posts: 1
Joined: Mon Aug 13, 2012 4:23 pm

Re: Mechanical Turk, timing, and the Pi as a server

Sun Sep 09, 2012 8:11 am

msec accurate timings server-side? What kind of sorcery is this? :lol:

No, seriously, you will get by magnitudes better results when doing such measurements client side.

Also note that standard multitasking (non-realtime) operating systems won't really let you measure timings down to the millisecond accuracy (on client and server side!). Expect at least +-10 milliseconds of deviations. If you really need millisecond (or better) accuracy you should probably think about doing your research in a lab environment with something like an arduino for timing.

mpettitt
Posts: 8
Joined: Sat Jun 02, 2012 10:09 pm

Re: Mechanical Turk, timing, and the Pi as a server

Sun Sep 09, 2012 10:38 am

I'm with Tyr here - the potential network delays will far outweigh any timing issues caused by a slow computer. Personally, I'd look at running something client side with decent timer resolution (well, as decent as you'll get - it's a trade off between convenience and accuracy: more convenient for people to do over internet, more accurate with specialist hardware in a lab), such as a Java applet, where you've got direct access to the local computer RTC, then only transmit the results back over the network.

Return to “Networking and servers”