User avatar
winkleink
Forum Moderator
Forum Moderator
Posts: 278
Joined: Mon Nov 07, 2011 9:12 am
Location: UK
Contact: Website

Test Results for performance of Scratch on a Model A

Tue Jun 04, 2013 8:26 am

With the new Raspbian 2013-05-25 image Scratch is suppose to be faster due to the work on the Pixman library.
So, I thought I would do a comparison between the new image and the 2012-07-15 image.
With the expectation that Scratch will mainly be used in schools and that these may be kitted out with Model As I did the testing on a Model A, so limited to 256MB of RAM

Video Below:
https://www.youtube.com/watch?v=YcPrdxszXgg

I download the images and flashed them fresh to 8GB SD cards - same make and model.
Then on boot up expanded the partition.
No updates were done and only a wireless keyboard/mouse USB dongle was plugged into the single USB port.

The Scratch program has 3 sprites + Stop Sprite. Each main sprite has 2 costumes and they move around the screen bouncing off the edge.
1 Sprite (Felix) also increments a global variable by 1 for each step.
This whole thing is repeated 500 times, giving a long enough time for the task to get a fair comparison. Shorter and the difference in speed would have been harder to measure. Longer and I would have fallen asleep.


Raspbian 2012-07-15 Image
Login was rejected a few times before I was allowed it. This is with the default username/password and using the same keyboard as for the 2013-05-25 image so it looks like some USB based issues were resolved since the 2012-07-15 image was released.
Scratch felt sluggish in loading and resizing the program to full screen
To keep it consistent I ran the program full screen. On the 2012-07-15 image full screen did not stretch but just put a 480 x 360 window in the middle of the screen
Test ran in 1m 22s


Raspbian 2013-05-25 Image
Login worked first time every time.
Scratch felt responsive in loading and resizing the program to full screen
Full Screen stretched to the resolution of the screen, so was true full screen experience.
You will see the updating of the Variable is not real time. It looks like variable updates on screen do not happen every time the variable is changed.
Test ran in 0m 51s
37% faster than the 2012-07-15
An amazing increase in performance purely based on a software update

Raspbian 2013-05-25 Image Turbo Mode (1GHz)
I used raspi-conf to set the the Raspberry Pi to Turbo Mode to see what the maximum performance is.
Login worked first time every time.
Scratch felt really responsive in loading and resizing the program to full screen
Full Screen stretched to the resolution of the screen, so was true full screen experience.

You will see the updating of the Variable is not real time. It looks like variable updates on screen do not happen every time the variable is changed.

Test ran in 0m 33s
60% faster than the 2012-07-15 (non-Turbo)
35% faster than 2013-05-25 (non-Turbo)

Compared to stock 2012-07-15 image the new 2013-05-25 image with Turbo mode set is nearly 3 times faster in Scratch.
All without a hardware upgrade and with user changeable settings.

The only limitation appears to be the speed at which variable are updated on the screen. The variables are updated in the background it is purely the screen update that is slow.

For most thing I'd expect the variable issue will not be a problem so a fantastic result.

Scratch is now definitely usable on the Raspberry Pi.
Twitter:Winkleink
Google+: Winkleink

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Tue Jun 04, 2013 9:06 am

37% faster than the 2012-07-15
And some people said my counting "1 and 2 and 3 and...." wasn't accurate enough :)

http://www.youtube.com/watch?v=PXdxgjamFi4 :)

Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

User avatar
winkleink
Forum Moderator
Forum Moderator
Posts: 278
Joined: Mon Nov 07, 2011 9:12 am
Location: UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Tue Jun 04, 2013 9:10 am

simplesi wrote:
37% faster than the 2012-07-15
And some people said my counting "1 and 2 and 3 and...." wasn't accurate enough :)

http://www.youtube.com/watch?v=PXdxgjamFi4 :)

Simon
I saw your video yesterday and thought it might be good to do a bit more testing.

Did you see the Variable not being updated on screen as well or was that just me?
Twitter:Winkleink
Google+: Winkleink

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Tue Jun 04, 2013 4:38 pm

Ye- saw the variable thingy - can you post your script please?

Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Tue Jun 04, 2013 9:34 pm

I've used this for testing https://www.dropbox.com/s/tbajr0b7j40zk ... %20test.sb and it certainly looks like the variable watcher isn't being updated everytime it changes

Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

timrowledge
Posts: 1237
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 1:39 am

Correct; I put in a change that reduces the update rate, the one submitted some time ago by Kazuhiro Abe. It may push things too far in the direction of saving time…

Two suggestions for getting your timing information more reliably
  • 1. film the test and use the timer on the video trimming tool in iPhoto, or whatever equivalent you use
  • 2. more complex -
  • shift-click on the upper portion of the 'r' in the Scratch top-left of the screen
  • choose 'turn fill screen off'
  • on the gray background that is revealed, click a mouse button
  • the 'World' menu should appear
  • click on 'new morph'
  • move moue to 'Demo' and a submenu should appear with 'FrameRateMorph' as the third entry
  • click on that and drag the resulting morph to the gray background, click to drop it
  • that is now timing each frame and showing an approx frames per second
  • run your test
I've noticed (the rather obvious) intermittent hesitation but not found a cause for it yet. I've also noticed that having the 'Sounds' tab active can sometimes cause a dramatic slowdown, which is definitely puzzling. Working ….
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

timrowledge
Posts: 1237
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 2:01 am

Just as a teaser, I filmed Asteroids shooting into the corner as per winkleink's youtube show; I timed it as 1.2secs on my non-turbo'd Pi. Of course, I have some slightly scary experimental VM code in use for this run, with some interesting problems yet to iron out. :D
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 6:30 am

Re variable watcher - as Kazuhiro says http://www.raspberrypi.org/phpBB3/viewt ... be#p297955

its not a mod for the main distribution :)

Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

User avatar
winkleink
Forum Moderator
Forum Moderator
Posts: 278
Joined: Mon Nov 07, 2011 9:12 am
Location: UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 7:57 am

Here is the code I used for Testing.
https://docs.google.com/file/d/0B23BvTk ... sp=sharing

Nothing too complex just enough to test costumes, move, loops, if statements.
Twitter:Winkleink
Google+: Winkleink

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 10:51 am

I don't get a working download :( Could you re-upload it please? :)

Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

User avatar
winkleink
Forum Moderator
Forum Moderator
Posts: 278
Joined: Mon Nov 07, 2011 9:12 am
Location: UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 11:36 am

Sorry.
When I get to my Pi tomorrow night I will upload again.

Albert.
Twitter:Winkleink
Google+: Winkleink

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 12:03 pm

no problem - its always the simple things that fail :)
Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

timrowledge
Posts: 1237
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Wed Jun 05, 2013 10:53 pm

So, simples & winkleink, how would you like to be part of a testing team for new releases? We could do with help from people not directly involved in making the changes; nobody ever spots the bugs they cause any more than you can find your own spilling mistakes.
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Thu Jun 06, 2013 8:02 am

Love to :)
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

User avatar
winkleink
Forum Moderator
Forum Moderator
Posts: 278
Joined: Mon Nov 07, 2011 9:12 am
Location: UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Fri Jun 07, 2013 1:23 pm

I'd be honoured to be involved.
Twitter:Winkleink
Google+: Winkleink

timrowledge
Posts: 1237
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Sat Jun 08, 2013 12:40 am

OK, if you'd like to email me at tim at rowledge dot org we can keep in touch. I might have something I can share with you late next week.
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

simplesi
Posts: 2327
Joined: Fri Feb 24, 2012 6:19 pm
Location: Euxton, Lancashire, UK
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Sat Jun 08, 2013 5:42 am

Do you you think you can get a "bug fix" version out for everyone into in the near future to sort out the variable display issue as its quite a noticeable one compared to the sounds issues.

Kids especially, they like seeing their scores in games going up when they do something :)

regards
Simon
Seeking help with Scratch and I/O stuff for Primary age children
http://cymplecy.wordpress.com/ @cymplecy on twitter

timrowledge
Posts: 1237
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: Test Results for performance of Scratch on a Model A

Tue Jun 18, 2013 1:17 am

simplesi has been helping me test some improvements and just FYI my latest system takes a tiny smidge over 1 second to do the simple shoot-the-corner test. That compares to ~3.5s a while ago.
Making Smalltalk on ARM since 1986; making your Scratch better since 2012


Return to “Scratch”