gkreidl
Posts: 5156
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Python speed, benchmarks, 2v3

Tue May 16, 2017 3:32 pm

While converting some of my Python 2 scripts to Python 3 I discovered notable performance differences. So I became interested in benchmarking Python 2 and 3. I also use Nuitka ( http://nuitka.net/ ) to compile some of my Python stuff to binaries for different reasons (speed may be one of them).

Here are the results of the Pystone benchmarks for Python 2.7 and 3.4 and for compiled programs of both verions. Except for the time measurement the Pystone benchmark only uses functions built into the main Python library.

Python 2.7: 24344.88 pystones/second
Python 3.4: 17459.89 pystones/second
Nuitka 2.7: 47243.92 pystones/second
Nuitka 3.4: 28658.92 pystones/second

Speed Comparisons:

Python 2.7 / Python 3.4: 139.43 %
Nuitka 2.7 / Python 2.7: 194.06 %
Nuitka 3.4 / Python 3.4: 164.14 %
Nuitka 2.7 / Nuitka 3.4: 164.85 %
Nuitka 2.7 / Python 3.4: 270.59 %

Obviously Python 2.7 runs almost 40% faster than Python 3.4 code. The speed gain when compiling it with Nuitka is also greater for 2.7: 94% instead of 64%. And a compiled Python 2.7 program runs 2.7 times faster than an interpreted Python 3.4 script.

Does than mean, that you should use Python 2.7 instead of 3.4? For beginners and new (long time) projects I would not recommend it. For many projects, speed is not an issue at all. Inside a GUI for example, most of the time is spent waiting for user input or user events. And often it is better to look for Python wrappers of compiled C libraries for time critical stuff.

But if your project contains time critical Python routines, you might consider writing code that will run on both Python 2.7 and Python 3.4. That's not an easy task, but there are a number of tools available which help a lot. The best I found is the "future" package from http://python-future.org (can be installed using pip and pip3). This website also provides a great "Cheat Sheet" which shows how to write code that will run on both Python versions (also available as PDF document http://python-future.org/compatible_idioms.pdf ). The "future" package also contains two great utilities: "futurize" ( to help converting 2.7 to 3.4) and "pasteurize" (to help creating 2.7 compatible code from 3.4 sources).

There's one drawback, though: if you want to distribute your code, you must make sure, that the "future" package is installed. Unfortunately it is not available as a Debian (Raspbian) package and so it's not possible to add it as a dependency to a Debian package.

Other tools: Cython and PyPy.

I've also compiled pystone with Cython (2.7 only), but got only about 10% speed gain. So it's not really worth the effort. Adding Cython specific code might help, but I didn't try that.

Often PyPy is recommended to speed up Python code using a JIT compiler. The Pystone benchmark gives different results, depending on the number of loops:
1000 loops: 5770.32 pystones/second (slower than 2.7 interpreter)
5000 loops: 27645.2 pystones/second (slightly faster than 2.7 interpreter)
10000 loops: 52145.1 pystones/second (faster than 2.7 compiled with Nuitka)
100000 loops: 248133 pystones/second (really much faster!).

Currently PyPy is only available on the RPi for Python 2.7. Using it in real world applications may lead to difficulties because of its memory usage. For my DVB application I needed a script to extract EPG information in real time from a DVB stream (up to 10 MBit/sec). I found a script from another project, which worked quite well, but needed some speed improvement. I ended up in running it as a separate application (compiled with Nuitka as 2.7 code), to make sure it will run on a separate core. I also tested using it with PyPy. It seemed to work fine, but my system almost froze after a short while (starting to swap). Watching the memory I discovered that it used up to 480 MB, while the compiled Nuitka script never used more than about 70 MB. Fast, yes, but not usable.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 17563
Joined: Sat Jul 30, 2011 7:41 pm

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 3:41 pm

That's pretty interesting. I did some quick tests the other day (after your suggestion of compiling Pythin with nuitka), and found much the same. Python 3.4 is pretty slow compared with older version. My nuitka compiles didn't show anywhere near that speed improvement though, but the program was very simple.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

bensimmo
Posts: 1569
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 4:03 pm

How does 3.5 compare (in stretch)?
3.4 may well be lagging in optimizations especially when 3.6.1 is the current latest version.

hippy
Posts: 2050
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 5:48 pm

bensimmo wrote:How does 3.5 compare (in stretch)?
3.4 may well be lagging in optimizations especially when 3.6.1 is the current latest version.
I did try some simplistic benchmarking and was getting the same '2.7 is quite a lot faster than 3.4' results consistent with other reports and benchmarks. I also tried 3.6.0 with the same simplistic benchmarks and that seems to somewhat improve on 3.4 but still not as fast as 2.7. Have not tried 3.6.1 -

viewtopic.php?f=32&t=172622&p=1104397#p1104397

hippy
Posts: 2050
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 6:10 pm

gkreidl wrote:Does than mean, that you should use Python 2.7 instead of 3.4? For beginners and new (long time) projects I would not recommend it. For many projects, speed is not an issue at all. Inside a GUI for example, most of the time is spent waiting for user input or user events.
That is a difficult question to answer. I would however also recommend new users start with Python 3, though through gritted teeth.

For myself I am sticking with 2.7. I tend to throw code together on a Windows PC, get that working, then copy to my Pi and run there. Most of my Python code is input-process-output-exit, isn't execution speed critical as such, but if it takes more than a second or two to run and return to the command prompt it can feel slow, and 3.4 can often make things feel too slow.

I will optimise my code when I need or want to improve things, but quite a lot of optimisation may need to be done for 3.4 just to get to 2.7 speed. Might as well use 2.7 to start with.

For a couple of GUI programs I wrote there is nothing much in it. As you note; code like that is mostly waiting for input.

Added : One trick for making a slow program feel quicker; output a progress bar. It is amazing how a progress bar seems to make things run quicker even though it changes nothing

bensimmo
Posts: 1569
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 8:07 pm

Does the slowness stay if using your PC (say the ixel x86 version.)

This is all just interest (learning) as I don't know enough about the language and background in them.
There was a thread and somebody mentioned it may/was due to the way Python3 handles some of the numbers but I might be wrong.

Also everything on the Python website seems to be x86 targeted (windows and osx) does any optimization happen for Arm machines?

hippy
Posts: 2050
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Python speed, benchmarks, 2v3

Tue May 16, 2017 8:18 pm

bensimmo wrote:Does the slowness stay if using your PC (say the ixel x86 version.)
Yes; from testing it myself and everything I have read agrees that 3.4 is considerably slower than 2.7 on any platform. Some put that down to the way integers are handled / enumerated / whatever, but there seems to be more to it than just that. Text file output seemed considerably slower.

I benchmarked microPython on the Pi, which is based on Python 3-dot-something and apparently has optimisations for handling integers. I can't find those benchmark timings now but I recall that got closer to 2.7 for my simplistic integer benchmark, about the same as 3.6 for the simplistic float benchmark.

gkreidl
Posts: 5156
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Python speed, benchmarks, 2v3

Wed May 17, 2017 4:11 am

I also did the same tests on my x86 Ubuntu Desktop. The results are similar, but the speed improvement by compiling 3.4 with nuitka is smaller:

Python 2.7: 121038.5 pystones/second
Python 3.4: 86386.7 pystones/second
Nuitka 2.7: 233031.5 pystones/second
Nuitka 3.4: 116628.96 pystones/second

Speed Comparisons:

Python 2.7 / Python 3.4: 140.11 %
Nuitka 2.7 / Python 2.7: 192.53 %
Nuitka 3.4 / Python 3.4: 135.01 %
Nuitka 2.7 / Nuitka 3.4: 199.81 %
Nuitka 2.7 / Python 3.4: 269.75 %
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

Return to “Python”

Who is online

Users browsing this forum: No registered users and 12 guests