Getting selenium python to work on Raspberry Pi Model B


10 posts
by MowingBar » Thu Dec 17, 2015 9:48 pm
I installed selenium on my headless raspberry pi model b using:
Code: Select all
pip install selenium
Since then I have tried running selenium using various different browsers including Firefox, Iceweasel, Chrome, Chromium and using phantomjs. I want to run the following code using Iceweasel:
Code: Select all
from pyvirtualdisplay import Display
import pyvirtualdisplay
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()
fp = webdriver.FirefoxProfile("/usb1/uxfq8qiv.default")
#binary = FirefoxBinary("/usr/bin/iceweasel")
binary = FirefoxBinary("/usr/lib/iceweasel/iceweasel")
profile = webdriver.FirefoxProfile()
profile.native_events_enabled = False
driver = webdriver.Firefox(profile)

driver.get("http://www.python.org")

In all cases I run into the same error:
Code: Select all
("Can't load the profile. Profile "
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.
I have searched for solutions to this however I cannot find anyone that actually runs selenium on a raspberry pi model b.

Is it possible to run selenium python scripts on a raspberry pi model b?
Last edited by MowingBar on Fri Jan 08, 2016 9:35 pm, edited 1 time in total.
Posts: 13
Joined: Thu Dec 17, 2015 9:33 pm
by ghans » Fri Dec 18, 2015 7:45 am
I can confirm that Selenium worked with X11 forwarding (XMing) and locally on the Pi (Raspbian default X) . A physical display was never connected to the Pi.

Code: Select all
import os

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

os.environ["DISPLAY"] = ":0.0"

fp = webdriver.FirefoxProfile()

#fp.set_preference("browser.download.folderList",2)
#fp.set_preference("browser.download.manager.showWhenStarting",False)
#fp.set_preference("browser.download.dir", os.getcwd())

browser = webdriver.Firefox(firefox_profile=fp)
browser.get("http://www.google.com")


Setting DISPLAY and then using the default X server might make the difference ?

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 7157
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by MowingBar » Mon Dec 21, 2015 12:10 pm
OK. After lots and lots of attempts I finally got it running. Not sure what finally did it but the last command I ran was:
Code: Select all
sudo apt-get install openssl gem ruby xvfb
Hopefully this helps anyone else trying to do this.

**********UPDATE***********
Please use the answer provided by glycerine to solve this issue.
Last edited by MowingBar on Fri Jan 08, 2016 9:36 pm, edited 1 time in total.
Posts: 13
Joined: Thu Dec 17, 2015 9:33 pm
by ipretender » Wed Dec 23, 2015 3:08 am
I have got same problem. Can you post 100% working sample?
I installed xvfb, selenium, iceweasel.
Posts: 1
Joined: Wed Dec 23, 2015 2:49 am
by MowingBar » Wed Dec 23, 2015 11:40 am
ipretender wrote:I have got same problem. Can you post 100% working sample?
I installed xvfb, selenium, iceweasel.


As I said in a previous post, make sure you have installed openssl, gem, ruby and xvfb. My working example is:

Code: Select all
from selenium.webdriver.firefox.webdriver import WebDriver
from selenium import webdriver
from pyvirtualdisplay import Display
import pyvirtualdisplay
import json

display = Display(visible=0, size=(800, 600))
display.start()

fp = webdriver.FirefoxProfile()
success = True
wd = WebDriver(fp)
wd.implicitly_wait(600)
wd.get("http://www.google.com")

display.stop()

Post again if you have any problems.
Last edited by MowingBar on Fri Jan 08, 2016 4:00 pm, edited 1 time in total.
Posts: 13
Joined: Thu Dec 17, 2015 9:33 pm
by tidaboy9 » Sat Jan 02, 2016 11:48 pm
Can anyone help, I've installed all the apt-get/pip/gem packages along with using your script on my B+ Rasbian Jessie but when I try to execute the script w/ sudo python test.py I get

("Can't load the profile. Profile "
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.

I think it has something to do with the newest version of iceweasal,but I'm not sure. Is their any log I can post that would help diagnost this problem?
Posts: 7
Joined: Wed Dec 30, 2015 4:15 am
by ghans » Sun Jan 03, 2016 9:15 am
Any reason for using sudo ?

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 7157
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by glycerine » Fri Jan 08, 2016 3:05 pm
Hi,

I was having the exact same issue, and found the answer here; https://github.com/SeleniumHQ/selenium/issues/1300

Essentially, the firefox instance wasn't starting before the selenium script timed out (in my case it was probably exacerbated by limiting all processes to 30% cpu usage max, but i'd expect it to start more slowly on a Pi regardless).

To fix, edit /usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py and change the value '30' on line 101 to something higher. I went with 120, but it could probably be much lower if you're sensitive to the amount of time it'll take before failing if there is genuinely an issue.

Hope that helps!
Posts: 1
Joined: Fri Jan 08, 2016 2:59 pm
by MowingBar » Fri Jan 08, 2016 9:33 pm
glycerine wrote:Hi,

I was having the exact same issue, and found the answer here; https://github.com/SeleniumHQ/selenium/issues/1300

Essentially, the firefox instance wasn't starting before the selenium script timed out (in my case it was probably exacerbated by limiting all processes to 30% cpu usage max, but i'd expect it to start more slowly on a Pi regardless).

To fix, edit /usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py and change the value '30' on line 101 to something higher. I went with 120, but it could probably be much lower if you're sensitive to the amount of time it'll take before failing if there is genuinely an issue.

Hope that helps!


This worked for me when I set the value to "240" as I was playing a video at the same time. This is a much better solution than my suggestion to install random packages.
Posts: 13
Joined: Thu Dec 17, 2015 9:33 pm
by honda4life » Sat May 14, 2016 7:25 am
This does not work anymore,
It works when you change the code, not the timeout value because it is a default value when no argument is given.

The real timeout comes from :
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py

Code: Select all
class WebDriver(RemoteWebDriver):

    # There is no native event support on Mac
    NATIVE_EVENTS_ALLOWED = sys.platform != "darwin"

    def __init__(self, firefox_profile=None, firefox_binary=None, timeout=30,



All you need to install is from minimal clean image:
Code: Select all
sudo apt-get install python-pip iceweasel xvfb
sudo pip install pyvirtualdisplay selenium

(not sure if python-pip is needed)

To run:
Code: Select all
from selenium import webdriver
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 600))
display.start()

driver = webdriver.Firefox()
Posts: 74
Joined: Thu Mar 15, 2012 7:27 pm