Getting selenium python to work on Raspberry Pi Model B

[MowingBar]

I installed selenium on my headless raspberry pi model b using:

Code: Select all

pip install selenium

Since then I have tried running selenium using various different browsers including Firefox, Iceweasel, Chrome, Chromium and using phantomjs. I want to run the following code using Iceweasel:

Code: Select all

from pyvirtualdisplay import Display
import pyvirtualdisplay
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()
fp = webdriver.FirefoxProfile("/usb1/uxfq8qiv.default")
#binary = FirefoxBinary("/usr/bin/iceweasel")
binary = FirefoxBinary("/usr/lib/iceweasel/iceweasel")
profile = webdriver.FirefoxProfile()
profile.native_events_enabled = False
driver = webdriver.Firefox(profile)

driver.get("http://www.python.org")

In all cases I run into the same error:

Code: Select all

("Can't load the profile. Profile "
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.

I have searched for solutions to this however I cannot find anyone that actually runs selenium on a raspberry pi model b.

Is it possible to run selenium python scripts on a raspberry pi model b?

[ghans]

I can confirm that Selenium worked with X11 forwarding (XMing) and locally on the Pi (Raspbian default X) . A physical display was never connected to the Pi.

Code: Select all

import os

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

os.environ["DISPLAY"] = ":0.0"

fp = webdriver.FirefoxProfile()

#fp.set_preference("browser.download.folderList",2)
#fp.set_preference("browser.download.manager.showWhenStarting",False)
#fp.set_preference("browser.download.dir", os.getcwd())

browser = webdriver.Firefox(firefox_profile=fp)
browser.get("http://www.google.com")

Setting DISPLAY and then using the default X server might make the difference ?

ghans

[MowingBar]

OK. After lots and lots of attempts I finally got it running. Not sure what finally did it but the last command I ran was:

Code: Select all

sudo apt-get install openssl gem ruby xvfb

Hopefully this helps anyone else trying to do this.

**********UPDATE***********
Please use the answer provided by glycerine to solve this issue.

[ipretender]

I have got same problem. Can you post 100% working sample?
I installed xvfb, selenium, iceweasel.

[MowingBar]

I have got same problem. Can you post 100% working sample?
I installed xvfb, selenium, iceweasel.

As I said in a previous post, make sure you have installed openssl, gem, ruby and xvfb. My working example is:

Code: Select all

from selenium.webdriver.firefox.webdriver import WebDriver
from selenium import webdriver
from pyvirtualdisplay import Display
import pyvirtualdisplay
import json

display = Display(visible=0, size=(800, 600))
display.start()

fp = webdriver.FirefoxProfile()
success = True
wd = WebDriver(fp)
wd.implicitly_wait(600)
wd.get("http://www.google.com")

display.stop()

Post again if you have any problems.

[tidaboy9]

Can anyone help, I've installed all the apt-get/pip/gem packages along with using your script on my B+ Rasbian Jessie but when I try to execute the script w/ sudo python test.py I get

("Can't load the profile. Profile "
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.

I think it has something to do with the newest version of iceweasal,but I'm not sure. Is their any log I can post that would help diagnost this problem?

[ghans]

Any reason for using sudo ?

ghans

[glycerine]

Hi,

I was having the exact same issue, and found the answer here; https://github.com/SeleniumHQ/selenium/issues/1300

Essentially, the firefox instance wasn't starting before the selenium script timed out (in my case it was probably exacerbated by limiting all processes to 30% cpu usage max, but i'd expect it to start more slowly on a Pi regardless).

To fix, edit /usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py and change the value '30' on line 101 to something higher. I went with 120, but it could probably be much lower if you're sensitive to the amount of time it'll take before failing if there is genuinely an issue.

Hope that helps!

[MowingBar]

Hi,

I was having the exact same issue, and found the answer here; https://github.com/SeleniumHQ/selenium/issues/1300

Essentially, the firefox instance wasn't starting before the selenium script timed out (in my case it was probably exacerbated by limiting all processes to 30% cpu usage max, but i'd expect it to start more slowly on a Pi regardless).

To fix, edit /usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py and change the value '30' on line 101 to something higher. I went with 120, but it could probably be much lower if you're sensitive to the amount of time it'll take before failing if there is genuinely an issue.

Hope that helps!

This worked for me when I set the value to "240" as I was playing a video at the same time. This is a much better solution than my suggestion to install random packages.

[honda4life]

This does not work anymore,
It works when you change the code, not the timeout value because it is a default value when no argument is given.

The real timeout comes from :
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py

Code: Select all

class WebDriver(RemoteWebDriver):

    # There is no native event support on Mac
    NATIVE_EVENTS_ALLOWED = sys.platform != "darwin"

    def __init__(self, firefox_profile=None, firefox_binary=None, timeout=30,

All you need to install is from minimal clean image:

Code: Select all

sudo apt-get install python-pip iceweasel xvfb
sudo pip install pyvirtualdisplay selenium

(not sure if python-pip is needed)

To run:

Code: Select all

from selenium import webdriver
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 600))
display.start()

driver = webdriver.Firefox()

[Pi_n_Peas]

I had a fully working Python script using Selenium Webdriver,
pyvirtualdisplay, Xvfb wrapper and Firefox (Iceweasel/geckodriver) that scraped perfectly for me.
Only problem was the hardware this was working on was a Pi3. My Pi3 is now busy pretending to be an Amazon Alexa so I am trying to migrate the code to run on an old Pi Rev2 Model B. My Pi3 code created all sorts of errors (as you can imagine ) when I tried to run my 'working' Python script on the Pi Rev2 Model B so I decided to get something simpler working first and go back to basics and start with the simple code on this thread posted by @honda4life.

Code: Select all

from selenium import webdriver
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 600))
display.start()

driver = webdriver.Firefox()

Error code I am getting is:

Code: Select all

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 140, in __init__
    self.service.start()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/common/service.py", line 96, in start
    self.assert_process_still_running()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/common/service.py", line 109, in assert_process_still_running
    % (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service geckodriver unexpectedly exited. Status code was: -11

My H/W is a 512kB RPi rev2 Model B

OS on 8GB Card is:
PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
NAME="Raspbian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=raspbian
ID_LIKE=debian

Python 2.7.9
Name: selenium
Version: 3.0.2
Location: /usr/local/lib/python2.7/dist-packages

and Geckodriver.exe is in /home/pi/Downloads and this path has been added to $PATH

[Pi_n_Peas]

Issue appears to be that geckodriver is only compatible with ARM7 architecture.
https://github.com/mozilla/geckodriver/issues/560

Getting selenium to work on RPi Model B will require a new (retro) build of geckodriver that is compiled for ARM6 architecture.

[salatielaraujo]

I changed the memory card to a class 10 and the problem stopped happening.