Per this previous question I updated Selenium to version 2.0.1
But now I have another error, even when the profile files exist under /tmp/webdriver-py-profilecopy:
File "/home/sultan/Repository/Django/monitor/app/request.py", line 236, in perform
browser = Firefox(profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 46, in __init__
self.binary, timeout),
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/extension_connection.py", line 46, in __init__
self.binary.launch_browser(self.profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 44, in launch_browser
self._wait_until_connectable()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 87, in _wait_until_connectable
raise WebDriverException("Can't load the profile. Profile Dir : %s" % self.profile.path)
selenium.common.exceptions.WebDriverException: Can't load the profile. Profile Dir : /tmp/webdriver-py-profilecopy
What is wrong? How can I resolve this issue?
Update:
Selenium team fixed in latest version. For almost all environments the fix is:
pip install -U selenium
Unclear at which version it was fixed (apparently r13122), but certainly by 2.26.0 (current at time of update) it is fixed.
This error means that _wait_until_connectable is timing out, because for some reason, the code cannot connect to the webdriver extension that has been loaded into the firefox.
I have just reported an error to selenium where I am getting this error because I'm trying to use a proxy and only 2 of the 4 configured changes in the profile have been accepted by firefox, so the proxy isn't configured to talk to the extension. Not sure why this is happening...
https://github.com/seleniumhq/selenium-google-code-issue-archive/issues/2061
I had the same issue after upgrading Ubuntu to 12.04.
The issue was on the package side and has been fixed in the latest version of the library. Just update the selenium library. For almost all Python environments this is:
pip install -U selenium
I faced the same problem with FF 32.0 and Selenium selenium-2.42.1-py2.7.egg. Tried to update selenium, but it is already the latest version.
The solution was to downgrade Firefox to version 30. Here is the process:
#Download version 30 for Linux (This is the 64 bit)
wget http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/30.0/linux-x86_64/en-US/firefox-30.0.tar.bz2
tar -xjvf firefox-30.0.tar.bz2
#Remove the old version
sudo rm -rf /opt/firefox*
sudo mv firefox /opt/firefox30.0
#Create a permanent link
sudo ln -sf /opt/firefox30.0/firefox /usr/bin/firefox
This solved all the problems, and this combination works better !
As an extension to Jeff Hoye's answer, a more 'Pythonic' way would be to subclass webdriver.firefox.firefox_profile.FirefoxProfile as follows:
class CygwinFirefoxProfile(FirefoxProfile):
#property
def path(self):
path = self.profile_dir
# Do stuff to the path as described in Jeff Hoye's answer
return path
Then, to create your driver:
driver = webdriver.Firefox(firefox_profile=CygwinFirefoxProfile())
If pip install -U selenium doesn't work (it didn't, in my case), try downgrading your Firefox to a previous version.
I had Firefox 49.0 and downgraded to 45.0 to make sure the version is supported by selenium. It worked perfectly then.
Here's a fast way to downgrade to Firefox 45.0:
sudo apt-get install firefox=45.0.2+build1-0ubuntu1
Hope this helps.
If you are running webdriver from cygwin, the problem is that the path to the profile is still in POSIX format which confuses windows programs. My solution uses cygpath to convert it into Windows format.
in this file/method:
selenium.webdriver.firefox.firefox_binary.launch_browser():
replace:
self._start_from_profile_path(self.profile.path)
with:
from subprocess import Popen, PIPE
proc = Popen(['cygpath','-d',self.profile.path], stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
path = stdout.split('\n', 1)[0]
self._start_from_profile_path(path)
#self._start_from_profile_path(self.profile.path)
Since Python is not even close to my primary programming language, if someone can recommend a more pythonic approach maybe we can push it into the distribution. It sure would be handy if it worked in cygwin right out of the box.
I had the same problem and believed it was the wrong combo of selenium / Firefox. Turned out that my .mozilla/ folder permissions were only accessible to the root user. Doing chmod 770 ~/.mozilla/ did the trick. I would suggest making sure this is not the issue before troubleshooting further.
pip install -U selenium
I had this same issue with Firefox 34.0.5 (Dec 1, 2014) and upgrading Selenium from 2.42.1 to 2.44.0 resolved my issue.
However, I've have since seen this issue again, I think with 2.44.0, and another upgrade fixed it. So I'm wondering if it might be fixed by simply uninstalling and then re-installing. If so, I'm not sure what that would indicate the underlying problem is.
I was using selenium 2.53 and firefox version 55.0. I solved this issue by installing the older version of firefox (46.0.1) since selenium 2.53 will not work for firefox version 47.0 & above.
This is not a proper solution but worked for me, if somebody can improve I would be glad to know. I just run my script as root: sudo python myscript.py. I guess I can solve by changing the profile default file or directory could work.
Related
I trying to web scrape using requests-html but it returns an error saying there is a missing file even though I pip install requests-html and it said all req fulfilled. how do I get around this.
from requests_html import HTMLSession
import time
url = 'https://soundcloud.com/jujubucks'
s = HTMLSession()
r = s.get(url)
r.html.render()
songs = r.html.xpath('//*[#id="content"]/div/div[4]/div[1]/div/div[2]/div/div[2]', first=True)
print(songs)
this produces a sxstrace error.
OSError: [WinError 14001] The application has failed to start because its side-by-side
configuration is incorrect. Please see the application event log or use the command-line
sxstrace.exe tool for more detail
apparently this is the missing file according the event log but I dont know where to get it.
Activation context generation failed for "C:\Users\houst\AppData\Local\pyppeteer\pyppeteer\local-chromium\588429\chrome-win32\chrome.exe". Dependent Assembly 71.0.3542.0,language="*",type="win32",version="71.0.3542.0" could not be found. Please use sxstrace.exe for detailed diagnosis.
I came here with the same question, but the only answer didn't apply to me. My win10x64 PC has 5 versions of python, 4 installed via anaconda and python 3.10 installed via the microsoft store. Debugging the process in vscode using the MS store version... with pip install requests-html installed for that version of python only.
VScode stack trace showed that subprocess.py failed to launch a subprocess.
Windows event viewer showed a failed attempt to launch chrome.exe in:
C:\Users\username\AppData\Local\pyppeteer\pyppeteer\local-chromium\588429\chrome-win32
Windows search showed that chrome.exe - which was downloaded and extracted automatically the first time an attempt was made to call response.html.render() - was actually located at:
C:\Users\username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\Local\pyppeteer\pyppeteer\local-chromium\588429\chrome-win32
As a work around, and although I've no idea why the issue occured, I moved the chrome-win32 directory to the location expected, and found that chrome ran the javascript on the page and returned html correctly.
requests_html depends upon pyppeteer but it seems your pypeteer has not installed chromium completely. Try installing chromium manually, just activate your environment containing pyppeteer and run pyppeteer-install.exe.
I'm trying to work with the openai gym module but I get this error:
>>> import atari_py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ssit5\AppData\Local\Programs\Python\Python38\lib\site-packages\atari_py\__init__.py", line 1, in <module>
from .ale_python_interface import *
File "C:\Users\ssit5\AppData\Local\Programs\Python\Python38\lib\site-packages\atari_py\ale_python_interface.py", line 17, in <module>
ale_lib = cdll.LoadLibrary(os.path.join(os.path.dirname(__file__),
File "C:\Users\ssit5\AppData\Local\Programs\Python\Python38\lib\ctypes\__init__.py", line 451, in LoadLibrary
return self._dlltype(name)
File "C:\Users\ssit5\AppData\Local\Programs\Python\Python38\lib\ctypes\__init__.py", line 373, in __init__
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\ssit5\AppData\Local\Programs\Python\Python38\lib\site-packages\atari_py\ale_interface\ale_c.dll' (or one of its dependencies). Try using the full path with constructor syntax.
I don't have an ale_c.dll and tried finding solutions but nothing worked.
I followed the solution here https://github.com/openai/gym/issues/1726 but when trying to import atari_py it comes up with the same error. I don't see why the __init__ would search for something that didn't come with the module either. There were other StackOverflow questions that I looked at but they also yielded no results. The only solution I can think of is to get a copy of ale_c.dll but I don't know how I would get it.
I was facing the same error. Fortunately, I was able to find one workaround. Follow this steps and you should be good to go.
Download ale_c.dll from here.
Copy it in C:\Users\Deep Raval\AppData\Local\Programs\Python\Python38\Lib\site-packages\atari_py\ale_interface (Your path can be different).
Try to download arari-py package from conda
conda install -c conda-forge atari_py
this will fix your problem even if you are using Python > 3.7
For anybody else in the future, my problem was that I got the same error but I already had the ale_c.dll file in the right folder.
I followed a solution on this link --> https://github.com/openai/gym/issues/1726
I have tried to make it work with python 3.8 and 3.9 on Windows 10. Installing Python 3.7 and using it as the Python Interpreter on PyCharm resolved the issue. I have searched for the missing file while having 3.8 + version of Python and it did not exist at all. Atari-py is supported only for Python 3.7 (https://github.com/openai/atari-py) so having a higher version of that would not work. It is a bad idea to download files from unlicensed places so I advise you to install the supported version of Python (3.5, 3.6, and 3.7). Even though gym can work on 3.9 the atari version will not.
Just an update because gym 0.21.0 came out which supports ale-py and no longer uses atari-py (so any game marked v0 - v4 will no longer work if you use gym 0.21.0 and ale-py (which is supposed to come with gym). Make sure when you install your libraries that they are actually compatible.
Link to new version of gym https://brosa.ca/blog/ale-release-v0.7. When running v5 I do manage to render the game but I cannot find the other supported games. If you use gym 0.21.0 you will not need atari-py (atari_py).
From the example:
import gym
env = gym.make('ALE/Breakout-v5',
obs_type='rgb', # ram | rgb | grayscale
frameskip=5, # frame skip
mode=0, # game mode, see Machado et al. 2018
difficulty=0, # game difficulty, see Machado et al. 2018
repeat_action_probability=0.25, # Sticky action probability
full_action_space=True, # Use all actions
render_mode=None # None | human | rgb_array
)
it does actually run but the game is broken even in older versions of gym.
What I managed to get working after trying various python, gym and atari-py versions is:
I would suggest to the downloade the wheel file for you Windows and Python version from here: https://pypi.org/simple/atari-py/
and then execute pip install on this file like:
pip install atari_py-0.2.9-cp39-cp39-win_amd64.whl
This one helped for me (solution from github seems much much better, but it was not working for me - maybe because of venv? ) - for me it's a more safe solution than downloading a .dll from google drive.
This is a problem that you may face when using Windows.
Uninstall and reinstall atari using:
pip install -f https://github.com/Kojoley/atari-py/releases atari_py
I've boiled my code down to the following:
from selenium import webdriver
from time import sleep
b = webdriver.Firefox()
sleep(10)
b.get('http://www.google.com')
This causes a BrokenPipeError on the b.get call. Is this an error with my environment? Something I'm doing wrong, or a bug?
ENV:
Fedora 27 (64bit)
Firefox Quantum 59.0.2 (64bit)
Python 3.6.5
selenium 3.13.0 (via pip)
geckodriver v0.21.0 from github
Update:
I upgraded my workstation and along with that moved to Fedora 28 with a brand new install of Firefox 61, Python 3.6.5, Selenium 3.13.0, and geckodriver 0.21.0 and I have the same problem with the provided script.
You need to specify the path of the webdriver.exe file
first specify path ='path of your webdriver.exe'
path ='path of your webdriver.exe'
and then b = webdriver.Firefox(path)
I use this method to fix this problem and I call it every time there is a risk for timeout. I hope there is a better solution, but this works for me right now.
def refresh(b):
try:
b.refresh()
except BrokenPipeError:
pass
I have been struggling to figure out why I keep getting errors trying to use selenium. I'm using a local install of anaconda3 on my /home/user unix drive at the company I work for. I already pip installed selenium, seemingly without issue, but when I try the following:
from selenium import webdriver
driver = webdriver.Firefox()
it fails with the following message:
WebDriverException: Message: Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line
I've tried downloading the most current chromedriver and trying with that, I've tried installing another gecko-driver, I've tried all kinds of things. But nothing is working. I'm happy to provide any amount of additional information, I just want to get this off the ground at some point...
Thank you!
from selenium import webdriver
path = r'C:\yourgeckodriverpath\geckodriver.exe'
driver = webdriver.Firefox(executable_path=path)
Alright, through a combination of the responses to this question, I have figured out what (I think) went wrong. I was using a linux anaconda install on my company's servers, which (I believe) meant my python had no access to a browser driver. The solution was sadly to install anaconda locally, manually download/unzip/install selenium and geckodriver, and then make sure I pass the whole "executable_path=path" parameter to the Firefox method. This didn't work for Chrome for some reason, which I'll assume has something to do with the unchangeable security specifications on my work machine. If any part of this doesn't sound right, feel free to chime in and shed more light on the issue. Thanks!
I'm trying to use pytractor.
When I write import statement
from pytractor.webdriver import Firefox
Firefox is not referenced-not found. Neither Chrome.
However, pytractor instructions and example clearly have no problem importing
Firefox:
readme file
Anybody has solved this?
(I read the pytractor is not actively maintained, however last activity on its Github page is currently 6 days old so I guess it is maintained.)
example
UPDATE:
FF was underlined with red squiggly line in Pycharm but the code still works. I just didn't run it. It's Pycharm's error checking that was confusing.
Don't have any issues with that. Installed it from github:
$ pip install git+https://github.com/kpodl/pytractor
And imported in the Python shell:
$ ipython
In [1]: from pytractor.webdriver import Firefox
In [2]:
Make sure you don't have your script named pytractor.