I am using selenium-wire to click a button using Chrome browser. Up until today, my code worked fine. However, now I get the following error and I am not sure why:
Traceback (most recent call last):
File "Scraping_fx.py", line 1, in <module>
from seleniumwire import webdriver
File "C:\Users\me\Anaconda3\envs\project\lib\site-
packages\seleniumwire\webdriver\__init__.py", line 3, in <module>
from .browser import Chrome, Edge, Firefox, Safari # noqa
File "C:\Users\me\Anaconda3\envs\project\lib\site-
packages\seleniumwire\webdriver\browser.py", line 14
def __init__(self, *args, seleniumwire_options=None, **kwargs):
^
SyntaxError: invalid syntax
I am using python 2.7.16; selenium-wire 1.0.4.
from seleniumwire import webdriver
from selenium.common.exceptions import ElementClickInterceptedException,NoSuchElementException
import logging
def scrape_website(url):
# Configure browser driver
options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--disable-gpu')
options.add_argument('--headless')
#Instantiate driver and navigate to URL
driver = webdriver.Chrome(chrome_options=options)
driver.implicitly_wait(30)
driver.get(url)
Am I missing something here?
Can't see anything wrong with your code :)
But selenium-wire 1.0.4 requires Python 3.4+.
You can find all the info on pypi.org - here
Related
I'm refactoring my app from procedural code to OOP. I am trying to do this Driver class.
UPDATE: this works in Windows but not in Mac.
# IMPORTS
from sys import platform
import os
from os import system
from selenium import webdriver
from selenium.webdriver import Firefox, FirefoxOptions
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
# import Action chains
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import Select
from selenium.webdriver.firefox.service import Service
class Driver():
def __init__(self):
#set executable path to driver
self.dirname = os.path.dirname(__file__)
if platform == "win32":
self.executable_path = os.path.join(self.dirname, 'geckodriver.exe') #must save the gecko file to same directory where python is. path to geckodriver (firefox drive/motor) on your machine
print("Gecko (Firefox) filepath is: ", self.executable_path)
if platform == "darwin":
self.executable_path = os.path.join(self.dirname, 'geckodriver') #must save the gecko file to same directory where python is. path to geckodriver (firefox drive/motor) on your machine
print("Gecko (Firefox) filepath is: ", self.executable_path)
self.service = Service(self.executable_path)
self.opts = FirefoxOptions()
#self.opts.add_argument(f"--width={int(screen_width/4)}")
#self.opts.add_argument(f"--height={int(screen_height/2)}")
self.driver = Firefox(service=self.service, options=self.opts)
self.driver.set_window_position(-10, 0)
self.driver.get("https://google.com/")
Driver()
This will give me the following error:
Traceback (most recent call last):
File "/driverClass.py", line 72, in <module>
Driver()
File "/driverClass.py", line 66, in __init__
self.driver = Firefox(service=self.service, options=self.opts)
TypeError: __init__() got an unexpected keyword argument 'service'
Why is this? I'm refactoring my code to OOP. The code worked before when using procedural code.
This is from the working code:
# driver configs
service = Service(executable_path) #pass in path to geckodriver
opts = FirefoxOptions()
#opts.add_argument(f"--width={int(screen_width/4)}")
#opts.add_argument(f"--height={int(screen_height/2)}")
driver = Firefox(service=service, options=opts)
driver.set_window_position(-10, 0)
#driver.set_window_size(int(screen_width/4), int(screen_height))
driver.get("https://google.com/")
This error message...
TypeError: __init__() got an unexpected keyword argument 'service'
...implies that service is an unexpected keyword argument.
The possible reason, you are still using Selenium v3.x and the keyword argument service
wasn't supported.
Solution
Since Selenium 4.0 Beta 1:
Deprecate all but Options and Service arguments in driver instantiation. (#9125,#9128)
So you need to upgrade to Selenium 4.x
References
You can find a couple of relevant detailed discussion in:
TypeError: init() got an unexpected keyword argument 'service' error using Python Selenium ChromeDriver with company pac file
self.driver = Firefox(service=self.service, options=self.opts)
The Firefox webdriver doesn't take a service keyword argument:
https://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.firefox.webdriver
It would be helpful to see the older working version.
I encountered the same error and fixed this problem by upgrading to Selenium 4.0
I am using Anaconda python.
To upgrade to Selenium 4.0 on Anaconda, run the following command;
$ conda install https://conda.anaconda.org/conda-forge/noarch::selenium
The following command also works fine.
$ conda install selenium --channel conda-forge/noarch
At this point of writing, you cannot upgrade Selenium using the normal conda command conda update selenium. Running this command will install selenium v3
You have to specify the installation on the noarch version which supports selenium v4.1
The link below teaches you how to do that.
Specify platform during package install
As you can see when you call
Firefox(service=self.service, options=self.opts)
it raises because Firefox class does not accept service kwarg
If you are trying to set up a Firefox web driver you can see similar question
I'm using Python with Selenium but I need to use it with extension (and probably with cookies). Extension is uploaded from ZIP file and I need to change something in this extension settings after instalation so it will be hard to reupload extension every start of project. Is there aby option to use it like that? I was trying to use profile from normal chrome but it doesn't work for me.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
service = Service('D:\\chromedriver.exe') # your driver path
chrome_options = Options()
chrome_options.add_argument \
(r"--user-data-dir=C:\\Users\\czarn\\AppData\\Local\\Google\\Chrome\\User Data") # your chrome user data directory
chrome_options.add_argument(r'--profile-directory=Member') # the profile with the extensions loaded
window = webdriver.Chrome(service=service, options=chrome_options)
I have this error:
Traceback (most recent call last):
File "C:\Users\czarn\PycharmProjects\trening\main.py", line 16, in <module>
window = webdriver.Chrome(service=service, options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'service'
As you mentioned, you can use an already existing Chrome Profile.
Example code where the paths are referring to my machine but it should be easy for you to adapt to your use case.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
service = Service('C:\\Users\\nicoc\\PycharmProjects\\WriteAI\\chromedriver.exe') # your driver path
chrome_options = Options()
chrome_options.add_argument \
(r"--user-data-dir=C:\\Users\\nicoc\\AppData\\Local\\Google\\Chrome\\User Data") # your chrome user data directory
chrome_options.add_argument(r'--profile-directory=Default') # the profile with the extensions loaded
driver = webdriver.Chrome(service=service, options=chrome_options)
driver.get("https://google.com/")
The code works for Selenium v4 (it's still in beta but it works fine)
im working on a project and have this as code:
import selenium
from selenium import webdriver
driver = webdriver.Chrome()
print(driver.current_url)
but i get an error and i dont know how to fix it, well mutle tracebacks to error i think, here is the error:
Traceback (most recent call last):
File "C:\Users\ytty\PycharmProjects\test hack\main.py", line 2, in <module>
from selenium import webdriver
File "C:\Users\ytty\PycharmProjects\test hack\venv\lib\site-packages\selenium\webdriver\__init__.py", line 18, in <module>
from .firefox.webdriver import WebDriver as Firefox # noqa
File "C:\Users\ytty\PycharmProjects\test hack\venv\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 36, in <module>
from .service import Service
File "C:\Users\ytty\PycharmProjects\test hack\venv\lib\site-packages\selenium\webdriver\firefox\service.py", line 21, in <module>
class Service(service.Service):
AttributeError: module 'selenium.webdriver.common.service' has no attribute 'Service'
Process finished with exit code 1
can anyone help?
I do it this way:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options, executable_path=ChromeDriverManager().install())
This way it will automatically download the optimal ChromeDriver version on your cache and use it every time you need. Also the experimental part helps to avoid some error that might appear due to driver bugs
My chromium version is 87.0.4280.88
My brave browser version is 87.0.4280.101
I tried a lot of codes but they didn't work.
Code:
from selenium import webdriver
option = webdriver.ChromeOptions()
option.binary_location = r'C:\Program Files (x86)\BraveSoftware\Brave-
Browser\Application\brave.exe'
driver = webdriver.Chrome(
executable_path=r'C:\WebDrivers\chromedriver.exe', options=option)
driver.get("https://www.google.com")
Error traceback:
Traceback (most recent call last):
File "c:\Users\MOHSEN\Desktop\test\Untitled-1.py", line 5, in <module>
driver = webdriver.Chrome(
TypeError: __init__() got an unexpected keyword argument 'options'
To initiate a brave Browsing Session using Selenium driven WebDriver you can use the following solution:
Code Block:
from selenium import webdriver
option = webdriver.ChromeOptions()
option.binary_location = r'C:\Program Files (x86)\BraveSoftware\Brave-Browser\Application\brave.exe'
driver = webdriver.Chrome(executable_path=r'C:\WebDrivers\chromedriver.exe', options=option)
driver.get("https://www.google.com")
Browser Snapshot:
I'm trying to find a way to implement:
- this code using python
- schedule a timed trigger
- select a tab on the google chrome
I've copied and pasted the solution into my python just to test it but seem to be getting a URL error below, can someone help me understand why a variable is facing a syntax error?
Here is my code:
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from Common_Methods.GenericMethods import *
import pyautogui #<== need this to click on extension
options = ChromeOptions()
#from stack overflow(https://stackoverflow.com/questions/53172127/click-on-elements-in-chrome-extension-with-selenium?rq=1):
#options.add_argument("--load-extension=" + r"C:\Users\supputuri\AppData\Local\Google\Chrome\User Data\Default\Extensions\fdpohaocaechififmbbbbbknoalclacl\5.1_0") #<== loading unpacked extension
options.add_argument("--load-extension=" + r"/Users/erikwayne/Library/Application Support/Google/Chrome/Profile 2/Extensions/fdpohaocaechififmbbbbbknoalclacl/6.5_0") #<== loading unpacked extension
driver = webdriver.Chrome(
executable_path=os.path.join(chrome_options=options)
url = "https://www.google.com/"
driver.get(url)
# get the extension box
extn = pyautogui.locateOnScreen(os.path.join(GenericMethods.get_full_path_to_folder('autogui_ref_snaps') + "/capture_full_screenshot.png"))
# click on extension
pyautogui.click(x=extn[0],y=extn[1],clicks=1,interval=0.0,button="left")
Here is my error message:
MBP:Testing chrome extension erikwayne$ python3 chromeClick_v1.py
File "chromeClick_v1.py", line 15
url = "https://www.google.com/"
^
SyntaxError: invalid syntax
Expanded debugging:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1703, in main
pdb._runscript(mainpyfile)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1572, in _runscript
self.run(statement)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/bdb.py", line 587, in run
exec(cmd, globals, locals)
File "<string>", line 1, in <module>
File "/Users/erikwayne/Downloads/Testing chrome extension/chromeClick_v1.py", line 11
url = "https://www.google.com/"
^
SyntaxError: invalid syntax
*Edit: Tried the modified code below from #RafalS, but had more error.
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from Common_Methods.GenericMethods import *
import pyautogui #<== need this to click on extension
import os
options = ChromeOptions()
#from stack overflow(https://stackoverflow.com/questions/53172127/click-on-elements-in-chrome-extension-with-selenium?rq=1):
#options.add_argument("--load-extension=" + r"C:\Users\supputuri\AppData\Local\Google\Chrome\User Data\Default\Extensions\fdpohaocaechififmbbbbbknoalclacl\5.1_0") #<== loading unpacked extension
options.add_argument("--load-extension=" + r"/Users/erikwayne/Library/Application Support/Google/Chrome/Profile 2/Extensions/fdpohaocaechififmbbbbbknoalclacl/6.5_0") #<== loading unpacked extension
driver = webdriver.Chrome(executable_path=os.path.join(chrome_options=options))
url = "https://www.google.com/"
driver.get(url)
# get the extension box
extn = pyautogui.locateOnScreen(os.path.join(get_full_path_to_folder('downloads') + "/capture_full_screenshot.png"))
# click on extension
pyautogui.click(x=extn[0],y=extn[1],clicks=1,interval=0.0,button="left")
driver = webdriver.Chrome(
close the parentheses
driver = webdriver.Chrome()
You'll also need to import os and replace
GenericMethods.get_full_path_to_folder
with
get_full_path_to_folder
since you did a star import:
from Common_Methods.GenericMethods import *
I did it and sorted it out by installing the chrome webdriver to a more accessible folder path, ignoring ospaths or GenericMethods altogether.
As for the driver = webdriver.Chrome issue, the original code was missing a comma between arguments, so I just filled in in.
driver = webdriver.Chrome(executable_path=os.path.join(r'/Users/USERNAME/Downloads/Testing chrome extension/chromedriver/chromedriver'), options=options)
It's hacked together but it works.