We Keep Coding

Why am I getting a No such element exception using Selenium in python flask?

I have written a program using selenium in python flask which does webscraping of a product. My intention is to first enter the product name(this is done programmatically)->after product is displayed-> it should display the price of product which I would be displaying in the terminal. My issue however is that it doesn't scrape the website and it throws a Selenium NoSuchElementException. Here is my code.
def scrape_johnLewis(product_name):
website_address = 'https://www.johnlewis.com/'
options = webdriver.ChromeOptions()
options.add_argument('start-maximized')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)
browser.get(website_address)
time.sleep(10)
browser.implicitly_wait(20)
browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()
browser.find_element_by_id('mobileSearch').send_keys(product_name)
browser.find_element_by_css_selector('div.input-with-submit.module-inputWrapper--63f9e > button.button.module-c-button--fe2f1').click()
time.sleep(5)
# browser.find_elements_by_class_name('button.module-c-button--fe2f1')[0].submit()
product_price_raw_list = browser.find_elements_by_xpath('//div[#class="info-section_c-product-card__section__2D2D- price_c-product-card__price__3NI9k"]/span')
product_price_list = [elem.text for elem in product_price_raw_list]
print(product_price_list)
if __name__ == "__main__":
scrape_johnLewis('Canon EOS 90D Digital SLR Body')
The error that I am getting is over here browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()
and here is the stacktrace:
Traceback (most recent call last):
File "scrapejohnLewis.py", line 32, in <module>
scrape_johnLewis('Canon EOS 90D Digital SLR Body')
File "scrapejohnLewis.py", line 20, in scrape_johnLewis
browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()
File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 598, in find_element_by_css_selector
return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
'value': value})['value']
File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH"}
(Session info: chrome=74.0.3729.108)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Linux 5.0.0-1034-oem-osp1 x86_64)
I have tried to replace it with find_element_by_tag_name however that didn't do the work either.
By inspecting the website I have come to find out the exact element but to my surprise the error says that there is no such element. What exactly could be the case? Please help.

The cookie prompt was not managed properly, it didn't go away and hence it blocks the code beneath thus stopping Selenium in finding the element. I've also made some tweaks to the code.
from selenium.webdriver.common.keys import Keys
def scrape_johnLewis(product_name):
website_address = 'https://www.johnlewis.com/'
options = webdriver.ChromeOptions()
options.add_argument('start-maximized')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
browser = webdriver.Chrome(ChromeDriverManager(log_level='0').install(), options=options)
browser.get(website_address)
time.sleep(3)
# browser.implicitly_wait(20)
browser.find_element_by_xpath('//*[#id="pecr-cookie-banner-wrapper"]/div/div[1]/div/div[2]/button[1]').click()
browser.find_element_by_id('desktopSearch').send_keys(product_name + Keys.ENTER)
time.sleep(5)
product_price_raw_list = browser.find_elements_by_xpath('//div[#class="info-section_c-product-card__section__2D2D- price_c-product-card__price__3NI9k"]/span')
product_price_list = [elem.text for elem in product_price_raw_list]
print(product_price_list)
# ouptut
['£1,249.00', '£1,629.99', '£1,349.99']
Also try using css_selector as the last resort if you cannot find better element locators like id, xpath, tag_name.

Once, I was getting the same error. You will wonder How it was fixed?
I kept my selenium web driver by mistake, then copy the xpath from there and my code worked.
Then, I found that the xpath of the particular element in webdriver windows differs from the xpath of the same element in actual browser window.
There was a single extra tag being added into the path in webdriver window but not in actual browser window.
May be there is the same issue on your side.

Selenium driver on google next button, NoSuchElementException

I am writing a script that deals with google. I have successfully searched for what I wanted using the selenium web driver however I would like to navigate to the next page of results. my code looks as follows:
import parameters
import csv
from parsel import Selector
from selenium import webdriver
from time import sleep
from selenium.webdriver.common.keys import Keys
empty=""
def validate_field(field):
if field == "":
field= 'No results'
return field
writer=csv.writer(open(parameters.file_name, 'w'))
writer.writerow(('Name','Job Title','Company','College','Location','URL'))
driver=webdriver.Chrome('/usr/local/bin/chromedriver')
driver.implicitly_wait(10)
driver.get('https://www.linkedin.com')
username=driver.find_element_by_id('session_key') #locating the email form using the class name
username.send_keys('')
sleep(0.5)
password=driver.find_element_by_id('session_password') #locating the password form using the class name
password.send_keys('')
sleep(0.5)
log_in_button=driver.find_element_by_class_name('sign-in-form__submit-button') #locating submit button by class name
log_in_button.click() #here we are mimicing a click
sleep(0.5)
driver.get('https:www.google.com') #navigating to google
sleep(3)
search_gog=driver.find_element_by_name('q')
search_gog.send_keys(parameters.search_gog)
sleep(0.5)
search_gog.send_keys(Keys.RETURN)
sleep(3)
list_links = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//div[#class='g']//div[#class='r']/a[contains(#href, 'https://www.linkedin.com')]")]
for link in list_links:
driver.get(link)
sel=Selector(text=driver.page_source)
name = sel.xpath('//*[starts-with(#class, "inline t-24 t-black t-normal break-words")]/text()').extract_first()
if name:
name=name.strip()
job_title= sel.xpath('//*[starts-with(#class, "mt1 t-18 t-black t-normal break-words")]/text()').extract_first()
if job_title:
job_title=job_title.strip()
education = sel.xpath('//*[starts-with(#class, "pv-profile-section-pager ember-view")]/text()').extract_first()
if education:
education=education.strip()
name=validate_field(name)
job_title=validate_field(job_title)
education=validate_field(education)
print('\n')
print('Name: ' + name)
print('Job Title: ' + job_title)
print('Education: ' + education)
print(education)
print('URL: ' + driver.current_url)
print('\n')
driver.find_element_by_link_text("Next").click()
the parameters file contains the google search query which states:
search_gog = 'site:linkedin.com/in/ AND "data analyst" AND "California"'
upon running the above I receive the following error:
Traceback (most recent call last):
File "app2.py", line 79, in <module>
driver.find_element_by_link_text("Next").click()
File "/Users/rubenolmos/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 428, in find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "/Users/rubenolmos/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Users/rubenolmos/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Users/rubenolmos/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Next"}
(Session info: chrome=85.0.4183.102)
I have also tried using the html code for the element which is:
<a href="/search?q=site:linkedin.com/in/+AND+%22data+analyst%22+AND+%22California%22&ei=lbpbX4GhDoPs9APDmITID
A&start=10&sa=N&ved=2ahUKEwjBv57e1uHrAhUDNn0KHUMMAckQ8NMDegQICxA_"
id="pnnext" style="text-align:left"><span class="SJajHc NVbCr"
style="background:url(/images/nav_logo299_hr.webp) no-repeat;background-position:-96px
0;background-size:167px;width:71px"></span><span style="display:block;margin-
left:53px">Next</span></a>
using the above html element I have tried finding the element using the element id "pnnext' by doing the following find_element_by_id("pnnext").click and it has been unsuccessful.
Any ideas?

First, if you have driver.implicitly_wait(10), then when you do driver.find_element_by_link_text("Next").click() following search_gog.send_keys(Keys.RETURN), the driver will wait for up to 10 seconds for the next page to load and a link with the text "Next" to appear before throwing an exception. So, the call to sleep(3) that you invoke between the two calls is rather superfluous. If for whatever reason you thought 10 seconds was not enough (hard to believe for google.com), then just increase the amount of time on the call to driver.implicitly_wait.
I cannot tell what your search argument to Google is, but it just may be possible that the number of returned results isn't enough to warrant a "Next" link. Modify your code as follows:
search_gog.send_keys(Keys.RETURN)
next_links = driver.find_elements_by_link_text("Next") # note it is find_elements with an s
if len(next_links):
next_links[0].click()
else:
print('There is no "Next" link')
Update
I have run the following code on my desktop 5 times in a row and each time it has successfully navigated to the second page (that is, it has found the "Next" link and clicked on it successfully).
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
options = webdriver.ChromeOptions()
#options.add_argument("headless")
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
try:
driver.implicitly_wait(10)
driver.get('https://www.google.com') #navigating to google
search_gog=driver.find_element_by_name('q')
search_gog.send_keys('site:linkedin.com/in/ AND "data analyst" AND "California"')
search_gog.send_keys(Keys.RETURN)
next_links = driver.find_elements_by_link_text("Next") # note it is find_elements with an s
if len(next_links):
print('Found "Next" link')
next_links[0].click()
else:
print('There is no "Next" link')
finally:
input('pausing (hit enter to terminate) ...')
driver.quit()

selenium.common.exceptions.NoSuchElementException means it can't find the element it may be caused by elements takings too long to load or a wrong XPath. There might be an iframe as well.
elem=WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.ID('pnnext')))
elem.click()
Import these.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Selenium: Element not clickable at point (x, y)

I have this site https://www.inc.com/inc5000/list/2017 where I want my script to insert a number in PAGE field and click GO, but I keep getting error:
File "/Users/anhvangiang/Desktop/PY/inc.py", line 34, in scrape
driver.find_element_by_xpath('//*[#id="page-input-button"]').click()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 77, in click
self._execute(Command.CLICK_ELEMENT)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
return self._parent.execute(command, params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error:
Element is not clickable at point (741, 697)
(Session info: chrome=61.0.3163.100)
(Driver info: chromedriver=2.30.477690
(c53f4ad87510ee97b5c3425a14c0e79780cdf262),platform=Mac OS X 10.12.6 x86_64)
This is my code:
def scrape(num):
driver = webdriver.Chrome('/Users/anhvangiang/Desktop/PY/chromedriver')
driver.get(main_site)
driver.find_element_by_id('page-input-field').send_keys(str(num))
driver.find_element_by_xpath('//*[#id="Welcome-59"]/div[2]/div[1]/span[2]').click()
time.sleep(5)
driver.find_element_by_xpath('//*[#id="page-input-button"]').click()
soup = BS(driver.page_source, 'lxml')
container = soup.find('section', {'id': 'data-container'})
return [source + tag.find('div', {'class': 'col i5 company'}).find('a')['href'] for tag in container.findAll('div', {'class': 'row'})]
If I put the function scrape inside a loop:
for i in range(1, 100):
print scrape(i)
For a few first i, it will go smoothly, but then it will throw the error like above.
Any suggestion how I can fix it?

This is because the button is not visible at that time, so the selenium WebDriver cannot access that.
As I run your code on my local machine , I found that the website shows a popup ad for 15-20 sec (See attached image : Popup_Ad), which is the actual cause of this error. For resolving this error you have to handle the popup ad, you can do this as.
check for the SKIP button, if button exist then first skip the add by
Clicking the skip button , then follow the normal flow of code.
Other Suggestions: You should use WebDriverWait to avoid the Element not found and element is not clickable like issues. For Example, you can write above code as
from selenium import webdriver
from bs4 import BeautifulSoup as BS
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
import time
def scrape(num,wdriver):
# define webdriver
driver = wdriver
# naviagte to url
driver.get("https://www.inc.com/inc5000/list/2017")
# define default wait time
wait = WebDriverWait(driver, 10)
while True:
if EC.presence_of_all_elements_located:
break
else:
continue
# handle Ad Popup
try:
skip_button = wait.until(EC.element_to_be_clickable((By.XPATH,"//*[#id='Welcome-59']/div[2]/div[1]/span[2]")))
skip_button.click()
print("\nSkip Button Clicked")
except TimeoutException:
pass
time.sleep(5)
# find the page number input field
page_num_elem = wait.until(EC.visibility_of_element_located((By.ID,"page-input-field")))
page_num_elem.clear()
page_num_elem.send_keys(num)
time.sleep(2)
while True:
try:
# find go button
go_button = wait.until(EC.element_to_be_clickable((By.ID, "page-input-button")))
go_button.click()
break
except TimeoutException :
print("Retrying...")
continue
# scrape data
soup = BS(driver.page_source, 'lxml')
container = soup.find('section', {'id': 'data-container'})
return [tag.find('div', {'class': 'col i5 company'}).find('a')['href'] for tag in container.findAll('div', {'class': 'row'})]
if __name__ == "__main__":
# create webdriver instance
driver = webdriver.Chrome()
for num in range(5):
print("\nPage Number : %s" % (num+1))
print(scrape(num,driver))
print(90*"-")
driver.quit()

It works for me:
Actions action = new Actions(Browser.WebDriver); action.MoveToElement(element).Click().Perform();

Create an automated script that login in into netflix

I'd like to make an automated script in python using selenium that log in here: https://www.netflix.com/Login.
I've tried with this code:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/Login")
while True:
element = driver.find_element_by_css_selector("ui-text-input[name = 'email']")
driver.execute_script("arguments[0].setAttribute('value','test1')", element)
element2 = driver.find_element_by_css_selector("ui-text-input[name = 'password']")
driver.execute_script("arguments[0].setAttribute('value','test1')", element2)
driver.refresh()
But it raises an error occurred in this line of code:
Traceback (most recent call last):
File "netflixlogin.py", line 6, in <module>
element = driver.find_element_by_css_selector("ui-text-input[name = 'email']")
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 437, in find_element_by_css_selector
return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 752, in find_element
'value': value})['value']
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: {"method":"css selector","selector":"ui-text-input[name = 'email']"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpk_odq6d0/extensions/fxdriver#googlecode.com/components/driver-component.js:10770)
at FirefoxDriver.prototype.findElement (file:///tmp/tmpk_odq6d0/extensions/fxdriver#googlecode.com/components/driver-component.js:10779)
at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmpk_odq6d0/extensions/fxdriver#googlecode.com/components/command-processor.js:12661)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmpk_odq6d0/extensions/fxdriver#googlecode.com/components/command-processor.js:12666)
at DelayedCommand.prototype.execute/< (file:///tmp/tmpk_odq6d0/extensions/fxdriver#googlecode.com/components/command-processor.js:12608)
So there's a problem in this line of code:
element = driver.find_element_by_css_selector("ui-text-input[name = 'email']")
Maybe a wrong syntax? I start using selenium recently so I am still not very good
UPDATED SCRIPT with not errors but it doesn't still work:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/Login")
while True:
element = driver.find_element_by_css_selector("form.login-form input[name=email]")
driver.execute_script("arguments[0].setAttribute('value','test1')", element)
element2 = driver.find_element_by_css_selector("form.login-form input[name=password]")
driver.execute_script("arguments[0].setAttribute('value','test1')", element2)
driver.refresh()
SOLVED. WORKING SCRIPT:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#gmail.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit2 = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit2.click()

Yeah, the ui-text-input[name = 'email'] selector is not going to match anything on this page since there is no <ui-text-input> element on the page, but there is an input with ui-text-input class. Fix your selector:
form.login-form input[name=email]
By the way, instead, of while True approach, use the built into selenium mechanism to wait for specific conditions to happen on a page.
And, should not you be choosing the login type (email/phone) first?

I think below is syntax for CSS selector in selenium
driver.findElement(By.cssSelector("ui-text-input[name = 'email']"))

Solved.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#gmail.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit2 = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit2.click()

Selenium Code works locally but not on PythonAnywhere

I have a webscraper that is running on my system and I wanted to migrate it over to PythonAnywhere, but when I moved it now it doesn't work.
Exactly the sendkeys does not seem to work - after the following code is executed I never move on to the next webpage so an attribute error gets tripped.
My code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import csv
import time
# Lists for functions
parcel_link =[]
token = []
csv_output = [ ]
# main scraping function
def getLinks(link):
# Open web browser and get url - 3 second time delay.
#Open web browser and get url - 3 second time delay.
driver.get(link)
time.sleep(3)
inputElement = driver.find_element_by_id("mSearchControl_mParcelID")
inputElement.send_keys(parcel_code+"*/n")
print("ENTER hit")
pageSource = driver.page_source
bsObj = BeautifulSoup(pageSource)
parcel_link.clear()
print(bsObj)
#pause = WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.ID, "mResultscontrol_mGrid_RealDataGrid")))
for link in bsObj.find(id="mResultscontrol_mGrid_RealDataGrid").findAll('a'):
parcel_link.append(link.text)
print(parcel_link)
for test in parcel_link:
clickable = driver.find_element_by_link_text(test)
clickable.click()
time.sleep(2)
The link I am trying to operate is:
https://ascendweb.jacksongov.org/ascend/%280yzb2gusuzb0kyvjwniv3255%29/search.aspx
and I am trying to send: 15-100*
TraceBack:
03:12 ~/Tax_Scrape $ xvfb-run python3.4 Jackson_Parcel_script.py
Traceback (most recent call last):
File "Jackson_Parcel_script.py", line 377, in <module>
getLinks("https://ascendweb.jacksongov.org/ascend/%28biohwjq5iibvvkisd1kjmm45%29/result.aspx")
File "Jackson_Parcel_script.py", line 29, in getLinks
inputElement = driver.find_element_by_id("mSearchControl_mParcelID")
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 206, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: 'Unable to locate element: {"method":"id","selector":"mSearchControl_mParcelID"}' ; Stac
ktrace:
at FirefoxDriver.findElementInternal_ (file:///tmp/tmpiuuqg3m7/extensions/fxdriver#googlecode.com/components/driver_component.js:9470)
at FirefoxDriver.findElement (file:///tmp/tmpiuuqg3m7/extensions/fxdriver#googlecode.com/components/driver_component.js:9479)
at DelayedCommand.executeInternal_/h (file:///tmp/tmpiuuqg3m7/extensions/fxdriver#googlecode.com/components/command_processor.js:11455)
at DelayedCommand.executeInternal_ (file:///tmp/tmpiuuqg3m7/extensions/fxdriver#googlecode.com/components/command_processor.js:11460)
at DelayedCommand.execute/< (file:///tmp/tmpiuuqg3m7/extensions/fxdriver#googlecode.com/components/command_processor.js:11402)
03:13 ~/Tax_Scrape $
Selenium Innitation:
for retry in range(3):
try:
driver = webdriver.Firefox()
break
except:
time.sleep(3)
for parcel_code in token:
getLinks("https://ascendweb.jacksongov.org/ascend/%28biohwjq5iibvvkisd1kjmm4 5%29/result.aspx")
PythonAnywhere uses a virtual instance of FireFox that is suppose to be headless like JSPhantom so I do not have a version number.
Any help would be great
RS

Well, maybe the browser used on PythonAnywhere does not load the site fast enough. So instead of time.sleep(3) try implicitly waiting for the element.
An implicit wait is to tell WebDriver to poll the DOM for a certain
amount of time when trying to find an element or elements if they are
not immediately available. The default setting is 0. Once set, the
implicit wait is set for the life of the WebDriver object instance.
Using time.sleep() with Selenium is not a good idea in general.
And give it more than just 3 second, with implicitly_wait() you specify the maximum time spent waiting for an element.
So if you set implicitly_wait(10) and the page loads, for example, in 5 seconds then Selenium will wait only 5 seconds.
driver.implicitly_wait(10)