Please click on the link below to see the link "BEAUTY" on which I am clicking
1. I am using this code to click on the "Beauty" link
driver = webdriver.Chrome("C:\\Users\\gaurav\\Desktop\\chromedriver_win32\\chromedriver.exe")
driver.maximize_window()
driver.get("http://shop.davidjones.com.au")
object = driver.find_elements_by_name('topCategory')
for ea in object:
print ea.text
if ea.text == 'Beauty':
ea.click()
I am getting the following exceptions after clickin on the link succesfully , can anybody tell me why I am getting it ?
Traceback (most recent call last):
File "C:/Users/gaurav/PycharmProjects/RIP_CURL/login_raw.py", line 10, in <module>
print ea.text
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py", line 73, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py", line 493, in _execute
return self._parent.execute(command, params)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=63.0.3239.132)
(Driver info: chromedriver=2.34.522940 (1a76f96f66e3ca7b8e57d503b4dd3bccfba87af1),platform=Windows NT 6.2.9200 x86_64)
Try this:
from selenium import webdriver
print("bot started")
#chromeOptions = webdriver.ChromeOptions()
#driver = webdriver.Chrome(chrome_options=chromeOptions)
def specific_text(text, ea):
return str(text) == ea.text
driver = webdriver.Chrome("C:\\Users\\gaurav\\Desktop\\chromedriver_win32\\chromedriver.exe")
driver.maximize_window()
driver.get("http://shop.davidjones.com.au")
object_ = driver.find_elements_by_name('topCategory')
text_headers = [str(specific_text('Beauty', ea)) for ea in object_]
#print(text_headers)
index_text = text_headers.index("True")
#print(index_text)
object_[index_text].click()
You need to take care of certain factors as follows :
You have tried to create a List by the name object. object is a reserved built-in symbol in most of the Programming Languages. So as per Best Programming Practices we shouldn't use the name object.
The line print ea.text is badly indented. You need to add indentation.
Once you invoke click() on the WebElement with text as Beauty you need to break out of the loop.
Here is your own working code with some minor tweaks :
from selenium import webdriver
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe')
driver.maximize_window()
driver.get("http://shop.davidjones.com.au")
object1 = driver.find_elements_by_name('topCategory')
for ea in object1:
print (ea.text)
if ea.text == 'Beauty':
ea.click()
break
Console Output :
Sale
Brands
Women
Men
Shoes
Bags & Accessories
Beauty
There's an easier way to do this. You can use an XPath that will specify the category name you want to click. That way you don't have to loop, it will find the desired element in one search.
//span[#name='topCategory'][.='Beauty']
I'm assuming you will be reusing this code. In cases like this, I would create a function that takes a string parameter which would be the category name that you want to click. You feed that parameter into the XPath above and you can then click any category on the page.
I tested this and it's working.
Related
I am trying to scrap this site https://www.youtube.com/#JohnWatsonRooney/videos for the video titles.
This short explination does not help with the problem of the code but I thought I would say it anyway. I am doing this so I can get my foot in the door to then scrap other sites for prices on stuff so I can make a program that gives me a link to the cheapest priced product from a list several sites
as well as a ranking of all the other products and there names incase that wasn't the one I was looking for.
here is the terminal output.
Traceback (most recent call last):
File "C:\\Users\\PRO\\Documents\\dynamic_web_scraping_test.py", line 11, in \<module\>
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 830, in find_element
return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})\["value"\]
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: \[name="style-scope ytd-rich-grid-media"\]
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
NoSuchElementError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:392:5
element.find/\</\<#chrome://remote/content/marionette/element.sys.mjs:275:16
C:\\Users\\PRO\\Documents\>
here is my code
import time
from selenium import webdriver
url1 = 'https://www.youtube.com/#JohnWatsonRooney/videos'
url2 = 'https://www.youtube.com/#MyCraftyDen/videos'
driver = webdriver.Firefox()
driver.get(url1)
time.sleep(10)
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
for video in videos:
titles = video.find_element_by_xpath('//\*\[#id="video-title-link"\]').text
print(title)
I don't know what to try I did my best looking up this problem before posting. I know waiting for the browser to load the page doesnt work hense the time.sleep(10) in my code.
If you are looking to find the links of all the videos in the given page, you can use below code
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
url1 = 'https://www.youtube.com/#JohnWatsonRooney/videos'
url2 = 'https://www.youtube.com/#MyCraftyDen/videos'
driver = webdriver.Firefox()
driver.get(url1)
time.sleep(10)
videos = driver.find_elements(By.ID, "video-title-link")
for video in videos:
title = video.get_attribute("href")
print(title)
EDITED: I incorporated the final lines suggested by Sushil. At the end, I am copying the output in my terminal. I still do not get the zipfile.
SOLVED: My error was due to an incompatibility between the driver and chrome versions. I fixed by following the instructions here: unknown error: call function result missing 'value' for Selenium Send Keys even after chromedriver upgrade
I am trying to use Selenium to fill out a form and download a zipfile.
After extensively googling, I have written a Python code, but I am currently unable to download the file. A browser opens, but nothing is filled out.
I am very new at Python, so I am guessing I am missing something very trivial since the website I am trying to get info from is super simple.
This is what I have tried:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path='/home/miranda/webscrap_Python/chromedriver')
#driver.wait = WebDriverWait(driver,5)
driver.get("http://www.cis.es/cis/opencms/EN/formulario.jsp?dwld=/Microdatos/MD3288.zip")
Name = '//*[#id="Nombre"]'
LastName = '//*[#id="Apellidos"]'
University = '//*[#id="profesion"]'
email = '//*[#id="Email"]'
ob_req = '//*[#id="objeto1"]'
terms = '//*[#id="Terminos"]'
download = '//*[#id="mediomicrodatos"]/form/div[3]/input'
driver.find_element_by_xpath(Name).send_keys("Miranda")
driver.find_element_by_xpath(LastName).send_keys("MyLastName")
driver.find_element_by_xpath(University).send_keys("MySchool")
driver.find_element_by_xpath(email).send_keys("my_email#gmail.com")
#lines added by Sushil:
ob_req_element = driver.find_element_by_xpath(ob_req) #Finds the ob_req element
driver.execute_script("arguments[0].click();", ob_req_element) #Scrolls down to the element and clicks on it
terms_element = driver.find_element_by_xpath(terms) #The same is repeated here
driver.execute_script("arguments[0].click();", terms_element)
driver.find_element_by_xpath(download).click() #Scrolling down is not needed for the download button as it would already be in view. Only when an element is not in view should we scroll down to it in order to click on it.
Output in my terminal:
Traceback (most recent call last):
File "myCode.py", line 18, in <module>
driver.find_element_by_xpath(Name).send_keys("Miranda")
File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
'value': keys_to_typing(value)})
File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: call function result missing 'value'
(Session info: chrome=86.0.4240.75)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 5.4.0-45-generic x86_64)
For each and every element below the email element, you have to scroll down to click them. Here is the full code to do it:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
#driver.wait = WebDriverWait(driver,5)
driver.get("http://www.cis.es/cis/opencms/EN/formulario.jsp?dwld=/Microdatos/MD3288.zip")
Name = '//*[#id="Nombre"]'
LastName = '//*[#id="Apellidos"]'
University = '//*[#id="profesion"]'
email = '//*[#id="Email"]'
ob_req = '//*[#id="objeto1"]'
terms = '//*[#id="Terminos"]'
download = '//*[#id="mediomicrodatos"]/form/div[3]/input'
driver.find_element_by_xpath(Name).send_keys("Miranda")
driver.find_element_by_xpath(LastName).send_keys("MyLastName")
driver.find_element_by_xpath(University).send_keys("MySchool")
driver.find_element_by_xpath(email).send_keys("my_email#gmail.com")
#All the lines below were added by me
ob_req_element = driver.find_element_by_xpath(ob_req) #Finds the ob_req element
driver.execute_script("arguments[0].click();", ob_req_element) #Scrolls down to the element and clicks on it
terms_element = driver.find_element_by_xpath(terms) #The same is repeated here
driver.execute_script("arguments[0].click();", terms_element)
driver.find_element_by_xpath(download).click() #Scrolling down is not needed for the download button as it would already be in view. Only when an element is not in view should we scroll down to it in order to click on it.
I tried Most of the solution of StackOverflow but didn't work for me
I am trying to send some course name to youtube search bar using selenium python it works fine before but now it gives this error while doing this
And search_bar.send_keys(course_name) works fine for other websites but not in YT
Traceback (most recent call last):
File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
File "/home/sh4d0w/PycharmProjects/AutoMate/venv/lib/python3.7/site-packages/eel/__init__.py", line 257, in _process_message
return_val = _exposed_functions[message['name']](*message['args'])
File "/home/sh4d0w/PycharmProjects/AutoMate/SmallTalk.py", line 72, in SingleQueryinputValue
RecommendCourse.getUdacityCourse(str(val))
File "/home/sh4d0w/PycharmProjects/AutoMate/RecommendCourse.py", line 160, in getUdacityCourse
getYoutubeCourse(course_name, driver)
File "/home/sh4d0w/PycharmProjects/AutoMate/RecommendCourse.py", line 98, in getYoutubeCourse
search_bar.send_keys(course_name)
File "/home/sh4d0w/PycharmProjects/AutoMate/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
'value': keys_to_typing(value)})
File "/home/sh4d0w/PycharmProjects/AutoMate/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "/home/sh4d0w/PycharmProjects/AutoMate/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/sh4d0w/PycharmProjects/AutoMate/venv/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: chrome=81.0.4044.129)
2020-04-29T08:00:02Z <Greenlet at 0x7fd2089c67b8: _process_message({'call': 2.1877049007713376, 'name': 'SingleQueryi, <geventwebsocket.websocket.WebSocket object at 0x7)> failed with ElementNotInteractableException
Code sample
option = webdriver.ChromeOptions()
option.add_argument("window-size=1200x600");
driver = webdriver.Chrome('/usr/bin/chromedriver', options=option)
driver.get("https://www.youtube.com")
getYoutubeCourse(course_name, driver)
getYoutubeCourse() function body
def getYoutubeCourse(course_name, driver):
time.sleep(2)
search_bar = driver.find_element_by_xpath('//*[#id="search"]')
search_bar.send_keys(course_name)
search_bar_button = WebDriverWait(driver, 5).until(EC.element_to_be_clickable(
(By.XPATH, '//*[#id="search-icon-legacy"]')))
search_bar_button.click()
......
then after this, the logic of scraping the youtube links is there
I also tried web driver wait and all, and my drivers are also up to date
Please help I am new in python and selenium
There is 3 elements found by xpath : //*[#id="search"]
You have to correct it to
//input[#id="search"]
I also got the same problem. After hours filled with agony I finally found a solution.
While copying xpath (or any another id you are using to refer it) it points out to the entire search box container. After repeated trials and observations I found that a particular element (of the webpage which you can view in developer) with input mentioned was the one to be looked for:
<input id="search" autocapitalize="none" autocomplete="off" autocorrect="off" name="search_query" tabindex="0" type="text" spellcheck="false" placeholder="Search" aria-label="Search" aria-haspopup="false" role="combobox" aria-autocomplete="list" dir="ltr" class="ytd-searchbox" style="outline: none;">
The problem while copying xpath is that you get id of the container.
So use 'copy full xpath'. This will refer to the particular element within the container which can be given input.
full xpath for the input element of search box is:
'/html/body/ytd-app/div/div/ytd-masthead/div[3]/div[2]/ytd-searchbox/form/div/div[1]/input'
You can use it like:
search_bar = driver.find_element_by_xpath('/html/body/ytd-app/div/div/ytd-masthead/div[3]/div[2]/ytd-searchbox/form/div/div[1]/input')
search_bar.send_keys(song_name)
time.sleep(1)#You can skip this.
search_bar.send_keys(Keys.ENTER)
P.S. Excuse the terminology. I am also a beginner and this is my first answer.
You can try this:
from selenium.webdriver.common.keys import Keys
def getYoutubeCourse(course_name, driver):
time.sleep(2)
search_bar = driver.find_element_by_xpath('//input[#id="search"]')
search_bar.send_keys(course_name)
time.sleep(1)
search_bar.send_keys(Keys.ENTER)
....
I'm the one who also got the same problem.
I tried full Xpath but it was still not working. And then, I plused [0] after xpath and it worked.
full xpath for the input element of search box is:
'/html/body/ytd-app/div/div/ytd-masthead/div[3]/div[2]/ytd-searchbox/form/div/div[1]/input[0]'
And this if full source code
chrome_driver = '/Users/Downloads/chromedriver'
driver = webdriver.Chrome(chrome_driver)
driver.get('https://www.youtube.com')
search_bar = driver.find_elements_by_xpath('/html/body/ytd-app/div/div/ytd-masthead/div[3]/div[2]/ytd-searchbox/form/div/div[1]/input')[0]
time.sleep(2)
search_bar.send_keys("black pink")
time.sleep(2)
search_bar.send_keys(Keys.RETURN)
Thank you
I dont know where the problem is
I used theese sources
https://medium.com/python/python-selenium-mod%C3%BCl%C3%BC-kullan%C4%B1m%C4%B1-ders-1-36983185164c
https://www.youtube.com/watch?v=olOswIyeRlY
Code
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver_path = r"C:\Users\ruzgaricatsirketi\Desktop\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(executable_path=driver_path)
driver.get("https://translate.google.com/#tr/tr")
dinle = driver.find_element_by_id('gt-speech')
dinle.click()
time.sleep(2)
dinle.click()
time.sleep(3)
getir = driver.find_element_by_id('result_box').text
print(getir)
Error
DevTools listening on ws://127.0.0.1:12239/devtools/browser/76eb55d7-8aef-4b19-9d6e-fda8f4f1aee4
Traceback (most recent call last):
File "C:\Users\ruzgaricatsirketi\Desktop\test.py", line 13, in <module>
getir = driver.find_element_by_id('result_box').text
File "C:\Users\ruzgaricatsirketi\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Users\ruzgaricatsirketi\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users\ruzgaricatsirketi\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\ruzgaricatsirketi\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"result_box"}
(Session info: chrome=78.0.3904.97)
(Driver info: chromedriver=2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.18362 x86_64)
In order to work with Python correctly, it is crucial to learn to read the stack trace. In your case, these lines
File "C:\Users\ruzgaricatsirketi\Desktop\test.py", line 13, in <module>
getir = driver.find_element_by_id('result_box').text
say that the exception has its origin on line 13 in your file test.py, where you are trying to search for web element with id attribute set to value "result_box". Exception you have got is NoSuchElementException:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"result_box"}
As you can find in official documentation or many other places (and I bet there are thousands of them), this exception is raised when the element you are searching for is not found.
Try it yourself — open your tested Google Translator page and try to search for element with any existing id. E.g. on line 8 in your test file you are trying to locate element with id "gt-speech". Write document.getElementById("gt-speech") to your browsers console and whoala! It works, this node matches your search:
<div id="gt-speech" class="speech-button goog-toolbar-button"...>...</div>
Note especially the id="gt-speech" attribute, it is the one selenium will use to identify the object. However, if you try the same with id "result_box", you will get nothing (null is returned if using JS console to search for element), because there is no such element on the page (thus NoSuchElementException...).
My advice for you:
inspect the element you are trying to locate (on the most of the popular web browsers on Windows/Linux systems it is as easy as right-clicking the element and selecting the inspect option)
select some unique properties, you can use also other properties than id (preferably not inner text; IDs are best, but they are not always present...)
use those properties to find the element using selenium
And please, next time try to search a bit on your own before you ask, because this is some really really trivial problem you should be able to handle after first 5 minutes of any selenium course (in other words, UTFG :P)
EDIT
P.S. I can see you have got that invalid id from the video you have marked as your source. The fact that element with such id was present one year ago does not mean it will be there forever (an for every platform). If you need to get that certain element, you have to find another way how to define it using selenium locators I have mentioned earlier.
I'm scraping reviews from the Google Play homepage.
I come out well and stop on the way.
I get the following error:
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:51812","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1505887578605\", \"sessionId\": \"cca93cc0-9dc9-11e7-a685-bd84ddef3ed2\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/cca93cc0-9dc9-11e7-a685-bd84ddef3ed2/element/:wdc:1505887578605/click"}}
Screenshot: available via screen
I did a search and many revisions, but I could not fix it.
I have been stopping for several days. I will upload my code.
Why does this error occur?
How should I fix it?
#python 3.6
from selenium import webdriver
from time import sleep
from bs4 import BeautifulSoup, Comment
import pandas as pd
#Setting up Chrome webdriver Options
#chrome_options = webdriver.ChromeOptions()
#setting up local path of chrome binary file
#chrome_options.binary_location =
"/Users/Norefly/chromedriver2/chromedriver.exec"
#creating Chrome webdriver instance with the set chrome_options
driver = webdriver.PhantomJS("C:/Python/phantomjs-2.1.1-
windows/bin/phantomjs.exe")
link = "https://play.google.com/store/apps/details?
id=com.supercell.clashofclans&hl=en"
driver.get(link)
#driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
Ptitle = driver.find_element_by_class_name('id-app-title').text.replace(' ','')
print(Ptitle)
#driver.find_element_by_xpath('//*[#id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]').click()
sleep(1)
driver.find_element_by_xpath('//*[#id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div').
click()
#select_newest.select_by_visible_text('Newest')
#driver.find_element_by_xpath('//*[#id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div').
click()
sleep(2)
#driver.find_element_by_css_selector('.review-filter.id-review-sort-filter.dropdown-menu-container').click()
driver.find_element_by_css_selector('.displayed-child').click()
#driver.find_element_by_xpath("//button[#data-dropdown-value='1']").click()
driver.execute_script("document.querySelectorAll('button.dropdown-child')[0].click()")
reviews_df = []
for i in range(1,10000):
try:
for elem in driver.find_elements_by_class_name('single-review'):
print(str(i))
content = elem.get_attribute('outerHTML')
soup = BeautifulSoup(content, "html.parser")
#print(soup.prettify())
date = soup.find('span',class_='review-date').get_text()
rating = soup.find('div',class_='tiny-star')['aria-label'][6:7]
title = soup.find('span',class_='review-title').get_text()
txt = soup.find('div',class_='review-body').get_text().replace('Full Review','')[len(title)+1:]
print(soup.get_text())
temp = pd.DataFrame({'Date':date,'Rating':rating,'Review Title':title,'Review Text':txt},index=[0])
print('-'*10)
reviews_df.append(temp)
#print(elem)
except:
print('s')
driver.find_element_by_xpath('//*[#id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div').
click()
reviews_df = pd.concat(reviews_df,ignore_index=True)
reviews_df.to_csv(Ptitle+'review_google.csv', encoding='utf-8')
#driver.close()
Because I do not know, I raise the phrases and paths of errors.
Traceback (most recent call last):
File "C:/Users/lobyp/Downloads/reviewex.py", line 51, in <module>
driver.find_element_by_xpath('//*[#id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div').
click()
File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 78, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 499, in _execute
return self._parent.execute(command, params)
File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 297, in execute
self.error_handler.check_response(response)
File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:53283","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1505888277040\", \"sessionId\": \"6b69b0a0-9dcb-11e7-bc02-87f5b92766da\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/6b69b0a0-9dcb-11e7-bc02-87f5b92766da/element/:wdc:1505888277040/click"}}
Screenshot: available via screen
I'm not an expert, but I believe that if an element has the hidden HTML attribute, then this exception will appear. I had the same issue with a drop down menu in a site once, I had to use the scroll event in Selenium to make the element visible.