We Keep Coding

Selenium Web Scraping

I would like my web scraper to go to a web site and press the log in button, then put random credentials and submit, here is the html code, please help
<div class="container">
<div class="row header-box">
<div class="col-md-8">
<h1>
Quotes to Scrape
</h1>
</div>
<div class="col-md-4">
<p>
Login
</p>
</div>
</div>
Here is the code I have by far
# Start driver
driver_path = "chromedriver"
driver = webdriver.Chrome(driver_path)
# Navigate to the website
driver.get('http://quotes.toscrape.com/')
driver.maximize_window()
driver.find_element('//bento/orange[contains(#Class,"small")]').click()
To press login button and put random credentials in

Use WebDriverWait() and wait for element to be clickable. User following xpath options
driver.get("http://quotes.toscrape.com/")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Login']"))).click()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[#id='username']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[#id='password']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[#value='Login']"))).click()
Import below libraries
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

To click on the Login link, fill in Username and Password and click on Login you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategies:
driver.get("https://quotes.toscrape.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href='/login']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#username"))).send_keys("Orkhan")
driver.find_element(By.CSS_SELECTOR, "input#password").send_keys("Karimov")
driver.find_element(By.CSS_SELECTOR, "input[value='Login']").click()

Click on item using Selenium and Python

I am trying to click on the "Training material statistics" by using the following code in Python, but it didn't work:
WebDriverWait(driver,20)\
.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="report-navigation"]/div[2]')))\
.click()
HTML:
<div id="report-navigation">
<div class="report-nav-btn active" onclick="Report.changeGrid(this, 'report-users-grid')">
User statistics
<div class="report-nav-arrow active"></div>
</div>
<div class="report-nav-btn" onclick="Report.changeGrid(this, 'report-objects-grid')">
Training material statistics
<div class="report-nav-arrow"></div>
</div>
<div class="report-nav-btn" onclick="Report.changeGrid(this, 'report-deliverables-grid')">
Learner assignments
<div class="report-nav-arrow"></div>
</div>
</div>
HTML Snapshot:

To click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#report-navigation div[onclick*='report-objects-grid']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#id='report-navigation']//div[contains(#onclick, 'report-objects-grid')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

I'm trying to get information from the site, using the right css selectors, but it still returns me as if the element didn't exist

trying to build a bot to my own services, but while i'm trying to run its crashs
login_button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "//div[#class='loginRegisterButtons sb-login-register-buttons clearfix ember-view']//a[#class='btn signin-btn sbLoginBtn buttons1571052392296']")))
login_button.click()
the html
<div class="loginRegisterButtons sb-login-register-buttons clearfix ember-view">
<div>
<a class="btn signin-btn sbLoginBtn buttons1571052392296">
<span class="btf-text">ENTRAR</span>
</a>
</div>
</div>

The desired element is a Ember.js enabled element, so to click on the element you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn.signin-btn.sbLoginBtn > span.btf-text"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#class='btf-text' and text()='ENTRAR']"))).click()
Note : You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Select input element by id using Selenium Python

I'm trying to click on a box that has the following HTML code:
<li>
<input id="mce-group[166]-166-0" type="checkbox" value="1" name="group[166][1] >
<label for="mce-group[166]-166-0">I agree</label>
</li>
I've tried it all: id, name, xpath, text,...
Running something like this:
select_box = driver.find_element_by_xpath('//*[#id="mce-group[166]-166-0"]')
select_box.click()
I get this error:
selenium.common.exceptions.ElementNotInteractableException: Message: Element <input id="mce-group[166]-166-0" name="group[166][1]" type="checkbox"> could not be scrolled into view

To click on the <input> element associated with the text I agree you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "label[for=\"mce-group[166]-166-0\"]"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[#for=\"mce-group[166]-166-0\"]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Python - Selenium Webdriver find an element where all the div nodes have similar names

I am trying to access a div where all divs have the same name. Let me explain. I am just starting out with selenium and python and I am trying to scrape a webpage just to learn. I ran into the following problem. I made the example html to show the buildup of the webpage. All the divs have the exact same class and title. Then there is the h1 tag for the item and the p tag for the color (which is a clickable link). I am trying to search a page when you give it certain instructions. Example: I am looking for a white racebike. I am able to find the bikes with the first line of code, but how do I find the right color within the racebike section? If I run the Python mentioned below I get an error message. Thanks in advance!
<!DOCTYPE html>
<html>
<body>
<div class=div title=example>
<h1>racebike</h1>
<p class='test'>black</p>
</div>
<div class=div title=example>
<h1>racebike</h1>
<p class='test'>white</p>
</div>
<div class=div title=example>
<h1>racebike</h1>
<p class='test'>yellow</p>
</div>
<div class=div title=example>
<h1>citybike</h1>
<p class='test'>yellow</p>
</div>
<div class=div title=example>
<h1>citybike</h1>
<p class='test'>green</p>
</div>
</body>
</html>
test = (self.driver.find_element_by_xpath("//*[contains(text(), racebike)]"))
test.self.driver.find_element_by_xpath(".//*[contains(text(), white)]").click

To locate/click() on the white racebike element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following xpath based Locator Strategies:
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//h1[text()='racebike']//following-sibling::p[#class='test' and text()='white']"))).click()
Using XPATH considering the parent <div>:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='div' and #title='example']/h1[text()='racebike']//following-sibling::p[#class='test' and text()='white']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can use same xpath which you tried in your solution. It might be possible server is taking too long to repond.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
element = WebDriverWait(page, 10).until(EC.presence_of_element_located((By.XPATH, "//p[contains(#class, 'white')]")))
element.click()
for multiple bikes with whiite color
elements= WebDriverWait(driver, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//p[contains(#class, 'white')]")))
for element in elements:
element.click()