We Keep Coding

can't select a checkbox in webdriver.Chrome python

I'm trying to create a script to show only pikachus on singapore poke map and the rest of the code is to go over the elements and get the coords for it and print the list.
I'm trying for a long time many suggestions I've seen here but still unable to make the checkbox be set with the latest code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
import time
def find_pokemon():
links = []
service = ChromeService(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)
driver.get('https://sgpokemap.com/index.html?fbclid=IwAR2p_93Ll6K9b923VlyfaiTglgeog4uWHOsQksvzQejxo2fkOj4JN_t-MN8')
driver.find_element(By.ID, 'filter_link').click()
driver.find_element(By.ID, 'deselect_all_btn').click()
driver.find_element(By.ID, 'search_pokemon').send_keys("pika")
driver.switch_to.frame(driver.find_elements(By.ID, "filter"))
driver.find_element(By.ID, 'checkbox_25').click()
The second part of the code is working when I'm checking the box manually after putting a breakpoint and ignoring the checkbox click() exception.
Do you have any suggestions what can I try?
Bonus question, how can I determine and close the donate view:

There are several problems with your code:
There is no element with ID = 'search_pokemon'
There is no frame there to switch into it.
You need to use WebDriverWait expected_conditions to wait for elements to be clickable.
And generally you need to learn how to create correct locators.
The following code works:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 30)
url = "https://sgpokemap.com/index.html?fbclid=IwAR2p_93Ll6K9b923VlyfaiTglgeog4uWHOsQksvzQejxo2fkOj4JN_t-MN8"
driver.get(url)
try:
wait.until(EC.element_to_be_clickable((By.ID, 'close_donation_button'))).click()
except:
pass
wait.until(EC.element_to_be_clickable((By.ID, 'filter_link'))).click()
wait.until(EC.element_to_be_clickable((By.ID, "deselect_all_btn"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "[name='search_pokemon']"))).send_keys("pika")
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='filter_checkbox'][not(#style)]//label"))).click()
The result is:
UPD
This time I saw the donation dialog so I added the mechanism to close it.
I still can't see there element with ID = 'search_pokemon' as you mentioned.
As about the XPath to find the relevant checkbox - when pokemon name is inserted you can see in the dev tools that there are a lot of checkboxes there but all of them are invisibly while only one in our case is visible. The invisible elements are all have attribute style="display: none;" while the enabled element does not have style attribute. This is why [not(#style)] is coming there. So, I'm looking for parent element //div[#class='filter_checkbox'] who is also have no style attribute. In XPath words //div[#class='filter_checkbox'][not(#style)] then I'm just looking for it label child to click it. This can also be done with CSS Selectors as well.
The list of invisible elements with the enabled one:

With the help and answers from #Prophet , the current code for crawling the map and getting all of the Pikachus coordinates:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from keep import saveToKeep
def find_pokemon():
links = []
options = Options()
options.add_argument("--headless")
options.add_argument("disable-infobars")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 30)
driver.get('https://sgpokemap.com')
try:
wait.until(EC.element_to_be_clickable((By.ID, 'close_donation_button'))).click()
except:
pass
wait.until(EC.element_to_be_clickable((By.ID, 'filter_link'))).click()
wait.until(EC.element_to_be_clickable((By.ID, "deselect_all_btn"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "[name='search_pokemon']"))).send_keys("pika")
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='filter_checkbox'][not(#style)]//label"))).click()
# count = 0
wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'pokemon_icon_img')))
pokeList = driver.find_elements(By.CLASS_NAME, 'pokemon_icon_img')
for poke in pokeList:
# count += 1
try:
poke.click()
links.append(driver.find_element(By.LINK_TEXT, "Maps").get_attribute('href'))
except Exception:
pass
# if count > 300:
# break
res = []
for link in links:
res.append(link.split("=")[1].replace("'", ""))
# for item in res:
# print(item)
if len(res) > 1:
saveToKeep(res)
print("success")
else:
print("unsuccessful")
find_pokemon()
if __name__ == '__main__':
find_pokemon()
Used the headless chrome option in hope to achieve better
performance.
Commented out 'count' in case I want to limit list results
(currently I'm getting like 15 results tops when unlimited
although there are many more...weird :( )
the following code wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'pokemon_icon_img'))) is needed for now since it's not always
showing icons right away, so it's either that or adding a constant
time delay.
Have made this method recursive in case it's unsuccessful(sometimes it still gives out exceptions)
Lastly, saveToKeep(res) method is a simple method I'm using to open
and write results into my google keep notes. Needed to get an app
password within google security settings and I'm using it with my google account credentials for login.
Any comments or regards for improvements are welcomed :D

Unable to write any text in the input field even though this element seems to be in focus

The following code is not writing any partial string in the From input field on the website even though this element seems to be an active element.
I spent lot of time trying to debug and make the code work but no success. Can anyone please provide some hint on what is wrong. Thanks.
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.common.keys import Keys
from colorama import init, Fore
class BookingTest1():
def __init__(self):
pass
def test1(self):
baseUrl="https://www.goibibo.com/"
driver=webdriver.Chrome()
driver.maximize_window()
#open airline site
driver.get(baseUrl)
driver.implicitly_wait(3)
# Enter origin location.
partialTextOrigin="New"
#select flight tab
driver. find_element(By.XPATH,"//ul[#class='happy-nav']//li//a[#href='/flights/']").click()
# select input box
textElement = driver.find_element(By.XPATH, "//input")
# check if input box is active
if textElement==driver.switch_to.active_element:
print('element is in focus')
textElement.send_keys(partialTextOrigin)
else:
print('element is not in focus')
print("Focus Event Triggered")
driver.execute_script("arguments[0].focus();", textElement)
time.sleep(5)
if textElement==driver.switch_to.active_element:
print('finally element is in focus')
print(partialTextOrigin)
textElement.send_keys(partialTextOrigin)
time.sleep(5)
#test the code
tst=BookingTest1()
tst.test1()

There are several issues here:
First you need to click on p element in the From block and only after that when input appears there you can insert the text to it.
You should use unique locators. (There more that 10 input elements on this page)
Using WebDriverWait expected conditions explicit waits are much better than implicitly_wait in most cases.
No need to set timeouts to too short values.
No need to use driver.switch_to.active_element here.
The following code works for me:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = "https://www.goibibo.com/"
flights_xpath = "//ul[#class='happy-nav']//li//a[#href='/flights/']"
from_xpath = "//div[./span[contains(.,'From')]]//p[contains(text(),'Enter city')]"
from_input_xpath = "//div[./span[contains(.,'From')]]//input"
partialTextOrigin = "New"
wait = WebDriverWait(driver, 10)
driver.get(url)
wait.until(EC.element_to_be_clickable((By.XPATH, flights_xpath))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, from_xpath))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, from_input_xpath))).send_keys(partialTextOrigin)
from_xpath and from_input_xpath XPath locators are a little complex.
I was not sure about the class names in that elements block if they are fixed so I based on the texts.
For example "//div[./span[contains(.,'From')]]//p[contains(text(),'Enter city')]" means:
Find such div that it has a direct span child so that span contains From text content.
From the div parent element above find inside it a p child that contains Enter city text.
Similarly to the above locator "//div[./span[contains(.,'From')]]//input" means: find parent div as described before, then find inside it an input child element.
The result of the code above is

selenium no way to use a visible select element python

I am working on a project that I first sign up instagram by given information and then scrape some data for the sign up form I have some problems with second form which is birthdate the Select class of selenium didn't work. Maybe it is because of visibility. when I tried to access the elements it says that there is no such elements but they are both visible graphically and in inspect page of chrome and I prefer to not use the pyautogui because it is kinda unflexible and hard to use.
here is my code
#information
driverSignUp = webdriver.Chrome()
while True:
while True:
try:
driverSignUp.get('https://www.instagram.com/accounts/emailsignup/')
break
except:
pass
try:
driverSignUp.find_element(By.CSS_SELECTOR, 'body.p-error.dialog-404')
driverSignUp.delete_all_cookies()
except:
break
driverSignUp.implicitly_wait(1)
driverSignUp.find_element(By.NAME, 'emailOrPhone').send_keys(email)
driverSignUp.find_element(By.NAME, 'fullName').send_keys(fullName)
driverSignUp.find_element(By.NAME, 'username').send_keys(username)
driverSignUp.find_element(By.NAME, 'password').send_keys(password)
element = WebDriverWait(driverSignUp, 3).until(
EC.all_of(EC.presence_of_element_located((By.CSS_SELECTOR, 'button.sqdOP.L3NKy.y3zKF')))
)
driverSignUp.find_element(By.CSS_SELECTOR, 'button.sqdOP.L3NKy.y3zKF').click()
#birthdate
actions = ActionChains(driverSignUp)
Month = driverSignUp.find_element(By.CSS_SELECTOR, 'select[title="Month:"]')
actions.move_to_element(Month).perform()
selectMonth = Select(Month)
selectMonth.select_by_visible_text('August')
Day = driverSignUp.find_element(By.CSS_SELECTOR, 'select[title="Day:"]')
actions.move_to_element(Day).perform()
selectDay = Select(Day)
selectDay.select_by_visible_text('1')
Year = driverSignUp.find_element(By.CSS_SELECTOR, 'select[title="Year:"]')
actions.move_to_element(Year).perform()
selectYear = Select(Year)
selectYear.select_by_visible_text('2000')
and the libraries I used:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import Select
thanks for your answers

Why does my Selenium URL_to_be statement not work?

Selenium does not seem to register that I manually go to the publish0x.com page.
Does anyone know a solution?
My goal is to manually do the captcha at the login page and afterwards, when I log in and land on the main page I want the script to resume.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import sys
from selenium.webdriver.support.ui import WebDriverWait
def waitForLoad(inputXPath):
Wait = WebDriverWait(driver, 10)
Wait.until(EC.presence_of_element_located((By.XPATH, inputXPath)))
email = '
password = '
options = Options()
options.add_experimental_option("detach", True)
options.add_argument("--window-size=1920,1080")
## options.add_argument("user-data-dir=/Users/vadim/Library/Application Support/BraveSoftware/Brave-Browser")
options.binary_location = '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'
driver_path = '/usr/local/bin/chromedriver'
driver = webdriver.Chrome(options=options, executable_path=driver_path)
driver.get('https://www.publish0x.com/login')
waitForLoad('//*[#id="email"]')
E_Mail_vak = driver.find_element_by_xpath('//*[#id="email"]')
E_Mail_vak.send_keys(email)
Pass_vak = driver.find_element_by_xpath('//*[#id="password"]')
Pass_vak.send_keys(password)
frame = driver.find_element_by_xpath('//iframe[contains(#src, "recaptcha")]')
driver.switch_to.frame(frame)
Captcha = driver.find_element_by_xpath("//*[#id='recaptcha-anchor']")
Captcha.click()
wait = WebDriverWait(driver, 500)
wait.until(EC.url_to_be("publish0x.com"))
driver.get('https://www.publish0x.com/newposts')
post = driver.find_element_by_css_selector('#main > div.infinite-scroll > div:nth-child(1) > div.content')
title = post.find_element_by_css_selector('h2 > a').text
author = post.find_element_by_css_selector('p.text-secondary > small:nth-child(4) > a').text
title.click()
slider = driver.find_element_by_xpath('//*[#id="tipslider"]')

There are two ways I can think of, one by adding an Input statement like this:
options.add_argument('--disable-gpu')#For properly seeing the outputs
input("Please do the captcha and press any key...)
In this way, the user would complete the data and then press any key for the script to continue.
The other way is by adding a try and except statement.
try:
driver.find_element_by_id("some-element")
except NoSuchElementException:
#do something like
print("Captcha Failed or Incomplete...")
In this, replace the element id "some-element" with any element that is present and only present after the user logs in, for e.g elements like Profile_nav or Settings are only present when someone logs in. So if the element doesn't exist then it would mean that the user didn't complete the captcha.

Using python and selenium how can i get the value of placeholder where the value is not utf-8?

I was trying to get the email address and click on the refresh button from the following screenshot. But I am getting errors.
My code for this is like the following:
from selenium import webdriver
url = 'http://od.obagg.com/ '
driver = webdriver.Chrome(executable_path='chromedriver')
driver.get(url)
s = driver.find_element_by_id('//*[#id="shortid"]').get_attribute('placeholder')
print(s)
Based on the inspect, i was trying to do and tried many ways to get that email field value and click on refresh button. But still no luck.
Do anybody know any tricks to share?

It may be due to the fact the element is disabled, also, find_element_by_id('//*[#id="shortid"]') is incorrect. It can be either:
find_element_by_xpath('//*[#id="shortid"]')
find_element_by_id("shortid") ?
The following works for me:
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Chrome(executable_path='chromedriver')
driver.maximize_window()
driver.get('http://od.obagg.com/')
wait = WebDriverWait(driver, 10)
el = wait.until(ec.visibility_of_element_located((By.ID, "shortid")))
placeholder = el.get_attribute("placeholder")
email = el.get_attribute('value')
print(placeholder, email)
# 请等待分配临时邮箱 -_ylp06tc#xxx.xxx
If you need 10 different emails, you can use:
from time import sleep
for x in range(10):
driver.find_element_by_id("refreshShortid").click()
sleep(0.15) # you may have to increase this value to give enough time to generate the new email
new_email = driver.find_element_by_id("shortid").get_attribute('value')
print(new_email)