We Keep Coding

Python Selenium problems with passing login information to user/password form

So, the last 3 hours or so I have tried to get Selenium to work without success. I managed to make it work with requests and Beautifulsoup, but apparently site uses javascript to load data after login so I cannot scrape the data I want after successful login.
Below is the script I am trying to work with.
``
import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
driver = webdriver.Chrome("/home/user/Desktop/chromedriver")
username = "FakeUsername"
password = "PasswordFake"
driver.get("https://www.helen.fi/kirjautuminen")
time.sleep(10)
# find username/email field and send the username itself to the input field
# find password input field and insert password as well
driver.find_element(By.XPATH,'//input[#id="username"]').send_Keys(username)
# click login button
``
(Yes, I know its missing password and submit actions, but I can't get it to write anything into username or password input boxes. Also same script seems to be working fine with github's login page, so I really can't understand what I am doing wrong here).
After running the script, chrome opens, site loads fine, but for some reason I get error
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//input[#id="username"]"}
I have tried with element ID and Name with similar errors, except it said something about "unable to locate element: css selector ..."
If anyone has some advice to give a newbie, it would be awesome. This is starting to give me headache.
I except the script to write username into username input box, but nothing happens.

this occurs because there is an iframe in the page code. It is necessary to switch to the iframe and then search for the element, follow the code with the correction.
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path= "chromedriver.exe")
username = "FakeUsername"
password = "PasswordFake"
driver.get("https://www.helen.fi/kirjautuminen")
# find username/email field and send the username itself to the input field
# find password input field and insert password as well
iframe = driver.find_element(By.XPATH, '//iframe[#class="login-iframe"]')
driver.switch_to.frame(iframe)
driver.find_element(By.XPATH,'//input[#id="username"]').send_keys(username)
# click login button

How to loop back if password/username is wrong and print to user

I am currently writing in python to login to a website but want to facilitate as much user error as possible.
I want the code to print "Username/Password Incorrect" and then loop back to enter the username and password again.
Sadly I cant share the website with you as its on an intranet.
My Code:
#Enter Username
username = driver.find_element(By.NAME, value='j_username')
username.send_keys(input('Enter Username:'))
#Enter Password
password = driver.find_element(By.NAME, value='j_password')
password.send_keys(input('Enter Password:'))
I was wondering if its possible to get the error and loop back when this element is the displayed on the website:
Thanks for anyone who helps

You have 2 possible scenarios: Login success and Login fail. You need to wait for the error message to appear in order to retry once it's here.
First let's import what we need
from selenium import webdriver as driver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
We can then start iterating
timeout = 5
error_msg_selector = '//div[#class="error"]' // example selector by class
while True:
try_login() // put your 4 lines in this function
try:
is_error_present = EC.presence_of_element_located((By.ID, 'element_id'))
WebDriverWait(driver, timeout).until(is_error_present)
print('Error message found, trying again')
driver.refresh()
continue
except TimeoutException:
print('Timed out while waiting for the error')
break
print('We got out of the loop, login succeeded')
Note: It might be better to find an element that indicates successful login instead of the opposite, so that you don't have to check for multiple error messages.

.current_url returns the url the driver is currently on. If you have a unique identifier in the successful login attempt you could run a while loop with a not statement in it and keep looping through until the user successfully loges in and the condition is met.

Unable to click on signs on a map

I've written a script in Python in association with selenium to click on each of the signs available in a map. However, when I execute my script, it throws timeout exception error upon reaching this line wait.until(EC.staleness_of(item)).
Before hitting that line, the script should have clicked once but It could not? How can I click on all the signs in that map cyclically?
This is the site link.
This is my code so far (perhaps, I'm trying with the wrong selectors):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://www.findapetwash.com/"
driver = webdriver.Chrome()
driver.get(link)
wait = WebDriverWait(driver, 15)
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "#map .gm-style"))):
item.click()
wait.until(EC.staleness_of(item))
driver.quit()
Signs visible on that map like:
Post script: I know that this is their API https://www.findapetwash.com/api/locations/getAll/ using which I can get the JSON content but I would like to stick to the Selenium way. Thanks.

I know you wrote you don't want to use the API but using Selenium to get the locations from the map markers seems a bit overkill for this, instead, why not making a call to their Web service using requests and parse the returned json?
Here is a working script:
import requests
import json
api_url='https://www.findapetwash.com/api/locations/getAll/'
class Location:
def __init__(self, json):
self.id=json['id']
self.user_id=json['user_id']
self.name=json['name']
self.address=json['address']
self.zipcode=json['zipcode']
self.lat=json['lat']
self.lng=json['lng']
self.price_range=json['price_range']
self.photo='https://www.findapetwash.com' + json['photo']
def get_locations():
locations = []
response = requests.get(api_url)
if response.ok:
result_json = json.loads(response.text)
for location_json in result_json['locations']:
locations.append(Location(location_json))
return locations
else:
print('Error loading locations')
return False
if __name__ == '__main__':
locations = get_locations()
for l in locations:
print(l.name)
Selenium
If you still want to go the Selenium way, instead of waiting until all the elements are loaded, you could just halt the script for some seconds or even a minute to make sure everything is loaded, this should fix the timeout exception:
import time
driver.get(link)
# Wait 20 seconds
time.sleep(20)
For other possible workarounds, see the accepted answer here: Make Selenium wait 10 seconds

You can click one by one using Selenium if, for some reasons, you cannot use API. Also it is possible to extract information for each sign without clicking on them with Selenium.
Here code to click one by one:
signs = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "li.marker.marker--list")))
for sign in signs:
driver.execute_script("arguments[0].click();", sign)
#do something
Try also without wait, probably will work.

Selenium Python bindings: dealing with inconsistent modals

I'm trying to fill out a form on a website that has different versions depending on language, location, etc. I am using the same firefox profile for every request, and the previously selected information is stored locally in cookies, so after choosing a setting the first time, the modal doesn't appear for a while. However: it seems to inconsistently appear, taking focus away from the form and causing an ElementNotInteractableException.
To add more difficulty, the modal will often appear some time after the page has loaded. For example, the first field will already be filled out, and then it appears.
My question is, what is the best way to handle this modal? Can I catch the exception cause by its appearance, check for the presence of the modal, and then continue populating the form fields? Or is there a better solution?
Thanks for any help.
The code I have tried so far:
url = "https://www.aircanada.com/ca/en/aco/home.html"
control_profile = webdriver.FirefoxProfile('/path/to/my/profile')
browser_control = webdriver.Firefox(control_profile)
browser_control.get(url)
# To deal with the modal, but obviously fails when it is not present
browser_control.find_element_by_id('enCAEdition').click()
# two text fields I tried to fill out, as a sanity check
departure = browser_control.find_element_by_id('origin_focus_0')
departure.send_keys("my departure location")
departure.send_keys(Keys.RETURN)
destination = browser_control.find_element_by_id('destination_label_0')
destination.send_keys("my destination")
destination.send_keys(Keys.RETURN)

You can wait some time until modal appears, close it and handle form:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
url = "https://www.aircanada.com/ca/en/aco/home.html"
control_profile = webdriver.FirefoxProfile('/path/to/my/profile')
browser_control = webdriver.Firefox(control_profile)
browser_control.get(url)
# Wait up to 10 seconds until modal appears to close it
try:
wait(browser_control, 10).until(EC.element_to_be_clickable(('xpath', '//button[text()="Confirm | Confirmer"]'))).click()
# If modal didn't appear- just continue
except TimeoutException:
pass
departure = browser_control.find_element_by_xpath('//input[#placeholder="FROM"]')
browser_control.execute_script('arguments[0].setAttribute("class","glyph-input glyph-left-input form-control ng-pristine ng-valid ng-touched");', departure)
departure.send_keys("Berlin")
wait(browser_control, 5).until(EC.visibility_of_element_located(("xpath", "(//div[#class='location-primary']/span)[1]")))
departure.send_keys(Keys.RETURN)
destination = browser_control.find_element_by_xpath('//input[#placeholder="TO"]')
browser_control.execute_script('arguments[0].setAttribute("class","glyph-input glyph-left-input form-control ng-pristine ng-valid ng-touched");', destination)
destination.send_keys("Oslo")
wait(browser_control, 5).until(EC.visibility_of_element_located(("xpath", "(//div[#class='location-primary']/span)[4]")))
destination.send_keys(Keys.RETURN)

Enter data into input field not working with selenium

I'm making basic script for sing-in into Instagram.
I faced with this error
Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox()
browser.get('https://www.instagram.com/accounts/login/')
username = browser.find_element_by_name("username").send_keys('login')

Try to add
time.sleep(5)
before
username = browser.find_element_by_name("username").send_keys('login')
Might be page not fully loaded

The error reads 'Window not found. The browser window may have been closed'. It happened in line browser.find_element_by_tag_name("body"). The Instagram login page has two such elements, and you need only one of them. You should make the query more specific, for instance by making it return only the first element with the tag.