We Keep Coding

trouble with selenium find.element

Hello stackoverflow comunity, i'm facing a problem with .find element of selenium. I'm programing in python and using google chrome.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time
chrome_driver_path = "C:\Development\chromedriver.exe"
driver = webdriver.Chrome(service=Service(chrome_driver_path))
driver.get("https://tinder.com/")
time.sleep(5)
accept = driver.find_element(By.XPATH, value='//*[#id="c-1351236777"]/div/div[1]/div/div/main/div/div[2]/div/div[3]/div/div/button[2]')
accept.click()
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: 'value' must be a string
(Session info: chrome=110.0.5481.104)
Stacktrace:
Backtrace:
(No symbol) [0x002B6643]
(No symbol) [0x0024BE21]
(No symbol) [0x0014DA9D]
(No symbol) [0x001813F3]
(No symbol) [0x0018147B]
(No symbol) [0x001B8DFC]
(No symbol) [0x0019FDC4]
(No symbol) [0x001B6B09]
(No symbol) [0x0019FB76]
(No symbol) [0x001749C1]
(No symbol) [0x00175E5D]
GetHandleVerifier [0x0052A142+2497106]
GetHandleVerifier [0x005585D3+2686691]
GetHandleVerifier [0x0055BB9C+2700460]
GetHandleVerifier [0x00363B10+635936]
(No symbol) [0x00254A1F]
(No symbol) [0x0025A418]
(No symbol) [0x0025A505]
(No symbol) [0x0026508B]
BaseThreadInitThunk [0x762600F9+25]
RtlGetAppContainerNamedObjectPath [0x77167BBE+286]
RtlGetAppContainerNamedObjectPath [0x77167B8E+238]
(No symbol) [0x00000000]
this is the code and it doesn't run. And i think it's because the body tag of the html code is closed, I need to open it with selenium. I actually can open it if i do right click in when i inspect the code manually. Then I click in "expand recursively", then the code runs well, but i don't know how to do it automatically with python.

The I accept element is a dynamic element, so to click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategy:
Using XPATH:
driver.get('https://tinder.com/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button//div[text()='I accept']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to properly use loop in selenium using chrome webdriver in Python?

I have a site that has multiple input fields containing URLs with the same value (X) that I want to replace with (Y). Basically, I want to automate replacing URL X with URL Y for all occurrences of URL X. There is an edit and save button to the right of each of these input fields. And each time you click save an alert pops up for each field you update. This is in short, the entire workflow I need to automate.
I'm at the point in my code that I can do it successfully for one field, but when I use a for loop it just updates one element/field twice and not the rest. FYI - I limited my for loop to two iterations using itertools.islice intentionally since I want to limit my testing to a small subset before I do a full run.
Below is a screenshot of what the page looks like:
HTML
New error:
C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py:21: DeprecationWarning: use options instead of chrome_options chrome_browser = webdriver.Chrome(service=service, chrome_options=chrome_options)
DevTools listening on ws://127.0.0.1:61787/devtools/browser/155908c1-8be2-40a1-85bf-27c44bf25506 Traceback (most recent call last): File "C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py", line 90, in <module>
auto_ftp_url() File "C:\Users\me\OneDrive\Documents\Python\Web page entry automation\preferences_v3 copy.py", line 57, in auto_ftp_url
number = elem.get_attribute("id").split("_")[-1]
^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webelement.py", line 177, in get_attribute
attribute_value = self.parent.execute_script(
^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webdriver.py", line 500, in execute_script
return self.execute(command, {"script": script, "args": converted_args})["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\webdriver.py", line 440, in execute
self.error_handler.check_response(response) File "C:\Users\me\AppData\Roaming\Python\Python311\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: chrome=109.0.5414.120) Stacktrace: Backtrace:
(No symbol) [0x00506643]
(No symbol) [0x0049BE21]
(No symbol) [0x0039DA9D]
(No symbol) [0x003A09E4]
(No symbol) [0x003A08AD]
(No symbol) [0x003A12E5]
(No symbol) [0x004079F7]
(No symbol) [0x003EFD7C]
(No symbol) [0x00406B09]
(No symbol) [0x003EFB76]
(No symbol) [0x003C49C1]
(No symbol) [0x003C5E5D]
GetHandleVerifier [0x0077A142+2497106]
GetHandleVerifier [0x007A85D3+2686691]
GetHandleVerifier [0x007ABB9C+2700460]
GetHandleVerifier [0x005B3B10+635936]
(No symbol) [0x004A4A1F]
(No symbol) [0x004AA418]
(No symbol) [0x004AA505]
(No symbol) [0x004B508B]
BaseThreadInitThunk [0x771300F9+25]
RtlGetAppContainerNamedObjectPath [0x77847BBE+286]
RtlGetAppContainerNamedObjectPath [0x77847B8E+238]
PS C:\Users\me\OneDrive\Documents\Python\Web page entry automation> [15148:25476:0201/123239.502:ERROR:device_event_log_impl.cc(215)] [12:32:39.502] USB: usb_service_win.cc:415 Could not read device interface GUIDs: The system cannot find the file specified. (0x2) [15148:25476:0201/123239.557:ERROR:device_event_log_impl.cc(215)] [12:32:39.557] USB: usb_device_handle_win.cc:1046 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F) [15148:25476:0201/123239.575:ERROR:device_event_log_impl.cc(215)] [12:32:39.575] USB: usb_device_handle_win.cc:1046 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
Below is my code:
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.alert import Alert
import itertools
def auto_ftp_url():
username, password = login()
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
service = Service(
r"C:\Users\me\OneDrive\Documents\chromedriver_win32\chromedriver.exe"
)
chrome_browser = webdriver.Chrome(service=service, chrome_options=chrome_options)
chrome_browser.get("https://somelink")
# send text to username field
userElem = chrome_browser.find_element(by="id", value="Username")
userElem.click()
userElem.send_keys(f"{username}")
# send password to password field
passwordElem = chrome_browser.find_element(by="id", value="Password")
passwordElem.click()
passwordElem.send_keys(f"{password}")
# click login
loginElem = chrome_browser.find_element(by="id", value="Button1")
loginElem.click()
# add some delay (2 seconds) to the webdriver so that the elements will load
chrome_browser.implicitly_wait(2)
# click on Admin button on left navigation pane
adminElem = chrome_browser.find_element(By.XPATH, "//*[#id='btnAdminPage']")
adminElem.click()
# click on Data FTP button in the top center
data_ftp_Elem = chrome_browser.find_element(
By.XPATH, "//*[#id='menu-outer']/div[12]/a/div"
)
data_ftp_Elem.click()
# find FTP URL with a value of 'aftersoftna.com'
ftp_url_Elem = chrome_browser.find_elements(
By.XPATH, "//*[#value='aftersoftna.com']"
)
for elem in itertools.islice(ftp_url_Elem, 2):
if elem: # if the value is 'aftersoftna.com' then do the following
# click edit button
editElem = chrome_browser.find_element(
By.XPATH, "//*[#id='ContentPlaceHolder1_gvFTP_btnSave_7']"
)
editElem.click()
# use WebDriverWait to wait for the element to be clickable
textElem = WebDriverWait(chrome_browser, 20).until(
EC.element_to_be_clickable(
(By.XPATH, "//*[#id='ContentPlaceHolder1_gvFTP_txtFtpUrl_7']")
)
)
textElem.click()
textElem.clear()
textElem.send_keys(
"catalog.com"
) # after clearing the old URL, enter the new URL
# click save
saveElem = chrome_browser.find_element(
By.XPATH, "//*[#id='ContentPlaceHolder1_gvFTP_btnSave_7']"
)
saveElem.click()
# popup or alert message comes up confirming changes saved. click ok
WebDriverWait(chrome_browser, 10).until(EC.alert_is_present())
chrome_browser.switch_to.alert.accept()
def login():
username = input("Username: ")
password = input("Password: ")
return username, password
auto_ftp_url()

Without seeing the page it's impossible to know but I'm going to guess that it's because you are using the same locators for editElem, textElem, and saveElem for each URL. Each of them end in "_7" but I would assume that there are multiple elements on the page, one set of three for each URL from what you are describing, e.g. "_8", "_9", etc. If this is true then your locators need to be relative to the URL that you want to edit.
If I were to guess even further, you should be able to get your list of ftp_url_Elem elements, grab the ID from each in the loop, and then extract the final number, e.g. 7 from ContentPlaceHolder1_gvFTP_txtFtpUrl_7, and then extrapolate that to the other ID, ContentPlaceHolder1_gvFTP_btnSave_7.
To get the magic number that corresponds to the row number, we can
Grab the first ID using elem.get_attribute("id"), ContentPlaceHolder1_gvFTP_txtFtpUrl_7
Split that ID by "_" resulting in ['ContentPlaceHolder1','gvFTP','txtFtpUrl','7']
Grab the last instance from that list which is the desired row number, '7'
Now we can append this number to the "prefix" of the ID for the other elements to get the correct element on the matching row of the table, e.g. ID prefix for the Edit/Save button, ContentPlaceHolder1_gvFTP_btnSave_, and add '7' to get ContentPlaceHolder1_gvFTP_btnSave_7
Then you just repeat that last step for all the elements on that row that you need to click.
Here's all the code as an example
for elem in itertools.islice(ftp_url_Elem, 2):
if elem: # if the value is 'aftersoftna.com' then do the following
# get row number from element ID
row_number = elem.get_attribute("id").split("_")[-1]
# click edit button
editElem = chrome_browser.find_element(
By.XPATH, "//*[#id='ContentPlaceHolder1_gvFTP_btnSave_" + row_number + "']"
)
... and so on
If this is enough to point you in the right direction to fix it yourself, great. If not, please post a few example rows of HTML that contain the edit, URL field, and save button so we can create code and relative locators.

Selenium open chrome default profile but get an error WebDriverException

I'm trying to open chrome with default browser by Selenium. Once the chrome is opened, the cell output shows an error:
"WebDriverException"
"Message: unknown error: Chrome failed to start: exited normally.\n (unknown error: DevToolsActivePort file doesn't exist)\n (The process started from chrome location C:\\\\Program Files\\\\Google\\\\Chrome\\\\Application\\\\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\nStacktrace:\nBacktrace:\n\t(No symbol) [0x00B0F243]\n\t(No symbol) [0x00A97FD1]\n\t(No symbol) [0x0098D04D]\n\t(No symbol) [0x009AC24E]\n\t(No symbol) [0x009A82E9]\n\t(No symbol) [0x009DF056]\n\t(No symbol) [0x009DEB2A]\n\t(No symbol) [0x009D8386]\n\t(No symbol) [0x009B163C]\n\t(No symbol) [0x009B269D]\n\tGetHandleVerifier [0x00DA9A22+2655074]\n\tGetHandleVerifier [0x00D9CA24+2601828]\n\tGetHandleVerifier [0x00BB8C0A+619850]\n\tGetHandleVerifier [0x00BB7830+614768]\n\t(No symbol) [0x00AA05FC]\n\t(No symbol) [0x00AA5968]\n\t(No symbol) [0x00AA5A55]\n\t(No symbol) [0x00AB051B]\n\tBaseThreadInitThunk [0x7579FEF9+25]\n\tRtlGetAppContainerNamedObjectPath [0x77297BBE+286]\n\tRtlGetAppContainerNamedObjectPath [0x77297B8E+238]\n"
I also run without profile argument, it works fine. But when profile argument is added, the error occurs.
Chrome version: 108.0.5359.125
Python version: 3.9.12
Selenium version: 4.7.2
Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=C:\\Users\\Rattapong.Pojpatin\\AppData\\Local\\Google\\Chrome\\User Data\\") # change to profile path
chrome_options.add_argument('--profile-directory=Profile 2')
browser = webdriver.Chrome(executable_path="D:\\Chromedriver\\chromedriver.exe", chrome_options=chrome_options)
browser.get('https://sellercenter.lazada.co.th/apps/seller/login')

selenium don't push button on site

i'm trying to make a program that as the first step must login on site edostavka.by having phone and password. If I use "https://edostavka.by/login?screen=withPassword" - program still redirected on "https://edostavka.by/login?screen=phone". Luckily there is button "Войти с паролем", which should open "https://edostavka.by/login?screen=withPassword" page. But selenium can't find it.
My code here:
from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from fake_useragent import UserAgent as ua
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
ua = ua(browsers='chrome')
driver_service = Service(executable_path=r'C:\Users\37533\Desktop\dostavka\dostavka\chromedriver\chromedriver.exe')
options = webdriver.ChromeOptions()
options.add_argument(f'user-agent={ua}')
driver = webdriver.Chrome(service=driver_service)
driver.maximize_window()
#url = 'https://edostavka.by/login?screen=withPassword'
url = 'https://edostavka.by/login?screen=phone'
try:
driver.get(url=url)
time.sleep(3)
driver.find_element(By.NAME, 'phone').send_keys('132456789')
driver.find_element(By.LINK_TEXT, 'Войти с паролем').send_keys(Keys.ENTER)
time.sleep(5)
except Exception as e:
print(e)
finally:
driver.close()
driver.quit()
It was no matter how i tried to find that button(through XPATH, CLASS_NAME, or LINK_TEXT) - none of this worked. I tried put WebDriverWait and EC instead of time.sleep, but it didn't help aswell as swapping "click" methods(I tried 'click()', 'submit()', 'send_keys(Keys.ENTER or \n).
Also, I figure that problem might be in iframe or scripts. But for iframe - there are 4 on the page and all of them seems to be empty. For scripts - I still didn't find answer: what and should I do?
Had to mention that it can find and even fill line with "phone number"
Every time i run the code i get that message
Message: no such element: Unable to locate element: {"method":"link text","selector":"Войти с паролем"}
(Session info: chrome=108.0.5359.125)
Stacktrace:
Backtrace:
(No symbol) [0x00FEF243]
(No symbol) [0x00F77FD1]
(No symbol) [0x00E6D04D]
(No symbol) [0x00E9C0B0]
(No symbol) [0x00E9C22B]
(No symbol) [0x00ECE612]
(No symbol) [0x00EB85D4]
(No symbol) [0x00ECC9EB]
(No symbol) [0x00EB8386]
(No symbol) [0x00E9163C]
(No symbol) [0x00E9269D]
GetHandleVerifier [0x01289A22+2655074]
GetHandleVerifier [0x0127CA24+2601828]
GetHandleVerifier [0x01098C0A+619850]
GetHandleVerifier [0x01097830+614768]
(No symbol) [0x00F805FC]
(No symbol) [0x00F85968]
(No symbol) [0x00F85A55]
(No symbol) [0x00F9051B]
BaseThreadInitThunk [0x771800F9+25]
RtlGetAppContainerNamedObjectPath [0x779C7BBE+286]
RtlGetAppContainerNamedObjectPath [0x779C7B8E+238]
Maybe my question easy and newbie, but i tried to fix this problem over six-seven hours and still didn't find the way of solving

First of read the selenium documentation. Then use Chrome Developer tools (you may call by pressing F12 or Ctrl+Shift+I) to inspect target page elements.
Below I provide an example solution for your case:
# accept coockies
driver.find_element(By.XPATH, "//button[#class='button button_size_medium button_type_subtle accept-cookies_accept__mok-Y']").click()
time.sleep(2)
# change login mode to with password
driver.find_element(By.XPATH, "//button[#class='button button_size_large button_type_subtle phone_button__2Z9fs phone_button__padding__e0VFd']").click()
time.sleep(2)
# type phone number
driver.find_element(By.XPATH, "//input[#class='index-module_wrapperPhone__input__KD5gF']").send_keys('132456789')
time.sleep(2)
# type password
driver.find_element(By.XPATH, "//input[#class='input__input input__input_size_medium']").send_keys('Password')
time.sleep(2)
# click to eye icon
driver.find_element(By.XPATH, "//button[#class='withPassword_login__form_password_button__3Zrhk']").click()
time.sleep(2)
This is how to use XPath

Can't run Chrome in headless mode using Selenium

So here's my code first:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
from fake_useragent import UserAgent
import random
ua = UserAgent()
options = Options()
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--blink-settings=imagesEnabled=false')
chrome_options.add_argument('--headless')
chrome_options.add_argument(f'user-agent={ua.random}')
driver = webdriver.Chrome(options=options, chrome_options=chrome_options)
driver.maximize_window()
url = "https://magiceden.io/marketplace/hasuki"
driver.get(url)
element = driver.find_element(By.CSS_SELECTOR, "#content > div.tw-w-full.tw-py-0.sm\:tw-mt-0 > div.tw-flex.tw-relative > div.tw-flex-auto.tw-max-w-full.tw-pt-0 > div.tw-flex.tw-items-center.md\:tw-justify-between.tw-gap-2.md\:tw-gap-4.md\:tw-sticky.tw-top-\[133px\].tw-bg-gray-100.tw-z-10.tw-flex-wrap.tw-p-5 > div.tw-flex.tw-flex-grow.tw-justify-center.tw-gap-x-2 > button > span:nth-child(4)")
print(f"The current instant sell price is {element.text}")
When I run it, I get weird long error, that ends with:
Backtrace:
(No symbol) [0x00806643]
(No symbol) [0x0079BE21]
(No symbol) [0x0069DA9D]
(No symbol) [0x006D1342]
(No symbol) [0x006D147B]
(No symbol) [0x00708DC2]
(No symbol) [0x006EFDC4]
(No symbol) [0x00706B09]
(No symbol) [0x006EFB76]
(No symbol) [0x006C49C1]
(No symbol) [0x006C5E5D]
GetHandleVerifier [0x00A7A142+2497106]
GetHandleVerifier [0x00AA85D3+2686691]
GetHandleVerifier [0x00AABB9C+2700460]
GetHandleVerifier [0x008B3B10+635936]
(No symbol) [0x007A4A1F]
(No symbol) [0x007AA418]
(No symbol) [0x007AA505]
(No symbol) [0x007B508B]
BaseThreadInitThunk [0x75EB00F9+25]
RtlGetAppContainerNamedObjectPath [0x77A27BBE+286]
RtlGetAppContainerNamedObjectPath [0x77A27B8E+238]
BUT if I comment out "chrome_options.add_argument('--headless')", my code works perfectly fine. What's the issue here? I suppose the problem is that website doesn't let me use headless mode, how can I solve this?
I want my program to run in headless mode, but I get restricted either by the website or chrome browser.

Basically, you are seeing a NoSuchElementException
Full stacktrace:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#content > div.tw-w-full.tw-py-0.sm\:tw-mt-0 > div.tw-flex.tw-relative > div.tw-flex-auto.tw-max-w-full.tw-pt-0 > div.tw-flex.tw-items-center.md\:tw-justify-between.tw-gap-2.md\:tw-gap-4.md\:tw-sticky.tw-top-\[133px\].tw-bg-gray-100.tw-z-10.tw-flex-wrap.tw-p-5 > div.tw-flex.tw-flex-grow.tw-justify-center.tw-gap-x-2 > button > span:nth-child(4)"}
(Session info: headless chrome=109.0.5414.75)
Stacktrace:
Backtrace:
(No symbol) [0x00E04AD3]
(No symbol) [0x00D99211]
(No symbol) [0x00C8D9CD]
(No symbol) [0x00CC1102]
(No symbol) [0x00CC123B]
(No symbol) [0x00CF8A72]
(No symbol) [0x00CDFB84]
(No symbol) [0x00CF67B9]
(No symbol) [0x00CDF936]
(No symbol) [0x00CB4788]
(No symbol) [0x00CB5C1D]
GetHandleVerifier [0x01079342+2502274]
GetHandleVerifier [0x010A7959+2692249]
GetHandleVerifier [0x010AABDC+2705180]
GetHandleVerifier [0x00EB3480+643008]
(No symbol) [0x00DA208F]
(No symbol) [0x00DA7AB8]
(No symbol) [0x00DA7BA5]
(No symbol) [0x00DB273B]
BaseThreadInitThunk [0x76D9FA29+25]
RtlGetAppContainerNamedObjectPath [0x77B37A4E+286]
RtlGetAppContainerNamedObjectPath [0x77B37A1E+238]
Solution
The desired element is a dynamic element, so ideally to print the desired text from the element you need to induce WebDriverWait for the visibility_of_element_located() and you can use the following Locator Strategy:
Using CSS_SELECTOR:
driver.get("https://magiceden.io/marketplace/hasuki")
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#content > div.tw-w-full.tw-py-0.sm\:tw-mt-0 > div.tw-flex.tw-relative > div.tw-flex-auto.tw-max-w-full.tw-pt-0 > div.tw-flex.tw-items-center.md\:tw-justify-between.tw-gap-2.md\:tw-gap-4.md\:tw-sticky.tw-top-\[133px\].tw-bg-gray-100.tw-z-10.tw-flex-wrap.tw-p-5 > div.tw-flex.tw-flex-grow.tw-justify-center.tw-gap-x-2 > button > span:nth-child(4)"))).text)
driver.quit()
Console Output:
0.3664
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC