We Keep Coding

Python Selenium, edge browser, I dont see Inspect Element on every element

I am trying to get data from a Power Bi table. There are some elements that appear when hovering over a table. When I right click on ... I don't see Inspect Element. However, when I left click on this element, I can see a menu, and if I right click on any items, I can see Inspect element.
My first question, is why I don't see Inspect Element in the right click menu for all elements in the browser. Am I somehow able to open this ... menu programmatically in Selenium?
the Export Data element only appears in HTML after the first left click. I'm assuming this is created using Javascript and in order to export data with Selenium I would have to programmatically instantiate this by clicking on the ... menu. Is selenium capable of triggering javascript functions that generate more html code in a dynamic webpage? Or do I need to somehow click on the ... element.
If I can execute a javascript function, how can I find out in Edge the javascript function that gets executed and how can I replicate this function in Selenium
Essentially, if I try to find the Export data element in Selenium, it is not able to find it, unless I set a breakpoint before search, then in EdgeDriver I open this menu, and then I can find it and click it through Python
If all else fails, can I programmatically open the left click menu by automating a mouse click at certain coordinates in Selenium?

1.1 why I don't see Inspect Element in the right click menu for all elements:
PowerBi has its own context menu so they suppress the browsers context menu. If the element is tricky to find the dev tools, you can press Ctrl + Shift + C (while dev tools is open) and then click the desired element. Your mouse needs to be already over the element before pressing the key combination.
1.2 Am I somehow able to open this ... menu programmatically in Selenium?
Seems a little tricky, but could work if you first find the title of that area and move the mouse there, like described here: https://stackoverflow.com/a/8261754/12914172
Then your element should be in the html and you can find it hopefully by its class name vcMenuBtn that seems to be unique on that page. But you need to verify that.
2. Is selenium capable of triggering javascript functions that generate more html code in a dynamic webpage? Or do I need to somehow click on the ... element.
Selenium is able to execute javascript like desribed here: https://stackoverflow.com/a/70544802/12914172
However in your sample, and I was quickly checking the PowerBI online page, this looks like a whole lot of reverse engineering to understand and can sometimes be dangerous as well. I would go for hoover over the area find the ... and click it.
3. How can I find out in Edge the javascript function that gets executed
In dev tools you can set breakpoints to debug the steps the pages does after an action. But again, I would not invest to much time in that.
4. Can I programmatically open the left click menu by automating a mouse click at certain coordinates in Selenium?
Yes but this never works as good as the way described above. If you still want to give it a try, maybe that answer helps: https://stackoverflow.com/a/26385456/12914172

Many thanks to r000bin, this solution works for me, downloading data from PowerBI using Selenium for Python:
import selenium, mouse, time
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
url = 'https://dataport.gasunie.nl/Vulling-Gasopslagen-Nederland'
driver = selenium.webdriver.Chrome(service=Service())
driver.get(url)
time.sleep(4)
#driver.fullscreen_window()
#driver.switch_to.window(driver.current_window_handle)
time.sleep(4)
iframe = driver.find_elements(By.TAG_NAME, 'iframe')
assert len(iframe)==1
driver.switch_to.frame(iframe[0])
time.sleep(4)
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver)
actions.move_to_element_with_offset(driver.find_element(By.TAG_NAME, 'html'), 0,0)
actions.move_by_offset('5', '5').click().perform()
time.sleep(4)
button = driver.find_element(By.CLASS_NAME, 'vcMenuBtn')
button.click()
button = driver.find_element(By.ID, '0')
button.click()
# 4 tabs and 1 enter
time.sleep(4)
for n in range(4):
element = driver.switch_to.active_element
time.sleep(2)
element.send_keys(Keys.TAB)
time.sleep(2)
element = driver.switch_to.active_element
time.sleep(2)
element.send_keys(Keys.ENTER)
driver.close()

RobotFramework - Unable to click element using selenium library but Execute Javascript click works

I am using RobotFramework to automate one application. I am using the selenium library. For the whole application, selenium keywords "Click Button" and "Click Element" throw an error stating "ElementClickInterceptedException: Message: element click intercepted: Element **** is not clickable at point (376, 289). Other element would receive the click: ..."
I am able to identify the element using ID and it is not under any iframe or shadow-root element. but still I am not able to click on the element. I also tried with adding wait commands to see if it is sync issue but it is not. I tried to click using Action class, mouse move and click etc but did not work.
I tried to take the screenshot of the element using "Capture Element Screenshot" and it captures the screenshot of an empty place however, when I try to locate element in the browser dev tool, it locate it exactly
Only working solution I found is to run "Execute Javascript" keyword to click on the element such as
Execute Javascript $('#id').click();
Question:
Though I am able to make it work, I am curious to know what could be the issue in the application. I am not able to share the application dom code due to restriction. Sorry for that

This means that the element you trying to click is
Out of the visible screen (view port) so you need to scroll the page to make that element accessible or
It is covered by some other element - for example you should open a drop down menu etc or
You trying to click the element while it is still not fully rendered - in this case you need to add some delay to make the element fully rendered and be ready to accept clicks.
Selenium generally imitates human GUI actions. So, as a human user you can't click element inside drop-down without opening it. And you can't click element out of the visible screen. This is why Selenium .click() methods can't click such elements.
JavaScript click is more powerful tool, it can click invisible, covered etc. elements. It doesn't imitate human GUI actions.

Finding XPath without using inspecting element?

I am making an automation for download data from a weather institution.
The issue is that in my effort to make it more independent I am trying to make Selenium to Tab keys to a certain spot, so the Browser focus can "Walk" to the download button. When I call the click() function it doesn't do anything. So I tried to Extract the XPath with the function get_attribute("xpath") but it returns None. How I can extract the XPath?
I am going to paste the issue down here:
Bandera=driver.find_element_by_xpath('/html/body/div[2]/div[2]/div/div[3]/div/div/div/div[1]/div/div/div[2]/div/table/tbody/tr[1]/td[1]/div/input')
Bandera.click()
Bandera.click()
## So Here i just select and dis-select a checkbox just to be near the Download button.
actions = ActionChains(driver)
actions.send_keys(Keys.TAB * 1 )
actions.perform()
#Here i just tabed to the button
Accion=driver.switch_to.active_element
#Maybe, here is when i lost the focus of the button?
Descarga_Actual=Accion.get_attribute("xpath")
Thank you and sorry to borrow your time.

To make a click on hover I would use the following sequence:
your_dropdown_locator.click()
dropdown_option = driver.find_element_by_xpath("dropdown option locator")
actions = ActionChains(driver)
actions.move_to_element(dropdown_option)
actions.click().perform()
But, this is the approach that is usually used for dropdowns.
You use: driver.find_element_by_xpath('/html/body/div[2]/div[2]/div/div[3]/div/div/div/div[1]/div/div/div[2]/div/table/tbody/tr[1]/td[1]/div/input') This is the main problem of your code.
If you pasted html code of this dropdown (or button you need to click), people would help you to find a better locator. XPath/CSS must be unique. In your case the locator is very bad.
Also, I see no sense making Bandera.click() two times.
In your case, as I understand, you just need to click the button. So the locator is your main problem.
You need to find the correct locator, to wait till the button is clickable and then to click it.
Another problem in what you are trying to do:
get_attribute("xpath") looks like incorrect expectation of how get_attribute function works. Check at least here what this function means Python Selenium: Find object attributes using xpath

Unable to locate/click pop-up button with Selenium in Python

I'm using Selenium in Python 3 to access webpages, and I want to click on a pop-up button, but I am unable to locate it with Selenium.
What I'm describing below applies to a number of sites with a pop-up, so I'll use a simple example.
url = "https://www.google.co.uk"
from selenium import webdriver
driver = webdriver.Firefox()
driver.implicitly_wait(10)
driver.get(url)
The page has a pop-up for agreeing to cookies.
I want the script to click on the "I agree" button, but I'm unable to locate it.
I've found a few questions and posts about this online (including on Stackoverflow), but all the suggestions I found seem to fall in one of the following categories and don't seem to work for me.
Wait longer for the pop-up to actually load.
I've tried adding delays, and in fact, I'm testing this interactively, so I can wait all I want for the page to load before I try to locate the button, but it doesn't make any difference.
Use something like driver.switch_to.alert
I get a NoAlertPresentException. The pop-up doesn't seem to be an alert.
Locate the element using driver.find_element.
This doesn't work either, regardless of which approach I use (xpath, class name, text etc.). I can find elements from the page under the pop-up, but nothing from the pop-up itself. For example,
# Elements in main page (under pop-up)
driver.find_element_by_partial_link_text("Sign in") # returns FirefoxWebElement
driver.find_element_by_class_name("gb_g") # returns FirefoxWebElement
# Elements on the pop-up
driver.find_element_by_partial_link_text("I agree") # NoSuchElementException
driver.find_element_by_class_name("RveJvd snByac") # NoSuchElementException
The popup just doesn't seem to be there in the page source. In fact, if I try looking at the loaded page source from the browser, I can't find anything related to the pop-up. I understand that many sites use client-side scripts to load elements dynamically, so many elements wouldn't show up in the raw source, but that was the point of using Selenium: to load the page, interpret the scripts and access the end result.
So, what am I doing wrong? Where is the pop-up coming from, and how can I access it?

selenium.common.exceptions.ElementNotVisibleException: Message: element not interactable and explicit wait not working with Selenium and Python

I am trying to access PenFed in order to get my current outstanding amount. I have done quite a bit of research and unfortunately, I am still stumped. I am using Python Selenium and I am trying to click on the initial login button on the side in order to see the username field. This is the element's HTML code:
Login
When I try to run the following code:
driver.find_element_by_id("mobile-login").click()
I get the following error:
selenium.common.exceptions.ElementNotVisibleException: Message: element not interactable
Even when I try to use WebDriver Wait functions such as these:
try:
WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.ID, "mobile-login"))).click()
except ElementNotVisibleException:
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.ID, "mobile-login"))).click()
No matter how long I make them wait, I get a timeout message:
raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:
All of my research says that invoking a wait function should fix it but it doesn't work for me. I also read that there might be an image overlay on top of the element that I would have to invoke before clicking on the button, but I didn't see anything in the website code as well. If I am testing it out, the only way that I am able to click on the button through code is if I physically click on it first, so I am unaware of anything else I can use. Thank you in advance for the help!
UPDATE: I have discovered that the following code works for me:
element = driver.find_element_by_id("mobile-login")
driver.execute_script("$(arguments[0]).click();", element)
But I do not know what the execute_script actually does. Can someone explain that piece of code works or if any other alternatives work for them?

The code you specified is JQuery. execute_script(p1, p2) runs a js script, where p1 is the script (in your case a JQuery line that clicks the element) and p2 is the desired element. It seems like you shouldn't need p2 if arguments[0] is equal to "element," but I'm not totally sure.
One potential fix is to use a counter for the number of times you clicked the element. If the counter reaches a certain number and the page doesn't change (you can check by finding a unique element/value on your current page), then you know it's not clickable.
Good luck!

The desired element is a dynamic element so to locate the element you have to induce WebDriverWait for the element to be clickable and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument('start-maximized')
options.add_argument('--disable-extensions')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.penfed.org/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.pfui-button.login-slide-button.pfui-button-login.dtm-global-nav[data-id='Open Log In Drawer']"))).click()
Browser Snapshot:

The link you are attempting to click is for the mobile site and is not visible if you are viewing the site at desktop resolutions. If you shrink your browser down until it changes layout, you will see the LOGIN button appear that corresponds to that link. That's why you are getting the ElementNotVisibleException.
To your second question, the reason that using .execute_script() works is that it executes JS directly and can click on anything... hidden or not. Selenium was designed to interact with the page as a user would so it won't let you click invisible elements, etc.
If you are intending for your script to act like a user, you will want to avoid using .execute_script() because it allows you to do things on the page that a user cannot do.
If you want to log in like a desktop user would, you need to click the "LOG IN" button using the CSS selector below
button[data-id='Open Log In Drawer']
That will open a side panel where you can enter your username, etc. and log in. FYI... you will likely need a wait to give the panel a chance to open before continuing the log in process.