I'm developing a data scraping script using Selenium and Python and I've got the following line of code which works on my Ubuntu PC, my wife's Windows PC but not my colleague's Windows PC.
ActionChains(driver).context_click(inputElement).send_keys(Keys.ARROW_DOWN).click().perform()
We're in the UK and he is in Canada is the only difference I know of.
EDIT
What I've realised is happening, but I do not know how to resolve, is the Keys.ARROW_DOWN is being performed on the webpage and not on the context_click element. Any thoughts? Is this a Chrome issue?
That might be a timing issue related to the quality of the network connection to the target site.
I'd strengthen it by introducing an Explicit Wait for the inputElement, use the WebDriverWait with element_to_be_clickable Expected Condition before starting the action chain.
You may also add a "scroll into view" of the element to tackle potential browser/resolution/etc specific layout issues:
driver.execute_script("arguments[0].scrollIntoView();", inputElement)
Solution:
I rewrote using Firefox instead of Chrome and it is working fine.
Related
I am using python to interact with webpages via selenium. I am using the geckodriver Firefox driver (not interested in Chrome solutions).
I want to zoom the browser window, equivalent to pressing ctrl+- (or cmd+- on Mac OS).
Granted, there are lots of questions on this topic. I have tried to do my due diligence in digging through them before posting here. The following are solutions that do not work for me (let driver be a WebDriver instance running with geckodriver):
No effect:
driver.execute_script("document.body.style.zoom='zoom 90%'")
from selenium.webdriver.common.keys import Keys
body = driver.find_element_by_tag_name('body')
body.send_keys(Keys.COMMAND, Keys.SUBTRACT)
from selenium.webdriver.common.action_chains import ActionChains
actor = ActionChains(driver)
actor.key_down(Keys.COMMAND).key_down(Keys.SUBTRACT).perform()
Wrong effect:
driver.execute_script("document.body.style.transform='scale(0.9)'")
driver.execute_script("document.body.style.MozTransform='scale(0.9)'")
Error:
driver.set_context("chrome")
win = driver.find_element_by_tag_name("window")
win.send_keys(Keys.CONTROL + "-")
driver.set_context("content")
(raises NoSuchElementException)
Related questions:
stackoverflow.com/questions/28111539
stackoverflow.com/questions/60941346
stackoverflow.com/questions/67699434
stackoverflow.com/questions/56193465
stackoverflow.com/questions/32135085
...and many others.
The only way I have found that actually works is to use pyautogui, however I would rather not install additional 3rd-party libraries, & I don't believe I'll be able to run this in a headless state without a lot of hassle.
However, it seems there should be some way to do this using Selenium/geckodriver's native functionality, rather than installing additional 3rd-party packages for this purpose.
Is there a way??? The abundance of questions on this topic makes it clear that I am not the only one to have had trouble with this...
EDIT: Testing on Python 3.9, Selenium 3.141.0, geckodriver 0.29.1, Firefox 90.0, Mac OS
In Python 3.7.7, selenium 3.141.0, geckodriver 0.29.1, firefox 90.0, Ubuntu 20.04.2 LTS, I combined 2 of your items and made it work.
I could not find the window tag either, but I found body. After I got the url (and only after, otherwise I got an error), I had to do the set context to "chrome." I don't know why, but if not, as you observed, nothing happened. Then I was able to send control + or control - with the desired effect (I took two + to really see the difference, but the 110% and then 120% also shows).
driver.get(URL)#URL is a constant containing the url
driver.set_context("chrome")#needed to make keys work, must come after there is a context
win = driver.find_element_by_tag_name("body")#there is a body even if not a window
win.send_keys(Keys.CONTROL, "+")#zoom to 110%
win.send_keys(Keys.CONTROL, "+")#zoom to 120%
I am using helium library for web scraping dynamic websites
I found it much faster to use tabs than using many windows yet some websites when I open them in a new tab they show some ads and I could not find a way to close them. That made me look for a lot of JS codes, which I do not understand, and most of the times they are not working when executed using helium library in python even though they work fine in the Console of Google Chrome.
However, the code should be running headless and I am doing all of this only for testing and here is my main question
Does it matter if used new tab or new window if I am going to use headless browser in python helium
is it going to be faster just like what happens when it runs normal or since it is headless it makes no difference ?
not to forget to mention that it is way easier for me to work with many windows since I won't have to use JS codes.
driver.execute_script('''$x('xpath')[0].remove();''')
driver.execute_script('''$x('xpath')[0].click();''')
$x('xpath')[0].remove();
$x('xpath')[0].click();
selenium.common.exceptions.JavascriptException: Message: javascript error: $x is not defined
they both work just fine in Google Chrome Console but for some reason, I can not execute them python helium.
I tried using to give more time using time.sleep(5) to load the page but it did not work
not to mention that I tried using the click() method from helium it gives me a LookupError()
I found a way to test it for myself it might not be accurate but the results gave me a clear winner I used the windows task manager to see the percentage of CPU usage in all the cases when running headless tabs were faster and used way less CPU than new windows
I am automating some task with selenium webdriver for python.
I am curently on a selection by classname that is behaving strangely (only 4 of the 10 elements are found), I believe that is because the elements I am looking for are in a flexbox. Could that be it?
If so, I saw to handle flexbox one should use a add on on selenium, I saw this one https://code.google.com/archive/p/sfapi/downloads, buts it's for Java. What would you advice for python?
Thanks
I've been working with selenium in Python recently.
I was curious if anyone has had experience with recording an instance of a headless browser? I tried finding a way to do this, but didn't find any solutions in Python - a code example would be excellent.
Some tips would be helpful.
I don't think they have any built in way to do this with any of the browsers. Your best bet would be to connected to the same instance of the browser (this is easier if you use the grid server) from another program then just take screenshots at short intervals.
So I have a website which has a combo that I need to select an item from, now the problem is it's a bit untraditional and doesn't have option's as elements but instead it has divs.
So I need my program to click combo box then wait (the best way to do this I found is via implicitly_wait(3)# 3 seconds) and then click the box element I need. Firefox is doing a great job with it but PhantomJS seem to throw:
Selenium.common.exceptions.ElementNotVisibleException:
Message: 'Error Message => \'Element is not currently visible and may not be manipulated\'
I'm not sure what's the cause of it, but I suspect that PhantomJS fails to correctly wait via implicitly_wait for some reason and tries to select non-visible element.
Any idea how to approach this without forced thread sleep?
Yup, your issue sounds exactly something I've just fixed in a UI test that was starting to anoy me. Quite a complex one, that passed on all browsers, except my favorite PhantomJs (which is the fastest).
It was quite anoying, when in the debugger I could clearly see that even the parent element was set to visible. Only the prime faces component I needed to click for whatever reason (not CSS or active Styles) was not visible.
After looking at Phantom JS screenshot, I realized the window was quite tiny, and indeed the UI element was not visually visible.
You see now where I am geting at?
Phantom JS is a headless browser, it does not render you the window, but it does use a window, and the window does have a size.
The exception is quite unexpected, because the engine behind is the same as the one used for chrome and safari ... things should just work.
Well, and they did in the end.
Try to tune your driver factory like this:
WebDriver driver = new PhantomJSDriver();
driver.manage().window().setSize(new Dimension(800, 600));
return driver;
Good luck.