I've been working with selenium in Python recently.
I was curious if anyone has had experience with recording an instance of a headless browser? I tried finding a way to do this, but didn't find any solutions in Python - a code example would be excellent.
Some tips would be helpful.
I don't think they have any built in way to do this with any of the browsers. Your best bet would be to connected to the same instance of the browser (this is easier if you use the grid server) from another program then just take screenshots at short intervals.
Related
I am using helium library for web scraping dynamic websites
I found it much faster to use tabs than using many windows yet some websites when I open them in a new tab they show some ads and I could not find a way to close them. That made me look for a lot of JS codes, which I do not understand, and most of the times they are not working when executed using helium library in python even though they work fine in the Console of Google Chrome.
However, the code should be running headless and I am doing all of this only for testing and here is my main question
Does it matter if used new tab or new window if I am going to use headless browser in python helium
is it going to be faster just like what happens when it runs normal or since it is headless it makes no difference ?
not to forget to mention that it is way easier for me to work with many windows since I won't have to use JS codes.
driver.execute_script('''$x('xpath')[0].remove();''')
driver.execute_script('''$x('xpath')[0].click();''')
$x('xpath')[0].remove();
$x('xpath')[0].click();
selenium.common.exceptions.JavascriptException: Message: javascript error: $x is not defined
they both work just fine in Google Chrome Console but for some reason, I can not execute them python helium.
I tried using to give more time using time.sleep(5) to load the page but it did not work
not to mention that I tried using the click() method from helium it gives me a LookupError()
I found a way to test it for myself it might not be accurate but the results gave me a clear winner I used the windows task manager to see the percentage of CPU usage in all the cases when running headless tabs were faster and used way less CPU than new windows
I'm building a Selenium ActionChain via Python to hold and execute keyboard and mouse input. When the ActionChain only contains several keystrokes (through action.key_down() or action.key_up()), it executes in well under 0.01 s. As soon as I add a single cursor movement (using action.move_by_offset()) the execution time shoots up to 0.3-0.4 s.
First off, what's the technical difference that makes the mouse input so much more expensive?
Is there any better Selenium-based alternative, or should I be using a different tool for realtime browser input?
(For context, I'm using Selenium to run a reinforcement learning model on HTML5 multiplayer web games, so I need to execute actions as quickly as possible or my bot's reaction time suffers. I'm using the Firefox webdriver - I also tried Chrome, but found the geckodriver to be about twice as fast for mouse input. Ideally, I need to get all of my input execution to run in about 0.01 s so it doesn't slow down my model.)
If anyone is still looking for a fix here is what I did:
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver, duration=10) # duration in milliseconds
actions.move_to_element_with_offset(website_element, x, y).perform()
I found this out from looking a bit in the source and seeing a duration argument that was left on the default of 250 ms. Does anyone know why they do this?
I dont know the answers to your other questions though
I still can't answer why the issue exists beyond orde's note above, but I did find that Selenium doesn't seem like the best tool for this, and was able to (somewhat crudely) work around it using additional libraries.
For anyone else with a similar need:
I ended up using D3DShot for video (much faster frame grab than Selenium) and PyAutoGUI for much faster mouse control (closer to 1 ms). I still utilize Selenium, but only to navigate to the site, log in, and put the game window in full screen.
Unfortunately this setup effectively prevents multiple bots from running on the same machine (you might be able to hack something together with one bot per display, but that would get pretty messy.)
I have started getting into python web automation(with chrome). I am using python 3 and selenium module. Is there a way of having the recaptcha pop up in a seperate window allowing me to solve it like normal and then carrying on with my set automation?
Any advice is much apprectaited
Thanks
Image of captcha
I have a Python 2.7 script that should cast mp3 audio from Windows 10 to Chromecast audio devices through Google Chrome browser.
It does the following using Send Keys (keyboard and mouse automation) from the PyWinAuto module:
Open Google Chrome maximized
Click the "Customize and control Google Chrome" menu on the right
Selects "Cast.."
Selects a casting group
Inputs local address of mp3 in the address bar and hits Enter
It involves using time.sleep() a lot and has many problems as an experienced programmer would imagine. So I'm looking for a pythonic way of accomplishing the same result.
P.S: A cross-platform solution is much appreciated (especially for Raspberry Pi). And I can learn anything you suggest, even if it's not Python. Thank you
I am trying to run a headless browser, to which when I pass a URL simulates the entire webpage as it would if run from any of the popular browser. Importantly it must manage to run Adobe Flash Player (and hence flash videos). I have heard things about selenium webkit but I am not sure about its capabilities as I have never used it especially when it comes to handling flash content.
Infact if I were to narrow down the problem, I just want to run a flash content in a web site but out of the internet browsing window under my program (preferably python). If this is possible can someone point me the right approach. Do let me know if any further clarification is needed in the question.
Give a try to http://phantomjs.org/ it works great with a headless webkit and flash.
You could look at http://jeanphix.me/Ghost.py/ to control phantomjs with Python.