I have a fully working Python script for taking screenshots of webpages using selenium and the FF driver(windows).
The script is taking full page screenshots instead of let's say 1024x768, which make the process slow. Here is my code:
driver = webdriver.Firefox()
driver.set_window_size(1024,768)
driver.get(url)
driver.get_screenshot_as_file("%s/%s.png"%(screen_dir,o.netloc))
driver.close()
Any ideas on how to make selenium/firefox take above the fold screenshots and not full page?
Thanks!
This is how the "take screenshot" functionality is working in Firefox - it would take a screenshot of a complete page from top to bottom even if part of it is not visible. This is actually not how it supposed to work by the specification.
Chrome would take a screenshot of the visible area / viewport only. Switch to it.
Related
Chrome seems to display a small preview of the element whose screenshot is being taken by screenshot_as_png on the top right corner.
Note: The following code will take repeated screenshots of Google's logo on its search page. Chrome tends to darken the background during this.
Epileptic warning: Your browser will flash rapidly while running the following code.
You can move the driver.find_element_by_xpath('//*[#id="hplogo"]').screenshot_as_png outside the while loop to avoid flashing but you will have to be quick to see the screenshot preview...
from selenium import webdriver
driver = webdriver.Chrome(executable_path=) #enter your executable path
driver.get('https://google.com')
while True:
driver.find_element_by_xpath('//*[#id="hplogo"]').screenshot_as_png
Before taking the screenshot:
Preview during screenshot:
Is there a way to disable this preview via options during creating a driver object?
Specifications:
Chrome version: 84.0.4147.89 (Official Build) (64-bit)
Chrome driver version: 84.0.4147.30 32-bit
Selenium version: 3.141.0
Additional information:
(This information is secondary information that isn't necessary to know to answer this question)
I am trying to apply Q-learning and NEAT algorithm (separately) to automate the chrome dino game. I am taking repeated screenshots of the canvas in that game and later processing the image using cv2.
As you can see by running the above code, this causes the browser to flash rapidly. Previously, I was trying to do this in Firefox on the "https://chromedino.com/" but I later decided to switch to Chrome to play the original game instead of a replica whose faithfulness to the original is questionable.
I didn't face any flashing/screenshot issues in Firefox.
Update: I haven't yet found a solution to this problem. Instead, I used the extracted dino game from Chrome and opened it in Firefox.
I am trying to make Python play a video which is embedded in a website (https://harsh10822.wixsite.com/intro).
I tried using ID,xpath etc etc but It didn't workout.
There is a similar Question to mine here (How to click on the play button of a youtube video embedded within smtebook through selenium and python) But I couldn't figure out how to apply the code.
If You could help me out by providing the code,I will be really happy.
Thankyou
This should work.
driver = webdriver.Firefox()
driver.get('https://harsh10822.wixsite.com/intro')
time.sleep(5)
video = driver.find_element_by_id('video-player-comp-ka1067od')
video.click()
Waiting is important in this case, because embedded video doesn't load instantly with the page so selenium needs to wait. You can change 5 seconds to any number that works for you.
See: How to click on the play button of a youtube video embedded within smtebook through selenium and python
But this uses a slightly different technique instead of WebDriverWait or sleeping for a fixed number of seconds, which can be wasteful if you are sleeping for longer than necessary.
Instead, use a call to driver.implicitly_wait(10). Then any calls that attempt to locate elements will implicitly wait for up to 10 seconds for the element to appear before timing out but will wait no longer than necessary. It's just a question of knowing which elements to look for and where:
driver = webdriver.Firefox()
driver.implicitly_wait(10) # wait for up to 10 seconds before timeout for any find operation
driver.get('https://harsh10822.wixsite.com/intro')
driver.switch_to.frame(driver.find_element_by_xpath('//iframe[starts-with(#src, "https://www.youtube.com/embed")]')) # switch to iframe
button = driver.find_element_by_xpath('//button[#aria-label="Play"]') # look for button
button.click()
Make sure that particular web driver is downloaded and placed in the directory wherein your code is saved. And for reference you can refer to Tech With Tim selenium tutorials.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
video=input()
driver=webdriver.Chrome()
driver.get("https://youtube.com")
search=driver.find_element_by_id("search")
time.sleep(2)
search.send_keys(video,Keys.RETURN)
time.sleep(2)
play=driver.find_element_by_id("video-title")
play.send_keys(Keys.RETURN)
I searched for this question,and I found an idea using driver.switch_to.window(),but it didn't work as expect:
from selenium import webdriver
driver1=webdriver.Chrome("D:\Python\Files\chromedriver.exe")
driver1.get('https://www.google.com')
driver2=webdriver.Chrome("D:\Python\Files\chromedriver.exe")
driver2.get('https://www.bing.com/')
driver1.switch_to.window(driver1.current_window_handle)
above code will first open a chrome window and go to google,then will open another chrome window and go to bing,then
driver1.switch_to.window(driver1.current_window_handle)
seems didn't work,the window showing bing still shows on top of the window showing google.
Anyone have any idea?I think
driver1.switch_to.window(driver1.current_window_handle)
may have some BUG.
As you have used two WebDriver instances as driver1 and driver2 respectively to openthe urls https://www.google.com (e.g. windowA) and https://www.bing.com/ (e.g. windowB) it is worth to mention that the function switch_to.window() is a WebDriver method. So, driver1 can control only windowA and driver2 can control only windowB.
For Selenium to interact with any of the Browsing Window, Selenium needs focus. So to iterate among the different Browsing Windows you can shift the focus to the different Browsing Window using JavascriptExecutor as follows :
Python:
driver1.execute_script("window.focus();")
driver2.execute_script("window.focus();")
Java:
((JavascriptExecutor) driver1).executeScript("window.focus();");
((JavascriptExecutor) driver2).executeScript("window.focus();");
I believe you have a different concept of "window" in driver.switch_to.window(). In chrome browser, it means "tab". It's not another chrome browser or browser window like what are you trying to do in your code.
If switch_to.window() what you really want, I'll give an example how to use it:
driver=webdriver.Chrome("D:\Python\Files\chromedriver.exe")
driver.get('https://www.google.com')
# open a new tab with js
driver.execute_script("window.open('https://www.bing.com')")
driver.switch_to.window(driver.window_handles[-1])
# now your driver is pointed to the "tab" you just opened
I am using selenium 3.0.2 and PhantomJS to scrape some AJAX-values off one specific site. I am on Python 2.7, OS X 10.8.
I have had periods, where PhantomJS is working fine, and suddenly out of the blue it can't find the html I am looking for, although it is present and I did not change the code.
Is PhantomJS prone to behaving erratically and is there another headless alternative that is more stable that works with my set-up? I can't get chromedriver to work.
EDIT: I am using
driver.get()
time.sleep(5) #I have played with this value
wait = WebDriverWait(driver, 10) #also played with this value up to 60...
try:
table = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, "bla bla")))
Thanks guys!
you didn't provide any details but i guess it is due to loading time of page, try to use Explicit Waits Explicit Waits, It'll wait until your div loaded completely then you can perform other actions on the page.
I've just started to learn coding this month and started with Python. I would like to automate a simple task (my first project) - visit a company's career website, retrieve all the jobs posted for the day and store them in a file. So this is what I would like to do, in sequence:
Go to http://www.nov.com/careers/jobsearch.aspx
Select the option - 25 Jobs per page
Select the date option - Today
Click on Search for Jobs
Store results in a file (just the job titles)
I looked around and found that Selenium is the best way to go about handling .aspx pages.
I have done steps 1-4 using Selenium. However, there are two issues:
I do not want the browser opening up. I just need the output saved to a file.
Even if I am ok with the browser popping up, using the Python code (exported from Selenium as Web Driver) on IDLE (i have windows OS) results in errors. When I run the Python code, the browser opens up and the link is loaded. But none of the form selections happen and I get the foll error message (link below), before the browser closes. So what does the error message mean?
http://i.stack.imgur.com/lmcDz.png
Any help/guidance will be appreciated...Thanks!
First about the error you've got, I should say that according to the expression NoSuchElementException and the message Unable to locate element, the selector you provided for the web-driver is wrong and web-driver can't find the element.
Well, since you did not post your code and I can't open the link of the website you entered, I can just give you a sample code and I will count as much details as I can.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("url")
number_option = driver.find_element_by_id("id_for_25_option_indicator")
number_option.click()
date_option = driver.find_element_by_id("id_for_today_option_indicator")
date_option.click()
search_button = driver.find_element_by_id("id_for_search_button")
search_button.click()
all_results = driver.find_elements_by_xpath("some_xpath_that_is_common_between_all_job_results")
result_file = open("result_file.txt", "w")
for result in all_results:
result_file.write(result.text + "\n")
driver.close()
result_file.close()
Since you said you just started to learn coding recently, I think I have to give some explanations:
I recommend you to use driver.find_element_by_id in all cases that elements have ID property. It's more robust.
Instead of result.text, you can use result.get_attribute("value") or result.get_attribute("innerHTML").
That's all came into my mind by now; but it's better if you post your code and we see what is wrong with that. Additionally, it would be great if you gave me a new link to the website, so I can add more details to the code; your current link is broken.
Concerning the first issue, you can simply use a headless browser. This is possible with Chrome as well as Firefox.
Check Grey Li's answer here for example: Python - Firefox Headless
from selenium import webdriver
options = webdriver.FirefoxOptions()
options.add_argument('headless')
driver = webdriver.Firefox(options=options)