I have the following png binary data that I was able to pull from a page utilizing selenium with the following code:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.polyvore.com/cgi/img-thing?.out=jpg&size=l&tid=39713077')
data = driver.get_screenshot_as_png()
However, the image looks like the following and I'd like to remove the black space around it:
The image is located here: http://www.polyvore.com/cgi/img-thing?.out=jpg&size=l&tid=39713077
Is there a way to remove the black space utilizing the binary data or get selenium to pull only the image and not the black background?
I've tried to utilize Pil, but I've only found ways to remove white space and not black space, plus it's difficult to turn it back to binary data, which I need.
I've also looked into the PNG Module, but I couldn't figure out how to turn it back to binary as well.
One solution would be to directly get the screenshot of the image:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.polyvore.com/cgi/img-thing?.out=jpg&size=l&tid=39713077')
element = driver.find_element_by_css_selector("img")
image = element.screenshot_as_png()
But unfortunately Firefox doesn't yet implement this feature.
Another way would be to crop the screenshot to the targeted element:
import StringIO
from selenium import webdriver
from PIL import Image
driver = webdriver.Firefox()
driver.get('http://www.polyvore.com/cgi/img-thing?.out=jpg&size=l&tid=39713077')
element = driver.find_element_by_css_selector("img")
rect = driver.execute_script("return arguments[0].getBoundingClientRect();", element)
screenshot = driver.get_screenshot_as_png()
img = Image.open(StringIO.StringIO(screenshot))
img_cropped = image.crop((rect['x'], rect['y'], rect['width'], rect['height']))
img_cropped.save('screenshot.png')
Related
I would like to know how to extract the data that popped up when the mouse hovers over a certain time frame inside a graph panel.
The website is
https://app.truflation.com/
Data Graph Image
I tried Use Selenium to webscrape the data that popped up. There was a solution from https://towardsdatascience.com/scraping-interactive-charts-with-python-2bc20a9c7f5c and https://youtu.be/lTypMlVBFM4 but the difference here is I cannot capture the XPATH of the popup message.
from selenium import webdriver
DRIVER_PATH = '/Users/hudso/OneDrive/Documents/UST course/chromedriver.exe'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get("https://app.truflation.com/")
datas = driver.find_element_by_class_name('overlay')
for data in datas:
Date = data.find_element_by_xpath('.//*[#id="trufrate-timeframe"]/g[2]/text[1]').text
Inflation = data.find_element_by_xpath('.//*[#id="trufrate-timeframe"]/g[2]').text
print(Date,Inflation)
I tried many different versions and this one seems to be closer. I am yet to print Date and Inflation, so not yet applied it to dataframe.
I am trying to take a screenshot of a particular webpage element based on class name. I have followed the methods described in How to take screenshot with Selenium WebDriver, How to screenshot a specified WebElement in Selenium using Python and How to take partial screenshot with Selenium WebDriver in python?
Following are the commands and their errors:
driver.find_element_by_class_name("views-field-body").screenshot("test.png")
and driver.find_element_by_class_name("views-field-body").screenshot_as_png
Both times I get the error message as
selenium.common.exceptions.WebDriverException: Message: unknown command: session/75c3765173a9cf726d35afa7978d9b6e/element/0.5926184656216698-3/screenshot
When I try
image = driver.find_element_by_class_name("views-field-body").screenshot
this commands executes, but the image object comes out as bound method as shown in the quoted text below
bound method WebElement.screenshot of selenium.webdriver.remote.webelement.WebElement (session="75c3765173a9cf726d35afa7978d9b6e", element="0.5926184656216698-3")
How does one save this bound method to image on disk? Why are the commands not executing? Using Python 3.8, if that matters.
I think you use firefox not chrome
I found a solution
from selenium import webdriver
from PIL import Image
fox = webdriver.Firefox()
fox.get('https://stackoverflow.com/')
# now that we have the preliminary stuff out of the way time to get that image :D
element = fox.find_element_by_id('hlogo') # find part of the page you want image of
location = element.location
size = element.size
fox.save_screenshot('screenshot.png') # saves screenshot of entire page
fox.quit()
im = Image.open('screenshot.png') # uses PIL library to open image in memory
left = location['x']
top = location['y']
right = location['x'] + size['width']
bottom = location['y'] + size['height']
im = im.crop((left, top, right, bottom)) # defines crop points
im.save('screenshot.png') # saves new cropped image
https://stackoverflow.com/a/37565356/8951071
I have been using selenium to automatic printing documents and I am stuck on the print screen. As far as I know, selenium does not interact with the print screen so I am looking for an alternate situation that I can use with selenium. My code so far is down below, and all I need is code that will let me choose a new printer and then print. Also I want to change that printer to Save as PDF and then save the pdf to a file, so if that gives me a shortcut that would help a lot.
from selenium import webdriver
from selenium import *
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
driver = webdriver.Remote(
command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities=DesiredCapabilities.CHROME) #This is because I am using remote web driver for my Mac, but it is the same as regular web driver
driver.execute("window.print()")
#Need code here
I used window.print() followed by execution of a python string containing the necessary js commands:
print_function = '''
let A = document.getElementsByTagName('print-preview-app')[0].shadowRoot;
let B = A.getElementById('sidebar').children[0].shadowRoot;
let C = B.getElementById('button-strip').children[0]
C.click()
'''
driver.execute_script(print_function)
Keep in mind that you also need to use driver.swich_to.window(window_handles[i]) to make sure you're interacting with the print box.
Once you enter into a shadowRoot element, you don't have the full compliment of driver.find_element_by methods available to you. You're limited to the methods available via JS when you are searching within a shadowRoot.
Found a suggestion that might work for you.
How to convert webpage into PDF by using Python
Maybe try pdfkit
import pdfkit
pdfkit.from_url('http://google.com', 'out.pdf')
Because of the need to check like 100+ registration numbers, I decided to create a script which will do this for me, as I need to test these often.
The idea is that I have to visit the following website:
https://www.anaf.ro/inactivi/index.jsp
Where I input a registration number in the first field, and I have to input the captcha in the second. The captcha is quite easy to solve, and it is straight forward, as there are already available libraries for python, but the problem that I am facing is that the "src" attribute of the image, goes to a dynamic url.
Is there any way of saving the image which is displayed when using the .get on the webdriver?
This is how I am trying to solve the problem:
from captcha_solver import CaptchaSolver
from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get("https://www.anaf.ro/inactivi/index.jsp")
time.sleep(5)
elem = driver.find_element_by_name('inputCui') # Find the first input box
elem.send_keys('17741254') # Input the desired code
Or maybe, if someone has another idea on how I can approach the problem, I am open to suggestions.
Ok, so I managed to solve it, I will simply take a screenshot of the page, then crop it, after that I will decode the captcha. Here is the code for those interested:
def get_captcha(driver, element, path):
location = element.location
size = element.size
driver.save_screenshot(path)
image = Image.open(path)
left = location['x']
top = location['y']
right = location['x'] + size['width']
bottom = location['y'] + size['height']
image = image.crop((left, top, right, bottom))
image.save(path, 'png')
img = driver.find_element_by_xpath("//img[1]")
get_captcha(driver, img, "captcha.png")
I am trying to capture all the visible content of a page as text. Let's say that one for example.
If I store the page source then I won't be capturing the comments section because it's loaded using javascript.
Is there a way to take HTML snapshots with selenium webdriver?
(Preferably expressed using the python wrapper)
Regardless of whether or not the HTML of the page is generated using JavaScript, you will still be able to capture it using driver.page_source.
I imagine the reason you haven't been able to capture the source of the comments section in your example is because it's contained in an iframe - In order to capture the html source for content within a frame/iframe you'll need to first switch focus to that particular frame followed by calling driver.page_source.
This code will take a screenshot of the entire page:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://dukescript.com/best/practices/2015/11/23/dynamic-templates.html')
driver.save_screenshot('screenshot.png')
driver.quit()
however, if you just want a screenshot of a specific element, you could use this:
def get_element_screenshot(element: WebElement) -> bytes:
driver = element._parent
ActionChains(driver).move_to_element(element).perform() # focus
src_base64 = driver.get_screenshot_as_base64()
scr_png = b64decode(src_base64)
scr_img = Image(blob=scr_png)
x = element.location["x"]
y = element.location["y"]
w = element.size["width"]
h = element.size["height"]
scr_img.crop(
left=math.floor(x),
top=math.floor(y),
width=math.ceil(w),
height=math.ceil(h))
return scr_img.make_blob()
Where the WebElement is the Element you're chasing. of course, this method requires you to import from base64 import b64decode and from wand.image import Image to handle the cropping.