Is there a way to open a browser from a Python application, while posting data to it in POST method?
I tried using webbrowser.open(url) to open the browser window, but I need to also post some data to it (via POST variables), to make sure the page opens with the proper information.
As an example, I would like to open http://duckduckgo.com, while posting "q=mysearchterm" as POST data to it, so the page will open with pre-filled data.
I believe you should be using a Web Driver such as Selenium. Here is an example:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome(r"PATHTO\chromedriver.exe") # download the chromedriver and provide the full path here
browser.get('http://duckduckgo.com/')
search = browser.find_element_by_name('q')
search.send_keys("mysearchterm")
search.send_keys(Keys.RETURN)
time.sleep(15)
Related
Im not sure if this is possible, but basically, I'm trying to open a tab in Chrome to show the outcome of my GET and POST requests. For instance, lets say im using Python requests to POST log in data to a site. I would want to then open a chrome tab after that POST request to see if I did it correctly, and also to continue on that webpage from there in Chrome itself. I know I can use response.text to check if my POST request succeeded, but is there a way I can physically open a chrome tab to see the result itself? Im guessing there would be a need to export some sort of cookies as well? Any help or insights would be appreciated.
When using selenium, you need to remove headless option for browser:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
#chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('<PATH_TO_CHROMEDRIVER>', options=chrome_options)
# load page via selenium
wd.get("<YOUR_TARGET_URL>")
# don't close browser
# wd.quit()
Also remove closing browser in the end of code.
After that browser will remain open and you will be able to continue work manually.
I would like to log into a website and download a file. I'm using selenium and the chromedriver. Would like to know if there is a better way. It currently opens up a chrome browser window and sends the info. I don't want to see the browser window opened up and the data being sent. Just want to send it and return the data into a variable.
from selenium import webdriver
driver = webdriver.Chrome()
def site_login(URL,ID_username,ID_password,ID_submit,name,pas):
driver.get(URL)
driver.find_element_by_id(ID_username).send_keys(name)
driver.find_element_by_id(ID_password).send_keys(pas)
driver.find_element_by_id(ID_submit).click()
URL = "www.mywebsite.com/login"
ID_username = "name"
ID_password = "password"
ID_submit = "submit"
name = "myemail#mail.com"
pas = "mypassword"
resp=site_login(URL,ID_username,ID_password,ID_submit,name,pas)
You can run chrome in headless mode. In which case, the chrome UI won't show up and still performing the task you were doing. Some article I found on this https://intoli.com/blog/running-selenium-with-headless-chrome/. Hope this helps.
First option: If you are able to change the driver, you can use phantom-js as driver. That was a headless browser and you can use it with selenium.
Second option: If the site are not dynamic (easily called it SPA) or you are able to trace packet (which can be done in chrome dev tools), you can directly use request with the help of beautifulsoup if you need to get some data on the page.
Just add this two lines
chrome_options = Options()
chrome_options.add_argument("--headless")
This should make chrome run in the background.
I had a doubt with my code . Why is it required to open a google webpage to search something in selenium .why can't we search directly search from chrome browser. below is my code
from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
chrome_options=webdriver.ChromeOptions()
chrome_options.add_argument("--disable-infobars")
browser=webdriver.Chrome('C:/Users/chromedriver.exe',chrome_options=chrome_options)
page=browser.get('http://www.google.com')
search=browser.find_element_by_name('q')
search.send_keys('aditya')
search.send_keys(Keys.RETURN)
time.sleep(5)
why is the line page=browser.get('http://www.google.com') required. I am anyways using browser to send keys why is page required?
When you first instantiate a new browser driver, it is just a blank browser. You need to navigate to a URL before you can interact with any objects on that page. In your case, you are trying to do a google search, so you must navigate to the page first before being able to find the search element.
You shouldn't have to assign it to the page variable though. You should be able to just write:
browser.get('http://www.google.com')
I'm using selenium to pull some automated phone reporting from our phone system (Barracuda Cudatel, very nice small business system but it doesn't have an API for what I need). There is a button on the report page that has some javascript attached to it on a click listener that then tells the browser to download the file.
Obviously selenium isn't really designed to pull files like this, however all I'm trying to do is get the href of the url that would have been sent to the browser. I can then turn around and use the session credentials with requests to pull the file and do processing on it.
How do I do the following (In Python):
Query for the event listener for 'click'
Fire off the javascript
Get the resulting URL
Edit: I'm aware download location can be configured on the browser in selenium however I'm not interested in completing this task in that fashion. This is running against a selenium grid of 20 machines and the request could be routed to any of them. Since I can't pull the file through selenium I'm going to just pull it directly with requests.
Code I'm twiddling with is below.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from time import sleep
dcap = webdriver.DesiredCapabilities.CHROME
driver = webdriver.Remote(command_executor='http://gridurl:4444/wd/hub', desired_capabilities=dcap)
driver.get("http://cudatelurl")
driver.find_element_by_name("__auth_user").send_keys("user")
driver.find_element_by_name("__auth_pass").send_keys("password")
driver.find_element_by_id("manage").click()
driver.get("http://cudatelurl/#!/cudatel/cdrs")
sleep(5)
date_dropdown = Select(driver.find_element_by_xpath('//*[#id="cui-content-inner"]/div[3]/div/div/div/div[2]/div/div/div[1]/div[2]/div/select'))
date_dropdown.select_by_value("last_week")
# This is the element that has javascript attached to it the click register is
# button.widgetType.csvButtonWidget.widgetized.actionButtonType.table-action
# but I'd prefer to not hard code it
driver.find_element_by_xpath('//*[#id="cui-content-inner"]/div[3]/div/div/div/div[2]/div/div/div[1]/div[2]/button[1]')
print(driver.get_cookies())
print(driver.title)
sleep(10)
driver.close()
driver.quit()
You can still approach it with selenium by configuring the target directory for the file of a specific mime-type to be automatically downloaded (without the Save As dialog), see:
Download PDF files automatically in Firefox using Selenium WebDriver
Access to file download dialog in Firefox
Firefox + Selenium WebDriver and download a csv file automatically
I need to open the browser using python.
I saw the Webbrowser module , which helps me fire a URL with a browser.
But i wanted to send a POST request by opening a browser.
The requests Module helps me to create a POST request alongwith the headers & Data .
Can i send the same by opening a browser tab or window ?
You should have a look at Selenium for Python
For example :
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com")
assert "Google" in driver.title
driver.quit()
That will launch a Firefox browser and get the URL you passed to the driver. It's up to you to find out how you can perform what you want to do now ;)