I need to open the browser using python.
I saw the Webbrowser module , which helps me fire a URL with a browser.
But i wanted to send a POST request by opening a browser.
The requests Module helps me to create a POST request alongwith the headers & Data .
Can i send the same by opening a browser tab or window ?
You should have a look at Selenium for Python
For example :
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com")
assert "Google" in driver.title
driver.quit()
That will launch a Firefox browser and get the URL you passed to the driver. It's up to you to find out how you can perform what you want to do now ;)
Related
I am automating data scraping from the website https://www.macquarieinsights.com/. Now the website needs login only the first time. After that, if you open the website its already logged in. But, when I use selenium to visit the website, it asks for login every time I visit it via selenium. How can I use the regular chrome browser where it's already logged in, using selenium? Or is there any other alternative?
You can login that site to make the cookies saved in your local browser profile and after that you can use that browser profile.
Existing browser profile settings can be loaded from your PC storage as following:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Path") #Path to your chrome profile
driver = webdriver.Chrome(executable_path="C:\\Users\\chromedriver.exe",chrome_options=options)
Try to save cookies 🍪 made by the website on your login. And then provide that cookies to selenium while starting it...
Like :
Manually login to browser.
Use requests to get the webpage with the previous cookies
import requests
import browser_cookie3
cookies = browser_cookie3.chrome()
response = requests.get(url, cookies=cookies)
Don't forgot to install module using pip3 install browser-cookie3
Im not sure if this is possible, but basically, I'm trying to open a tab in Chrome to show the outcome of my GET and POST requests. For instance, lets say im using Python requests to POST log in data to a site. I would want to then open a chrome tab after that POST request to see if I did it correctly, and also to continue on that webpage from there in Chrome itself. I know I can use response.text to check if my POST request succeeded, but is there a way I can physically open a chrome tab to see the result itself? Im guessing there would be a need to export some sort of cookies as well? Any help or insights would be appreciated.
When using selenium, you need to remove headless option for browser:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
#chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('<PATH_TO_CHROMEDRIVER>', options=chrome_options)
# load page via selenium
wd.get("<YOUR_TARGET_URL>")
# don't close browser
# wd.quit()
Also remove closing browser in the end of code.
After that browser will remain open and you will be able to continue work manually.
I have a scenario in which I need to process some the web request automatically and finally show the result in the browser. The website I am querying uses partial response and ajax call to get the whole page.
Is there a way in which I can open a web browser with response object or is there a way in which I can pass data and cookies with the url to web browser.
You can use selenium for it
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.your-website.com?q=test_param")
# You can sendout a wait to view the results
driver.close()
You can very well use chrome driver too if you are very much concerned about the browser
Is there a way to open a browser from a Python application, while posting data to it in POST method?
I tried using webbrowser.open(url) to open the browser window, but I need to also post some data to it (via POST variables), to make sure the page opens with the proper information.
As an example, I would like to open http://duckduckgo.com, while posting "q=mysearchterm" as POST data to it, so the page will open with pre-filled data.
I believe you should be using a Web Driver such as Selenium. Here is an example:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome(r"PATHTO\chromedriver.exe") # download the chromedriver and provide the full path here
browser.get('http://duckduckgo.com/')
search = browser.find_element_by_name('q')
search.send_keys("mysearchterm")
search.send_keys(Keys.RETURN)
time.sleep(15)
geckodriver does launch Firefox but firefox doesn't get url. Please see and point what's wrong with my function. would be great help as im very new to selenium and python
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
def Login(SiteUrl):
driver = webdriver.Firefox()
driver.get(SiteUrl)
if __name__ =="__main__":
url = "www.google.com"
Login(url)
There could be multiple reasons -
Try including HTTP protocol in the url i.e. - http://www.google.com
You might be behind a proxy server. See this SO question -> Selenium WebDriver: Firefox starts, but does not open the URL & Selenium WebDriver.get(url) does not open the URL
Your versions of the driver and the browser does not match.
That simply means either your libs are old or Gecko binary is old. New gecko-libs are available now.
Download it from :
https://github.com/mozilla/geckodriver/releases
Download new version of selenium python libs from below URL:
https://pypi.python.org/pypi/selenium
Another thing as Martin point out
Add the protocol like http or https before passing URL
So use URL as :
url = "https://www.google.com"