I am not sure if this is possible, but I was wondering if it would be possible to write a script or program that would automatically open up my web browser, go to a certain site, fill out information, and click "send"? And if so, where would I even begin? Here's a more detailed overview of what I need:
Open browser
Go to website
Fill out a series of forms
Click OK
Fill out more forms
Click OK
Thank you all in advance.
There are a number of tools out there for this purpose. For example, Selenium, which even has a package on PyPI with Python bindings for it, will do the job.
maybe you can use zope.testbrowser it's really easy to use.
Related
I am working on application where I am giving a functionality to user where they can signin from dropbox by using my application but when I am trying to get the file from Dropbox I am unable to do it. When user choose the desired file from dropbox and click on choose and then nothing happen. Can anyone please help out either is it possible to do it or not? If yes, how can we do it? I am at beginner level. Please help me out and explain the whole process to do it in detail.
Are you using public Dropbox files?
Then you just need to fetch the item by URL and download it. If this happens in a browser tool, you'll need JavaScript and not Python to download it.
Or you leave out JS and just use Python to render an HTML page where a button is for a Dropbox file and clicking the button triggers a download of the file. That is a generic HTML task you can search for.
If you need access to sign in Dropbox and view private files, consider using a Python library built around Dropbox.
See the Python Dropbox guide. Are you using a library like that? Please share as your question was vague.
https://www.dropbox.com/developers/documentation/python#
Also please share an explanation of what your logic is or a small code snippet. I can't see what you are doing yet so I don't know where you are missing something or making a mistake.
From the screenshot, I see you're using the Dropbox Chooser. That's a pre-built way to let your end-users select files from their Dropbox accounts and give them to your app.
Make sure you implement the success and cancel callback methods as documented there for the Chooser.
In the success callback, you'll get the information for the selected file(s). That occurs in JavaScript in the browser though, so if you need that on your server, you'll need to write some JavaScript to send that up to your server, e.g., via an AJAX call or a form or whatever means you use in your app.
The problem: I want to write a Python script that takes a screenshot of a website I have opened in a browser each time it loads.
The thing is that I have a website where there are like 300 exam questions which I can get through, try each one of them and I will have the correction when I submit my answer. I will not have access to this questionnaire after a certain date, but I want to keep the questions (which I could write down, but laziness is strong in me, and want to learn Python).
The "attempt": I thought of doing a simple Python script with imgkit to take the screenshots. I'm opened to other suggestions, as imgkit was the first thing I saw while looking for this, and the code looks plain and simple to me:
import imgkit
imgkit.from_url('http://webpage.com', 'out.jpg')
But I have to provide the url for each webpage, and that will be more tedious than taking a screenshot with OS features, thus I want to automatize it.
The questions:
There is a way to make Python monitor a browser tab and take a screenshot each time it reloads (that will be when a new question appears)?
Or maybe get the tab's URL to pass it to imgkit and take the screenshot.
Another thing that I saw is that imgkit can generate a "screenshot" from a HTML file. Can Python download the HTML code from a tab I have open in my browser?
Selenium is your friend here. It is a framework designed for testing but it will make what you want really easy.
Selenium allows you to spin-up a web browser and control it. So you can instruct it to go to the web address you want and then do things. Normally you would instruct it to click here, write in a form, etc.
In your case you only want it to open a certain address, take a screenshot, go the the next address and repeat.
Here you have a tutorial on how to do exactly what you want.
The specific code is:
from selenium import webdriver
#1. Get the driver to manage the web-browser you choose
driver = webdriver.Chrome()
#2. Go the the webadress you want
driver.get('https://python.org')
#3. Take a screenshot
driver.save_screenshot("screenshot.png")
driver.close()
PS: In order for the tutorial to run you will need to have installed the web driver for Selenium to be able to spin-up and run Chrome. Here are the instructions for that.
I am new to web programming.
I am trying to add parameters to one of my Chrome extension.
I know I can enter, for example, "window.localStorage.setItem()" in the extension console. However, I cannot find a way to navigate my webdriver to that extension background page. I have seen, in the past, people would use chrome//extensions:extension_id as url to get to that page, but now this method seems not to work.
Is there any way that I can go to that page directly without telling my webdriver to click programmer mode and then click the extension?
Thanks in advance. This has been bothered me for hours.
Probably one thing to correct here is that chrome://extensions/ opens up the Extensions on Chrome browser. So this shall work fine :
driver.get("chrome://extensions/")
Just a note now the chrome://extensions/extension_id does not take you to the extension page anymore.
In case you are interested in the Details or Options of an extension, you shall try and access it via the links only since the uri is not consistent for different extensions as well. e.g
Extension 1
chrome-extension://gighmmpiobklfepjocnamgkkbiglidom/options/index.html
Extension 2 ::
chrome-extension://fngmhnnpilhplaeedifhccceomclgfbg/options_pages/support.html
The general format has extension_id but not certainly defines the entire uri to get you to the page.
I would suggest if you want to work around with the Extensions, there is this Management API which seems useful to dive in.
My program opens a certain page on using
webbrowser.open(url)
How is it possible to reload the tab containing the url several times?
I could use sleep to set the time limit in which it has to wait before it has to reload.
But how do I refresh the tab after that? (Not open it in a new tab.)
I don't think it would be possible to implement a pure python solution for this which works with different browsers. A solution I would think of is using JavaScript. Vaguely the idea is to create a html file which has an iframe with the url you want and has javascript for reloading the iframe in regular interval. Then use webbrowser module to open that file.
This may sound ugly but this may be the only solution given the security concerns of a browser.
*If you are interested with this idea I can help you writing the code for this.
Hope this helps.
EDIT: below is my OLD answer, I'm not deleting it because it shows the ambiguity in the docs, and could possibly serve as a learning experience to someone.
If you read the docs, they make it sound like its possible. However, it is not possible to do with this module, further more, it seems like no matter what option you give to "new" it always opens in a new tab. Perhaps this behavior is specific to my system, or browser(IE9) but I believe it is more likely a bug in the program.
I investigated further, there is questions about this all over SO. you can't do it with webbrowser or anything built into python.
If you install selenium, you should be able to do what you want.
I am assuming you don't have access to the source code of this webpage, otherwise, you could just use html to do the refresh. If you don't want to install selenium and don't have source access, then you need to make a wrapper for the webpage, and use HTML/JS to refresh the wrapper.
the docs say:
webbrowser.open(url, new=0, autoraise=True)
Display url using the default browser. If new is 0, the url is opened in the same browser window if possible. If new is 1, a new browser window is opened if possible. If new is 2, a new browser page (“tab”) is opened if possible. If autoraise is True, the window is raised if possible (note that under many window managers this will occur regardless of the setting of this variable).
so...
to refresh the page, it would just be:
for i in range(refresh_limit):
time.sleep(wait_time)
webbrowser.open(url)
^^^ this does not actually work^^^
link text
This is a link from a digital book library.There are forward and backward buttons to see next and previous page.I want to download these pictures automatically. I have once used urllib in python but the website baned it soon. I just want to download this book for study purpose so can anyone recommend me some programming tools such as web-spiders which can simulate the process of turning pages and get the pictures automatically. Thanks!
That site uses Javascript, so you can't easily scrape it with Python. Two suggestions:
Work out what requests are being made when clicking the next button. You can do this with a tool like firebug. You might then find you can scrape it without processing any JS.
Use a tool such as Selenium which allows for browser scripting which lets you "execute" the JS.
As for the site blocking you, there are two ways to reduce the chance of being blocked:
Change your user-agent to that of a common browser, e.g. Firefox.
Add random delays between accessing the next image, so that you appear more human-like.
wget is an excellent web spider
http://linux.die.net/man/1/wget
You need a real browser to work with this (kind of) site. Selenium is one option, but it is more geared towards web testing. For web scraping iMacros is really nice. I had a quick test and it works well with iMacros for Firefox/IE.
Chris