I have a website (my college website) for which each student has a username/password to login into his account. The website consists of many files uploaded in different formats. I want to programmatically enter my credentials and login to my account and download all files that have been uploaded in the site.
Is there any python module that would authenticate html form and provide access to the pages and then allow downloading files present in those pages?
I would look at http://scrapy.org. It's python based crawler/scraper that you can customize to perform any workflow you want. Here's an example of how user authentication is handled: http://doc.scrapy.org/topics/request-response.html?highlight=username#request-usage-examples
Related
I am working on a Chrome extension that users submit the URL of a website's privacy policy, and the backend(with flask) retrieves and analyzes the text. Now I want to store the analysis results in a public file or database so that the results are updated dynamically, and new users can directly use the results.
Are there any tools or methods that allow chrome extension users to access a public file or database?
I'm building a very simple website where a user authenticates and accesses a personal page from where they can download their paychecks. I've already implemented the login view, that redirects the user to the profile page. Now in the profile page, I need to add a list of paychecks to download. In order to do that, I need to take them from a static sub-directory with the same name as the user:
static/paychecks/username
I've been reading about serving static files, but I can't figure out how to do this specifically. Any help will be much appreciated.
I have a Windows10 + Python3.7
Right now i using requests for download web content and analyse it. But i can change it to selenium or something else if it needed...
I Want to login to shutterstock website through python to get some information from user pages (my sales data and etc). Shutterstock don't have any API for this actions and for many actions too... I want to write a console script for collecting data only. I will saving data to CSV file
I can parse my page to find all what i'm interested, but before it i'm need to login. And here is a problem because site have a login page with google "I'm not a robot" widget.
I tried to use requests auth functions - but it's doesn't working.
This is a login page:
https://contributor-accounts.shutterstock.com/login
Site don't have any tokens and other auth items. I saw software who connect to shutterstock. It just show popup window with all of this items from login page and then starting work without any windows.
My questions:
How to login to shutterstock?
How to store session for a long time if it possible, because i want to run my script every 4 hours and don't want to see login window every time
Shutterstock is working on building out a contributor api that would allow contributors to programmatically fetch their earnings and downloads data. These specific endpoints are still in progress, plan is to get them out early next year. If you reach out to contributor support they can probably add you as a beta user for these (and other) features that you're looking for.
How can I generate facebook page tokens dynamically with python for different users of my app? I want different users to post to their own pages directly from my app. most of the solutions I have seen and tried allows for posting to one page that doesn't change
Okay, so I've got a website. It's a site, where I can check my university timetable. So I've decided to write an application in Python, which logs in, scraps the timetable (classrooms, hours, lecturers, etc.), and display them, lets say for the beggining, in a .txt file. So I've done some Basic Authentication with HTTP Requests, it looked just like this:
url = "http://httpbin.org"
authURL = "http://httpbin.org/basic-auth/user/passwd"
r=requests.get(authURL, auth=HTTPBasicAuth("user", "passwd"))
print (r.content)
It's a freely hosted service, just to practise. Okay, but there are many other types of authentication. And here's my question: How can I actually determine which one is this website using, and then use that information in my application?