I am struggling to post to a javascript/react form with Pythons Requests. I understand the regular way would be something like this
payload = {"user": "me", "password": "12345"}
s = requests.Session()
html = s.post(url, data=payload) `
The url part is the problem, since I cannot find it in the source. The source of the form looks like this:
<form class="Login-form" method="POST" data-reactid="19"> … </form>
I assumed a value for the action parameter but, well, it ain't there. I also tried to find an url in the javascript but to be honest, I can't read it very well.
So my question would be: How – if at all – can I make a post with Requests to a react formular?
Edit:
To make the question more concise and reflect the accepted answer:
If an html-form with javascript has no obvious url in the source where it posts to, how can I find out the url?
A form without an action= attribute POSTs to the current URL. But since you're dealing with React, it's probably handled by an action to an API endpoint behind the scenes. Watch the network tab under developer tools in your browser of choice to see how it's actually implemented and what URL the React application talks to.
Related
I have worked with a few API's, but not sure how to get started with sending requests for Star Citizen. Does anyone know how you might go about using python to send a get request for say getting some data on game items. Here is their official API documentation but not sure where to start!
https://starcitizen-api.com/gamedata.php#get-items
Could anyone post an example get request that return data?
from the docs, the urls seems to be /xxxxxxxx/v1/gamedata/get/3.6.1/ship?name=Avenger or some such where i guess the xxx is your personal key or account or whatever
try this:
import requests
url = '/xxxxxxxx/v1/gamedata/get/3.6.1/ship?name=Avenger'
response = requests.get(url, verify = False)
contents = response.json()
just make sure the url is complete, should work the same for any web API really
EDIT:
from the docs it looks like the url should look like this (since the host is listed as Host: api.starcitizen-api.com
https://api.starcitizen-api.com/xxxxxxx/v1/gamedata/get/3.6.1/ship?name=Avenger
I am trying to make a post request to that website: http://archive.eso.org/wdb/wdb/asm/dimm_paranal/form
so far I did that:
import requests
import bs4
url = 'http://archive.eso.org/wdb/wdb/asm/dimm_paranal/form'
p = {'search': 'Search',
'start_date' : '2019-09-17..2019-09-18'}
post = requests.post(url,data=p)
when I analyse the text from the post I only get the form webpage html code and not the result of the query. How can I simulate the query?
Additional question: How can I check the checkboxes in the form?
The form has an action, in this case it is /wdb/wdb/asm/dimm_paranal/query. Try to send the request there...
In devtools (Ctrl+Shift+I) you have "Network". Go there and see what is actually requested, check all the data, response, headers and so on.
Another help I would recommend is a programm caled Postman. You can create requests there, no need to code it.
Additional answer to your additional question: The checkboxes have no default value. Just set anything. 1, true, whatever. It should work.
I am trying to post a request to log in to a website using the Requests module in Python but its not really working. I'm new to this...so I can't figure out if I should make my Username and Password cookies or some type of HTTP authorization thing I found (??).
from pyquery import PyQuery
import requests
url = 'http://www.locationary.com/home/index2.jsp'
So now, I think I'm supposed to use "post" and cookies....
ck = {'inUserName': 'USERNAME/EMAIL', 'inUserPass': 'PASSWORD'}
r = requests.post(url, cookies=ck)
content = r.text
q = PyQuery(content)
title = q("title").text()
print title
I have a feeling that I'm doing the cookies thing wrong...I don't know.
If it doesn't log in correctly, the title of the home page should come out to "Locationary.com" and if it does, it should be "Home Page."
If you could maybe explain a few things about requests and cookies to me and help me out with this, I would greatly appreciate it. :D
Thanks.
...It still didn't really work yet. Okay...so this is what the home page HTML says before you log in:
</td><td><img src="http://www.locationary.com/img/LocationaryImgs/icons/txt_email.gif"> </td>
<td><input class="Data_Entry_Field_Login" type="text" name="inUserName" id="inUserName" size="25"></td>
<td><img src="http://www.locationary.com/img/LocationaryImgs/icons/txt_password.gif"> </td>
<td><input class="Data_Entry_Field_Login" type="password" name="inUserPass" id="inUserPass"></td>
So I think I'm doing it right, but the output is still "Locationary.com"
2nd EDIT:
I want to be able to stay logged in for a long time and whenever I request a page under that domain, I want the content to show up as if I were logged in.
I know you've found another solution, but for those like me who find this question, looking for the same thing, it can be achieved with requests as follows:
Firstly, as Marcus did, check the source of the login form to get three pieces of information - the url that the form posts to, and the name attributes of the username and password fields. In his example, they are inUserName and inUserPass.
Once you've got that, you can use a requests.Session() instance to make a post request to the login url with your login details as a payload. Making requests from a session instance is essentially the same as using requests normally, it simply adds persistence, allowing you to store and use cookies etc.
Assuming your login attempt was successful, you can simply use the session instance to make further requests to the site. The cookie that identifies you will be used to authorise the requests.
Example
import requests
# Fill in your details here to be posted to the login form.
payload = {
'inUserName': 'username',
'inUserPass': 'password'
}
# Use 'with' to ensure the session context is closed after use.
with requests.Session() as s:
p = s.post('LOGIN_URL', data=payload)
# print the html returned or something more intelligent to see if it's a successful login page.
print p.text
# An authorised request.
r = s.get('A protected web page url')
print r.text
# etc...
If the information you want is on the page you are directed to immediately after login...
Lets call your ck variable payload instead, like in the python-requests docs:
payload = {'inUserName': 'USERNAME/EMAIL', 'inUserPass': 'PASSWORD'}
url = 'http://www.locationary.com/home/index2.jsp'
requests.post(url, data=payload)
Otherwise...
See https://stackoverflow.com/a/17633072/111362 below.
Let me try to make it simple, suppose URL of the site is http://example.com/ and let's suppose you need to sign up by filling username and password, so we go to the login page say http://example.com/login.php now and view it's source code and search for the action URL it will be in form tag something like
<form name="loginform" method="post" action="userinfo.php">
now take userinfo.php to make absolute URL which will be 'http://example.com/userinfo.php', now run a simple python script
import requests
url = 'http://example.com/userinfo.php'
values = {'username': 'user',
'password': 'pass'}
r = requests.post(url, data=values)
print r.content
I Hope that this helps someone somewhere someday.
The requests.Session() solution assisted with logging into a form with CSRF Protection (as used in Flask-WTF forms). Check if a csrf_token is required as a hidden field and add it to the payload with the username and password:
import requests
from bs4 import BeautifulSoup
payload = {
'email': 'email#example.com',
'password': 'passw0rd'
}
with requests.Session() as sess:
res = sess.get(server_name + '/signin')
signin = BeautifulSoup(res._content, 'html.parser')
payload['csrf_token'] = signin.find('input', id='csrf_token')['value']
res = sess.post(server_name + '/auth/login', data=payload)
Find out the name of the inputs used on the websites form for usernames <...name=username.../> and passwords <...name=password../> and replace them in the script below. Also replace the URL to point at the desired site to log into.
login.py
#!/usr/bin/env python
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
payload = { 'username': 'user#email.com', 'password': 'blahblahsecretpassw0rd' }
url = 'https://website.com/login.html'
requests.post(url, data=payload, verify=False)
The use of disable_warnings(InsecureRequestWarning) will silence any output from the script when trying to log into sites with unverified SSL certificates.
Extra:
To run this script from the command line on a UNIX based system place it in a directory, i.e. home/scripts and add this directory to your path in ~/.bash_profile or a similar file used by the terminal.
# Custom scripts
export CUSTOM_SCRIPTS=home/scripts
export PATH=$CUSTOM_SCRIPTS:$PATH
Then create a link to this python script inside home/scripts/login.py
ln -s ~/home/scripts/login.py ~/home/scripts/login
Close your terminal, start a new one, run login
Some pages may require more than login/pass. There may even be hidden fields. The most reliable way is to use inspect tool and look at the network tab while logging in, to see what data is being passed on.
I'm not sure if such a thing is possible, but I am trying to submit to a form such as https://lambdaschool.com/contact using a POST request.
I currently have the following:
import requests
payload = {"name":"MyName","lastname":"MyLast","email":"someemail#gmail.com","message":"My message"}
r = requests.post('http://lambdaschool.com/contact',params=payload)
print(r.text)
But I get the following error:
<title>405 Method Not Allowed</title>
etc.
Is such a thing possible to submit using a POST request?
If it were that simple, you'd see a lot of bots attacking every login form ever.
That URL obviously doesn't accept POST requests. That doesn't mean the submit button is POST-ing to that page (though clicking the button also gives that same error...)
You need to open the chrome / Firefox dev tools and watch the request to see what happens on form submit and replicate that data in Python.
Another option would be the mechanize or Selenium webdriver libraries to simulate a browser and fill out the form
params is for query parameters. You either want data, for a form encoded body, or json, for a JSON body.
I think the url should be 'http://lambdaschool.com/contact-form'.
Code snippets:
app.config
$locationProvider.html5Mode({
enabled: true,
requireBase: false
});
$urlRouterProvider
.when('logout', '/logout')
.otherwise('/');
Relevant HTML
<li>Logout</li>
Flask endpoint
#app.route("/logout")
#login_required
def logout():
logout_user()
return redirect(url_for("login"))
I have also set <base href="/"></base> in my HTML's header. Clicking on the link, however, does not result into anything happening (literally nothing happens).
What gives?
Because the request to logout never reaches your server. You need to create a LogoutController associated with the logout route that actually makes a request to your flask endpoint.
Solved this by virtue of reading the AngularJS documentation twice and found this little gem there. Providing the answer here so that it helps others who are beginning developing in Flask-Angular.
To quote:
Html link rewriting
When you use HTML5 history API mode, you will not need special
hashbang links. All you have to do is specify regular URL links, such
as: link
When a user clicks on this link,
In a legacy browser, the URL changes to /index.html#!/some?foo=bar
In a modern browser, the URL changes to /some?foo=bar
In cases like the following, links are not rewritten; instead, the
browser will perform a full page reload to the original link.
Links that contain target element
Example: link
Absolute links that go to a different domain
Example: link
Links starting with '/' that lead to a different base path
Example: link
Basically, changing this:
<li>Logout</li>
to this:
<li>Logout</li>
solved the issue.