I am trying to submit a multipart POST request in Python. I looked around and found 2 variations:
Using 'reqests' (http://docs.python-requests.org/en/latest/)
Using urllib2 (https://docs.python.org/2/library/urllib2.html#module-urllib2)
I tried both of them and am able to submit the request successfully.
Below is the sample code for both:
----------requests--------------
resp = requests.post(submiturl, files=multipart_form_data, headers=headers,timeout=5)
where multipart_form_data contains my file object as well as string parameters
---------------urllib2------------
items.append(MultipartParam(name, value))
fileObj = open(inputFile,'r')
items.append(MultipartParam('file', filename=inputFile, fileobj=fileObj))
res = urllib2.urlopen(request)
My Question:
Which one should I use?
Correct me if I am wrong but I have seen that while submitting with urllib2 I get the HTTPError for response code like 500. However, while using "request" it does not throw the HTTPError for response code like 500s instead I have to manually add the condition:
Response.raise_for_status():
or:
resp.status_code != 200: raise Execption(...)
Is this correct or I am missing something?
Thanks!
Response.raise_for_status() raises for HTTP response code in the 4xx and 5xx ranges. The src is very clear and readable.
You'll get a 2xx response for successful requests, but you may also want to consider other response codes, for example redirects.
Related
Here is the code:
import requests
url = "https://api.yelp.com/v3/businesses/search"
response = requests.get("https://api.yelp.com/v3/businesses/search")
print(response)
and I got this error:
PS D:\HelloWord> cd "d:\HelloWord"
PS D:\HelloWord> python -u "d:\HelloWord\app.py"
<Response [400]>
And I don't know what is wrong here I follow up the tutorial and is the same.
Thank you in advance!
To understand what is happening, you need to look at the response object's properties. Take a look at the docs for response object. Printing response.text will probably be informative.
HTTP status 400 means there is some problem from the client-side. Please go through the doc again. Looking at the URL it seems that this Yelp API will need some sort of API key as param or some request header. These kinds of API are often not free for use.
For details place a line.
print(response.text)
It will give this response.
{"error": {"code": "VALIDATION_ERROR", "description": "Authorization is a required parameter.", "field": "Authorization", "instance": null}}
Thus it requires authorization details in some way which you can find on the doc itself.
According to docs, you need to set api_key into header of your request:
requests.get("https://api.yelp.com/v3/businesses/search", headers={'Authorization': 'Bearer <Your Api Key>'})
Read more about 400 error
Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.
>>> import requests
>>> r = requests.get('https://api.yelp.com/v3/businesses/search')
>>> r.text
When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access r.text. You can find out what encoding Requests is using, and change it, using the r.encoding property:
I'm trying to build an app that alerts when air quality rises above a certain level. I'm trying to get some json data from the api at https://api-docs.iqair.com, and they kindly provide simple copy and paste code. However, when I run this (with my API key), I get this error message:
requests.exceptions.MissingSchema: Invalid URL '{{urlExternalAPI}}v2/city?city=Los Angeles&state=California&country=USA&key={{my_key}}': No schema supplied. Perhaps you meant http://{{urlExternalAPI}}v2/city?city=Los Angeles&state=California&country=USA&key={{my_key}}?`
I tried putting in the http, but then nothing happened.
Here's the code they provide:
import requests
url = "{{urlExternalAPI}}v2/city?city=Los Angeles&state=California&country=USA&key={{YOUR_API_KEY}}"
payload = {}
headers= {}
response = requests.request("GET", url, headers=headers, data = payload)
print(response.text.encode('utf8'))
First of all, you have to put in the URL, and not use the curly brackets. Also, I couldn't find the correct URL, but after googling it, I found that I merely had to use the correct URL, which was https://api.airvisual.com.
I am trying to login into a website by passing username and password.It says session cookie is missing.I am beginner to api .I dont know if I have missed something here.The website is http://testing-ground.scraping.pro/login
import urllib3
http = urllib3.PoolManager()
url = 'http://testing-ground.scraping.pro/login?mode=login'
req = http.request('POST', url, fields={'usr':'admin','pwd':'12345'})
print(req.data.decode('utf-8'))
There are two issues in your code that make you unable to log in successfully.
The content-type issue
In the code you are using urllib3 to send data of content-type multipart/form-data. The website, however, seems to only accept the content-type application/x-www-form-urlencoded.
Try the following cURL commands:
curl -v -d "usr=admin&pwd=12345" http://testing-ground.scraping.pro/login?mode=login
curl -v -F "usr=admin&pwd=12345" http://testing-ground.scraping.pro/login?mode=login
For the first one, the content-type in your request header is application/x-www-form-urlencoded, so the website takes it and logs you in (with a 302 Found response).
The second one, however, sends data with content-type multipart/form-data. The website doesn't take it and therefore rejects your login request (with a 200 OK response).
The cookie issue
Another issue is that urllib3 follows redirect by default. More importantly, the cookie is not handled (i.e. stored and sent in the following requests) by default by urllib3. Thus, the second request won't contain the cookie tdsess=TEST_DRIVE_SESSION, and therefore the website returns the message that you're not logged in.
If you only care about the login request, you can try the following code:
import urllib3
http = urllib3.PoolManager()
url = 'http://testing-ground.scraping.pro/login?mode=login'
req = http.request('POST', url, data={'usr':'admin','pwd':'12345'}, encode_multipart=False, redirect=False)
print(req.data.decode('utf-8'))
The encode_multipart=False instructs urllib3 to send data with content-type application/x-www-form-urlencoded; the redirect=False tells it not to follow the redirect, so that you can see the response of your initial request.
If you do want to complete the whole login process, however, you need to save the cookie from the first response and send it in the second request. You can do it with urllib3, or
Use the Requests library
I'm not sure if you have any particular reasons to use urllib3. Urllib3 will definitely work if you implements it well, but I would suggest try the Request library, which is much easier to use. For you case, the following code with Request will work and get you to the welcome page:
import requests
url = 'http://testing-ground.scraping.pro/login?mode=login'
req = requests.post(url, data={'usr':'admin','pwd':'12345'})
print(req.text)
import requests
auth_credentials = ("admin", "12345")
url = "http://testing-ground.scraping.pro/login?mode=login"
response = requests.post(url=url, auth=auth_credentials)
print(response.text)
I am trying to post multipart/form data using requests library according to website on submiting the form you are redirected to page where your data is created but when I am trying using requests library it gives 200 as response instead it should give 302 as response please could any one help me in this i dont know what i am doing wrong
By default requests will follow "302" redirection responses. You can disable this as follows:
r = requests.get('http://github.com/', allow_redirects=False)
See https://requests.kennethreitz.org/en/master/user/quickstart/#redirection-and-history
I've never really attempted to try and write my own code that calls an API. I have some Python code that I created after discovering Python Requests library.
However, I can't get past this "authentication failed" error message.
The API key I have acquired is from https://fortnitetracker.com/site-api
The code I am using is as follows:
import requests
url = 'https://api.fortnitetracker.com/v1/profile/pc/ninja'
headers = {"TRN-Api-Key": "MY_KEY"}
r = requests.get(url, headers=headers)
After asking for the r.status_code I get a 403 Forbidden. When I ask for r.text I get
u'{"message":"Invalid authentication credentials"}\n'
On the API page,all they say is to pass the API key in the request headers using a GET method.
I even tried passing my credentials I used to register on the site using
r = requests.get(url, headers=headers,auth=('User', 'Pass'))
Still got the same invalid authentcation credentials error.
Is what I am doing correct? What am I missing?
Thanks for your help in advance.
If you are calling an API you need to send post :
r = requests.post(url, headers=headers)
Are you sure you don't have to supply the API key something like this?
{'Auth': 'Token my_key'}
That's how most API's do it, but I can't find any reference to their API docs anywhere.