We Keep Coding

How to add headers and body to a python request

I created a GET request in Python for an API and I would like to add headers and body
import urllib2
import os
proxy = 'http://26:Do#proxy:8080'
os.environ['http_proxy'] = proxy
os.environ['https_proxy'] = proxy
os.environ['HTTP_PROXY'] = proxy
os.environ['HTTPS_PROXY'] = proxy
contents = urllib2.urlopen("https://xxxx/lista?zile=50 ").read()
I tried in Postman and I received a response and I would like to receive the same response in python. How can I add headers and body ?
Thanks in advance

You can use the urlopen function with a Request object:
https://docs.python.org/2/library/urllib2.html#urllib2.urlopen
This Request object can contain headers and body:
https://docs.python.org/2/library/urllib2.html#urllib2.Request
Example: https://docs.python.org/2/howto/urllib2.html#data
P.S: HTTP GET requests don't have a body. Maybe you meant POST or PUT?

the best way is to use the request library which is pretty simple to use. https://realpython.com/python-requests/
example:
import requests
headers = {'Content-Type': 'application/json'}
data_json = {"some_key": "some_value"}
response = requests.post("https://xxxx/lista?zile=50", headers=headers, json=data_json)

Getting the JSON response from a POST request Python

I'm currently posting to a sever like so:
req = urllib2.Request('http://xxx.xxx.xx.xx/upload/')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json_string)
print(response.getcode())
I get a 200 code back however I want to read the JSON the server is sending back. How do I do this? (tying to avoid using the requests library)

I did not get code, because I did not have a url.
Try:
req = urllib2.Request('http://xxx.xxx.xx.xx/upload/')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json_string)
print(response.read())

To get the actual json object from the response not just the json serialised string you need to parse the response with the json library
import json
req = urllib2.Request('http://xxx.xxx.xx.xx/upload/')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json_string)
json_response = json.loads(response.read().decode('ascii'))
The encoding may also be utf-8 depending on what the server sends back yo you.
Alternatively you could use the requests library which I find much easier to interact with, you'll need to install it separately though with pip install requests
import requests, json
response = requests.post('http://xxx.xxx.xx.xx/upload', data={'data': json_string})
if response.ok:
response_json = response.json()
else:
print('Something went wrong, server sent code {}'.format(response.status_code))
requests library docs

Python Requests API call not working

I'm having an issue converting a working cURL call to an internal API to a python requests call.
Here's the working cURL call:
curl -k -H 'Authorization:Token token=12345' 'https://server.domain.com/api?query=query'
I then attempted to convert that call into a working python requests script here:
#!/usr/bin/env python
import requests
url = 'https://server.domain.com/api?query=query'
headers = {'Authorization': 'Token token=12345'}
r = requests.get(url, headers=headers, verify=False)
print r
I get a HTTP 401 or 500 error depending on how I change the headers variable around. What I do not understand is how my python request is any different then the cURL request. They are both being run from the same server, as the same user.
Any help would be appreciated

Hard to say without knowing your api, but you may have a redirect that curl is honoring that requests is not (or at least isn't send the headers on redirect).
Try using a session object to ensure all requests (and redirects) have your header.
#!/usr/bin/env python
import requests
url = 'https://server.domain.com/api?query=query'
headers = {'Authorization': 'Token token=12345'}
#start a session
s = requests.Session()
#add headers to session
s.headers.update(headers)
#use session to perform a GET request.
r = s.get(url)
print r

I figured it out, it turns out I had to specify the "accept" header value, the working script looks like this:
#!/usr/bin/env python
import requests
url = 'https://server.domain.com/api?query=query'
headers = {'Accept': 'application/app.app.v2+json', 'Authorization': 'Token token=12345'}
r = requests.get(url, headers=headers, verify=False)
print r.json()

Access data in Token protected API with python

I have been trying to use python to get data from an API - which I am able to access manually like below
curl --location --compressed --header “Authorization: Bearer [ACCESS_TOKEN]” [STREAM_URL]
How do I go about using it in python - I have read examples it says use requestsbut how to handle the Access_Token part in python
Any help would be appreciated?
Regards
VB

Just define your headers like you do in your curl example. In this case though, it's a dict. Assuming you want to use requests you would do something like this:
import requests
url = [STREAM_URL]
headers = {"Authorization": "Bearer [ACCESS TOKEN]"}
r = requests.get(url, headers=headers)
print(r.text)

There's no different what to use. urllib come with the python and if you don't like to download additional package, use urllib.
As OP mentioned --compressed in curl command, we should do the same to get gzipped content:
Edited ars answer:
import sys
if sys.version_info.major == 3:
import urllib.request as urllib2
else:
import urllib2
from StringIO import StringIO
import gzip
request = urllib2.Request('http://example.com/')
request.add_header('Accept-encoding', 'gzip')
request.add_header('Authorization', 'Bearer [ACCESS_TOKEN]')
response = urllib2.urlopen(request)
if response.info().get('Content-Encoding') == 'gzip':
buf = StringIO( response.read())
f = gzip.GzipFile(fileobj=buf)
data = f.read()
Does python urllib2 automatically uncompress gzip data fetched from webpage?

BLS API data using cURL and requests library?

All, I'm trying to implement a curl request to get data from the BLS. Following their example here (they show the curl request), my code looks like this:
import requests
headers = {'Content-type': 'application/json'}
params = {"seriesid":["LEU0254555900", "APU0000701111"],"startyear":"2002", "endyear":"2012"}
p = requests.post('http://api.bls.gov/publicAPI/v1/timeseries/data/', params = params,headers = headers)
print p.url
print p.content
I'm getting the following (error) output:
http://api.bls.gov/publicAPI/v1/timeseries/data/?seriesid=LEU0254555900&seriesid=APU0000701111&endyear=2012&startyear=2002
{"status":"REQUEST_FAILED","responseTime":0,"message":["Sorry, an
internal error occurred. Please check your input parameters and try
your request again."],"Results":null}
Anyone had to deal with the BLS api and python?
Is the requests library the best for this?

You need to send the data as json, not pass it as a params dict. params sets the url parameters, which is not what you want, you need to pass it as data.
This should work:
import requests
import json
headers = {'Content-type': 'application/json'}
data = json.dumps({"seriesid":["LEU0254555900", "APU0000701111"],"startyear":"2002", "endyear":"2012"})
p = requests.post('http://api.bls.gov/publicAPI/v1/timeseries/data/', data=data, headers=headers)
print p.url
print p.content