I'm trying to convert the following cURL command into a Request in Python.
curl -H 'customheader: value' -H 'customheader2: value' --data "Username=user&UserPassword=pass" https://thisismyurl
From what I understand, one can GET headers and POST data. So how do I do both like a cURL?
This is what I'm trying:
url = 'myurl'
headers = {'customheader': 'value', 'customheader2': 'value'}
data = 'Username=user&UserPassword=pass'
requests.get(url, headers=headers, data=data)
Which returns: HTTPError: HTTP 405: Method Not Allowed
If I use post: MissingArgumentError: HTTP 400: Bad Request
When you use the --data command line argument, curl switches to a POST request. Do the same in Python:
requests.post(url, headers=headers, data=data)
From the curl manual:
-d, --data <data>
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded.
You may want to manually set the Content-Type header accordingly, or use a dictionary for the data parameter (and have requests encode those to the right format for you; the Content-Type header is set for you as well):
url = 'myurl'
headers = {'customheader': 'value', 'customheader2': 'value'}
data = {'Username': 'user', 'UserPassword': 'pass'}
requests.post(url, headers=headers, data=data)
Related
I am getting http code 400 (bad request) when I am trying to post a file to server using python requests library.
Corresponding curl request which is successful:
curl -X POST -i https://de.api.labs.sophos.com/analysis/file/static/v1 \
-H 'Authorization: auth_string' \
-H 'Content-Type: multipart/form-data' \
-F "file=#filename"
API documentation: https://api.labs.sophos.com/doc/analysis/file/static.html
Can someone help me out what I might be doing wrong?
My code so far:
import requests
url = "https://de.api.labs.sophos.com/analysis/file/static/v1"
headers = {'content-type': 'multipart/form-data', 'Authorization': authorization}
with open(filepath, 'rb') as f:
files = {'file': f} # Even tried {'file': f.read()}
r = requests.post(url, files=files, headers=headers)
if r.status_code in [200, 202]:
return r.json()
else:
return r
TL;DR
Try to do it this way:
import requests
url = "https://de.api.labs.sophos.com/analysis/file/static/v1"
headers = {'Authorization': authorization} # no Content-Type here
r = requests.post(url, headers=headers, files={"file": open(filepath, "rb")})
print(r.status_code, r.text)
Why
You shouldn't set Content-Type header manually when posting files with requests.
There are 2 reasons why:
requests will set Content-Type to multipart/form-data implicitly before making an actual HTTP request (as it does for Content-Length for example)
When using Content-Type: multipart/form-data you should specify a boundary as well. If no boundary is set, server will not be able to read data from request body correctly. So boundary is a required part of Content-Type header, if you use multipart/form-data.
In your example you haven't set boundary for the request. The fact is that requests does not set it for you if you override Content-Type header (which you do). And then server is not able to read your file in request body. Therefore, it returns you 400 Bad Request.
You can check it by typing print(r.request.headers["Content-Type"]) after you've made your request. It will output this:
multipart/form-data
, but it must look like this instead:
multipart/form-data; boundary=6387a52fb4d1465310a2b63b2d1c6e70
On the other hand, curl adds boundary implicitly, so you everything is fine and you receive 200 OK.
You can check it as well:
curl -H 'Content-Type: multipart/form-data' -F "file=#123.txt" -v http://httpbin.org/post
Which outputs:
* Connected to httpbin.org (34.230.136.58) port 80 (#0)
> POST /post HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.47.0
> Content-Type: multipart/form-data; boundary=------------------------d257f5f4377a3997
...
The case is that I met a post request, the Content-Type is text/plain, the request data is not Form Data, but is Request Payload, I would like to know how can I post data correctly using python? especially in requests.
Request payload is a dict or text?
Try to send request without specifying headers like:
payload = {"callcount": 1,
"c0-scriptname": "newLoginDwr",
...
"xml": true}
requests.post(url, data=payload)
There is a cURL example in section Triggering a Build of TeamCity 9.x Documentation:
curl -v -u user:password http://teamcity.server.url:8111/app/rest/buildQueue --request POST --header "Content-Type:application/xml" --data-binary #build.xml
I'd like to know how to convert it into an equivalent Python script (using POST request from the requests module)?
BTW, I tried the following Python script but got such a response code 400 (Bad Request):
url = "http://myteamcity.com:8111/httpAuth/app/rest/buildQueue/"
headers = {'Content-Type': 'application/json'}
data = json.dumps({'buildTypeId': 'MyTestBuild'})
r = requests.post(url, headers=headers, data=data, auth=("username", "password"), timeout=10)
print "r = ", r
>> r = <Response [400]>
If change the Content-Type in headers into Accept, got another response code 415 (Unsupported Media Type):
headers = {'Accept': 'application/json'}
>> r = <Response [415]>
The documentation for triggering a build shows you need to send XML, not JSON:
<build>
<buildType id="buildConfID"/>
</build>
The TeamCity REST API is a bit of a mixed bag; some methods accept both XML and JSON, some only accept XML. This is one of those latter methods. They'll respond with either XML or JSON, based on what you set the Accept header to.
Send the above with your required build ID; for an XML document that simply you could use templating:
from xml.sax.saxutils import quoteattr
template = '<build><buildType id={id}/></build>'
url = "http://myteamcity.com:8111/httpAuth/app/rest/buildQueue/"
headers = {'Content-Type': 'application/xml'}
build_id = 'MyTestBuild'
data = template.format(id=quoteattr(build_id))
r = requests.post(url, headers=headers, data=data, auth=("username", "password"), timeout=10)
Note that I used the xml.sax.saxutils.quotattr() function to make sure the value of build_id is properly quoted for inclusion as a XML attribute.
This'll produce XML; add 'Accept': 'application/json' to the headers dictionary if you want to process a JSON response.
FYI json request works in TeamCity 10.
Since this question was written & answered, a contemporary OSS alternative now exists: pyteamcity.
Pip command to install (or add pyteamcity to requirements.txt, etc.)
pip install pyteamcity
Code:
from pyteamcity import TeamCity
tc = TeamCity('username', 'password', 'server', 'port')
result = tc.trigger_build('build_id')
print(f'Build triggered. Web URL: {result['webUrl']}')
I am supposed to send this:
curl --header "Content-Type: text/plain" --request POST --data "ON" example.com/rest/items/z12
Instead, I am sending this:
import requests
headers = {"Content-Type": "text/plain"}
url = 'http://example.com/rest/items/z12'
_dict = {"ON": ""}
res = requests.post(url, auth=('demo', 'demo'), params=_dict, headers=headers)
And I am getting an Error 400 (Bad Request?)
What am I doing wrong?
The POST body is set to ON; use the data argument:
import requests
headers = {"Content-Type": "text/plain"}
url = 'http://example.com/rest/items/z12'
res = requests.post(url, auth=('demo', 'demo'), data="ON", headers=headers)
The params argument is used for URL query parameters, and by using a dictionary you asked requests to encode that to a form encoding; so ?ON= is added to the URL.
See the curl manpage:
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button.
and the requests API:
data – (optional) Dictionary, bytes, or file-like object to send in the body of the Request.
params parameter in the requests.post method is used to add GET parameters to the URL. So you are doing something like this :
curl --header "Content-Type: text/plain" --request POST example.com/rest/items/z12?ON=
You should instead use the data parameter.
import requests
headers = {"Content-Type": "text/plain"}
url = 'http://example.com/rest/items/z12'
res = requests.post(url, auth=('demo', 'demo'), data="ON", headers=headers)
Moreover, if you give a dictionnary to the data parameter, it will send the payload as "application/x-www-form-urlencoded". In your curl command, you send raw string as payload. That's why I changed a bit your example.
I`m trying to make those 2 requests in python:
Request 1:
curl -X POST -H "Content-Type: application/json" -d '{ "auth_token": "auth1", "widget": "id1", "title": "Something1", "text": "Some text", "moreinfo": "Subtitle" }' serverip
Request 2:
vsphere_dict = {}
vsphere_dict['server_name'] = "servername"
vsphere_dict['api_version'] = apiVersion
vsphere_dict['guest_count'] = guestCount
vsphere_dict['guest_on'] = guestOnLen
vsphere_dict['guest_off'] = guestOffLen
#Convert output to Json to be sent
data = json.dumps(vsphere_dict)
curl -X POST -H "Content-Type: application/json" -d 'data' serverip
Neither of them seems to work. Is there any way I can send them in Python?
Update:
The part that I cannot handle is the pass auth and widget. I have tried the following without success:
import urllib2
import urllib
vsphere_dict = dict(
server_name="servername",
api_version="apiVersion",
guest_count="guestCount",
guest_on="guestOnLen",
guest_off="guestOffLen",
)
url = "http://ip:port"
auth = "authid89"
widget = "widgetid1"
# create request object, set url and post data
req = urllib2.Request(auth,url, data=urllib.urlencode(vsphere_dict))
# set header
req.add_header('Content-Type', 'application/json')
# send request
response = urllib2.urlopen(req)**
Resulting in "urllib2.HTTPError: HTTP Error 500: Internal Server Error"
Any ideas how I can pass the auth and widget correctly?
UPDATE:
To see what is different I have started a nc server locally. Here are the results:
Correct curl request using this code:
curl -X POST -H "Content-Type: application/json" -d '{ "auth_token": "auth", "widget": "widgetid", "title": "Something", "text": "Some text", "moreinfo": "Subtitle" }' http://localhost:8123
sends this which does work:
POST / HTTP/1.1
User-Agent: curl/7.21.0 (i386-redhat-linux-gnu) libcurl/7.21.0 NSS/3.12.10.0 zlib/1.2.5 libidn/1.18 libssh2/1.2.4
Host: localhst:8123
Accept: */*
Content-Type: application/json
Content-Length: 165
{ "auth_token": "token", "widget": "widgetid", "title": "Something", "text": "Some text", "moreinfo": "Subtitle" }
And request using this code
import requests
import simplejson as json
url = "http://localhost:8123"
data = {'auth_token': 'auth1', 'widget': 'id1', 'title': 'Something1', 'text': 'Some text', 'moreinfo': 'Subtitle'}
headers = {'Content-type': 'application/json'}
r = requests.post(url, data=json.dumps(data), headers=headers)
sends this which does not work:
POST / HTTP/1.1
Host: localhst:8123
Content-Length: 108
Content-type: application/json
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/2.0.1 CPython/2.7.0 Linux/2.6.35.14-106.fc14.i686
{"text": "Some text", "auth_token": "auth1", "moreinfo": "Subtitle", "widget": "id1", "title": "Something1"}
Requests provides you with the simplest and yet (very) powerful way to deal with HTTP requests in Python.
Maybe try something like this:
import requests
import simplejson as json
url = "http://ip:port"
data = {'auth_token': 'auth1', 'widget': 'id1', 'title': 'Something1', 'text': 'Some text', 'moreinfo': 'Subtitle'}
headers = {'Content-type': 'application/json'}
r = requests.post(url, data=json.dumps(data), headers=headers)
If the API requests authentication:
r = requests.post(url, data=json.dumps(data), headers=headers, auth=('user', 'pass'))
See [Requests auth] for details.
Well sure, using Python-Requests which is a Python library for sending requests like Curl. You can take a look at the Complicated Post Requests section.
Or, if you'd like to use curl inside of Python, you can use pyCurl.
In the example from the Dashing website, they use:
curl -d '{ "auth_token": "YOUR_AUTH_TOKEN", "current": 100 }' http://localhost:3030/widgets/karma
From the cURL man page, maybe you need to post it as form-urlencoded?
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data #foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.
You might also want to try python-requests http://requests.readthedocs.org/en/latest/user/quickstart/#more-complicated-post-requests
Update: I got it to work
import requests
import json
payload = {'auth_token': 'YOUR_AUTH_TOKEN', 'title': "pythontest"}
r = requests.post("http://localhost:3030/widgets/welcome", data=json.dumps(payload))
print r.text
You need to post the json like a form.
why not use urllib2?
import urllib2
import urllib
vsphere_dict = dict(
server_name="servername",
api_version=apiVersion,
guest_count=guestCount,
guest_on=guestOnLen,
guest_off=guestOffLen,
)
# create request object, set url and post data
req = urllib2.Request(some_url, data=urllib.urlencode(vsphere_dict))
# set header
req.add_header('Content-Type', 'application/json')
# send request
response = urllib2.urlopen(req)
UPD:
sorry, by i not understand that is auth and widget. Maybe this is also POST data?
HTTP Error 500 - can mean that server received not all POST parameters.