We Keep Coding

translating simple curl call to python/django request

I'm attempting to translate the following curl request to something that will run in django.
curl -X POST https://api.lemlist.com/api/hooks --data '{"targetUrl":"https://example.com/lemlist-hook"}' --header "Content-Type: application/json" --user ":1234567980abcedf"
I've run this in git bash and it returns the expected response.
What I have in my django project is the following:
apikey = '1234567980abcedf'
hookurl = 'https://example.com/lemlist-hook'
data = '{"targetUrl":hookurl}'
headers = {'Content-Type': 'application/json'}
response = requests.post(f'https://api.lemlist.com/api/hooks/', data=data, headers=headers, auth=('', apikey))
Running this python code returns this as a json response
{}
Any thoughts on where there might be a problem in my code?
Thanks!

Adding to what nikoola said, I think you want that whole data line to be as follows so you aren't passing that whole thing as a string. Requests will handle serializing and converting python objects to json for you [EDIT: if you use the json argument instead of data].
source: https://requests.readthedocs.io/en/master/user/quickstart/#more-complicated-post-requests
Instead of encoding the dict yourself, you can also pass it directly
using the json parameter (added in version 2.4.2) and it will be
encoded automatically:
Note, the json parameter is ignored if either data or files is passed.
Using the json parameter in the request will change the Content-Type
in the header to application/json.
data = {"targetUrl":hookurl}

import requests
headers = {
'Content-Type': 'application/json',
}
data = '{"targetUrl":"https://example.com/lemlist-hook"}'
response = requests.post('https://api.lemlist.com/api/hooks', headers=headers, data=data, auth=('', '1234567980abcedf'))
You can visit this url:-
https://curl.trillworks.com/

Python requests: having a space in header for posting

I'm trying to post to a server using the following script:
import requests
data = {
'query': 'GetProcess',
'getFrom': '2018-12-06 10:10:10.000',
}
response = requests.post('http://localhost/monitor', data=data)
I cannot find where exactly, but the space character in the getFrom element is being replaced with a +: '2018-12-06+10:10:10.000'
This doesn't match the syntax SQL expects on our server, so the query fails.
I read here (https://stackoverflow.com/a/12528097) that setting the Content-type might help. I tried text/html, text/plain, application/json, and nothing seems to change.
Interestingly, the following (equivalent?) bash command succeeds:
curl -d 'query=GetProcess&getFrom=2018-12-06 10:10:10.000' localhost/monitor
I'm looking for a way to make my server receive "getFrom" : "2018-12-06 10:10:10.000" in the header.

I found a way to make this work: the problem I was having was due to the use of the urlencode function used in requests. In the requests documentation, it is shown how to go around this default behavior using PreparedRequests: http://docs.python-requests.org/en/master/user/advanced/#prepared-requests
Essentially, instead of using the requests.post() wrapper, make the function calls explicitly. This way, you will be able to control exactly what is sent. In my case, the solution was to do:
import requests
data = {
'query': 'GetProcess',
'getFrom': '2018-12-06 10:10:10.000'
}
s = requests.Session()
req = requests.Request('POST', 'http://'+ipAddress+'/monitor', data=data)
prepped = s.prepare_request(req)
prepped.body = prepped.body.replace("+", " ")
response = s.send(prepped)

How to view Terminal output when running Requests in Python?

I have a simple Python scripts that POSTs a local file to a given URL via Requests:
import requests
url = 'http://myWebsite.com/extension/extension/extension'
files = {'file': open("myLocalFile.csv")}
r = requests.post(url, files=files)
print r.headers
When I run a cURL command in Terminal that does the exact same thing:
curl -k -F docfile=#myLocalFile.csv http://myWebsite.com/extension/extension/extension
I get the output:
{"success":true, "data":{"uploaded":39, "errors":0, "unchanged":39, "skipped":0, "updated":0, "created":0, "failed":[]}, "numRows":1}
which indicates that I have successfully uploaded the file. How can I view this same output when I run my python script? I want to be able to parse through this output and check to see if "success" is true/false.
Sorry for the weird formatting, I tried to fix it but couldn't :(

print r.text
gives you the response body and then you can parse through the response body to check if success=true or false.

Not having used the library before the example on their home page seems helpful
python-requests.
More specifically:
print(r.json()) #possibly print(r.content)
# prints {u'private_gists': 419, u'total_private_repos': 77, ...}
Also see this question

Python requests module: urlencoding json data

I'm working on an API wrapper. The spec I'm trying to build to has the following request in it:
curl -H "Content-type:application/json" -X POST -d data='{"name":"Partner13", "email":"example#example.com"}' http://localhost:5000/
This request produces the following response from a little test server I setup to see exatly what headers/params etc are sent as. This little script produces:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data={"name":"Partner13", "email":"example#example.com"}
So that above is the result I want my python script to create when it hits the little test script.
I'm using the python requests module, which is the most beautiful HTTP lib I have ever used. So here is my python code:
uri = "http://localhost:5000/"
headers = {'content-type': 'application/json' }
params = {}
data = {"name":"Partner13", "email":"example#exmaple.com"}
params["data"] = json.dumps(data)
r = requests.post(uri, data=params, headers=headers)
So simple enough stuff. Set the headers, and create a dictionary for the POST parameters. That dictionary has one entry called "data" which is the JSON string of the data I want to send to the server. Then I call the post. However, the result my little test script gives back is:
uri: http://localhost:5000/,
method: POST,
api_key: None,
content_type: application/json,
params: None,
data: data=%7B%22name%22%3A+%22Partner13%22%2C+%22email%22%3A+%22example%40example.com%22%7D
So essentially the json data I wanted to send under the data parameter has been urlendcoded.
Does anyone know how to fix this? I have looked through the requests documentation and cannot seem to find a way to not auto urlencode the send data.
Thanks very much,
Kevin

When creating the object for the data keyword, simply assign a variable the result of json.dumps(data).
Also, because HTTP POST can accept both url parameters as well as data in the body of the request, and because the requests.post function has a keyword argument named "params", it might be better to use a different variable name for readability. The requests docs use the variable name "payload", so thats what I use.
data = {"name":"Partner13", "email":"example#exmaple.com"}
payload = json.dumps(data)
r = requests.post(uri, data=payload, headers=headers)

Requests automatically URL encodes dictionaries passed as data here. John_GG's solution works because rather than posting a dictionary containing the JSON encoded string in the 'data' field it simply passes the JSON encoded string directly: strings are not automatically encoded. I can't say I understand the reason for this behaviour in Requests but regardless, it is what it is. There is no way to toggle this behaviour off that I can find.
Best of luck with it, Kevin.

Http POST Curl in python

I'm having trouble understanding how to issue an HTTP POST request using curl from inside of python.
I'm tying to post to facebook open graph. Here is the example they give which I'd like to replicate exactly in python.
curl -F 'access_token=...' \
-F 'message=Hello, Arjun. I like this new API.' \
https://graph.facebook.com/arjun/feed
Can anyone help me understand this?

You can use httplib to POST with Python or the higher level urllib2
import urllib
params = {}
params['access_token'] = '*****'
params['message'] = 'Hello, Arjun. I like this new API.'
params = urllib.urlencode(params)
f = urllib.urlopen("https://graph.facebook.com/arjun/feed", params)
print f.read()
There is also a Facebook specific higher level library for Python that does all the POST-ing for you.
https://github.com/pythonforfacebook/facebook-sdk/
https://github.com/facebook/python-sdk

Why do you use curl in the first place?
Python has extensive libraries for Facebook and included libraries for web requests, calling another program and receive output is unecessary.
That said,
First from Python Doc
data may be a string specifying additional data to send to the server,
or None if no such data is needed. Currently HTTP requests are the
only ones that use data; the HTTP request will be a POST instead of a
GET when the data parameter is provided. data should be a buffer in
the standard application/x-www-form-urlencoded format. The
urllib.urlencode() function takes a mapping or sequence of 2-tuples
and returns a string in this format. urllib2 module sends HTTP/1.1
requests with Connection:close header included.
So,
import urllib2, urllib
parameters = {}
parameters['token'] = 'sdfsdb23424'
parameters['message'] = 'Hello world'
target = 'http://www.target.net/work'
parameters = urllib.urlencode(parameters)
handler = urllib2.urlopen(target, parameters)
while True:
if handler.code < 400:
print 'done'
# call your job
break
elif handler.code >= 400:
print 'bad request or error'
# failed
break