I am using anki-connect to communicate with Anki, a spaced repetition software.
In readme.md, it uses following command to get deck name.
curl localhost:8765 -X POST -d "{\"action\": \"deckNames\", \"version\": 5}"
It works right in my Windows system.
How can I use python instead of cURL?
I've tried this but get no luck.
import requests
r = requests.post("http://127.0.0.1:8765", data={'action': 'guiAddCards', 'version': 5})
print(r.text)
When creating request you should:
provide Content-Type header
provide data in format that matches Content-Type header
make sure application supports the format
Both curl and python examples you gave sends request with Content-Type: application/x-www-form-urlencoded, the default one. The difference is curl passes string and python passes an array.
Let's compare curl and requests and what is really posted:
Curl
$ curl localhost -X POST -d "{\"action\": \"deckNames\", \"version\": 5}"
Headers:
Host: localhost
User-Agent: curl/7.52.1
Accept: */*
Content-Length: 37
Content-Type: application/x-www-form-urlencoded
Posted data:
[
'{"action": "deckNames", "version": 5}'
]
Python
import requests
r = requests.post("http://127.0.0.1", data={'action': 'guiAddCards', 'version': 5})
print(r.text)
Headers:
Host: 127.0.0.1
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.10.0
Content-Length: 28
Content-Type: application/x-www-form-urlencoded
Posted data:
[
'action' -> 'guiAddCards',
'version' -> '5',
]
As you can see, incorrect post data format breaks your app.
To be sure, that posted JSON data will be properly read by application you should make requests like that:
Curl
$ curl localhost:8765 -H 'Content-Type: application/json' -d '{"action": "deckNames", "version": 5}'
Python
import requests
r = requests.post("http://127.0.0.1:8765", json={'action': 'guiAddCards', 'version': 5})
print(r.text)
I've tried following after digging and this works.
Can anybody share the reason. Thanks.
import requests
import json
#r = requests.post("http://127.0.0.1:8765", data={'action': 'guiAddCards', 'version': 5})
r = requests.post('http://localhost:8765', data=json.dumps({'action': 'guiAddCards', 'version': 5}))
print(r.text)
This is a reply to user2444791's answer. I can't reply with a comment because I don't have the reputation to comment (I'm new, please forgive a breech of etiquette!)
Without the exact error message, it's hard to be sure, but...
Looking at the Anki Connect API, it expects its POST-ed data to be a single string which contains a JSON object, not a key/value dictionary equivalent to that JSON object.
Every request consists of a JSON-encoded object containing an action, a version, and a set of contextual params.
Their example code (in Javascript): xhr.send(JSON.stringify({action, version, params}));
It might be as simple as sending your data in the wrong format. In the first example, you are sending a dictionary with the key/vale pairs already parsed. In the second example, you're sending a string for them to parse instead.
Related
After writing a file with the snippet below
with open("temp.trig", "wb") as f:
f.write(data)
I use curl to load it into the server
curl -X POST -F file=#"temp.trig" -H "Accept: application/json" http://localhost:8081/demo/upload
which works fine.
I am trying to replace the curl with python requests, as follows:
with open("temp.trig", "rb") as f:
result = requests.post("http://localhost:8081/demo/upload", files={'file': f},
headers = {"Accept": "application/json"})
which attempted to follow the curl as closely as possible. This code results in an error 500 from the server. I suspect it must be something related to the request, because the same server is ok via `curl. Any ideas?
There probably is nothing wrong with your python script.
Differences I've noticed between curl and requests are the following:
obviously, User-Agent headers are different — curl/7.47.0 vs. python-requests/2.22.0
multipart boundary format in Content-Type header is different — ------------------------6debaa3504bbc177 in curl vs. c1e9f4f617de4d0dbdb48fcc5aab67e0 in requests
therefore Content-Length value will almost certainly be different
multipart/form-data format in body is slightly different — curl adds an extra line (Content-Type: text/plain) before file contents
So depending on your file format, server may not be able to parse requests HTTP request format.
I think the best solution for you now is to compare raw HTTP requests from curl and requests and find what differences are significant.
For example:
Open terminal
Launch netcat with nc -l -p 1234 command. This will listen to HTTP requests on localhost on port 1234 and output raw HTTP requests to terminal.
Send your curl request as it is to localhost:1234 in another tab
Execute your python script as it is using URL localhost:1234 in another tab
Compare raw requests from your netcat output
Here's my attempt:
import requests
headers = {
'Accept': 'application/json',
}
files = {
'file': ('temp.trig', open('temp.trig', 'rb')),
}
response = requests.post('http://localhost:8081/demo/upload', headers=headers, files=files)
In case this doesn't work we really need to read more data on the server side, as Ivan Vinogradov explained well.
I am trying to rewrite a curl command to python. The idea is the following: we upload an audio file, and get in return what it was said in it in text - e.g. speech to text.
The site has provided us a curl example, which makes a post request and in return receives a simple response.
When I tried to convert the curl request into python one (with little help from https://curl.trillworks.com/ ) I get ConnectionError number 104, connection reset by peer.
I strongly suspect that this happens because when I make the connection vie curl firstly I get the response: < HTTP/1.1 100 Continue, and after short time of waiting another response: < HTTP/1.1 200 OK, with the some sample data. I think that python requests.post just hangs at the first HTTP/1.1 100.
I could not handle or reset the connection using try... except..., neither succeeded in looping it with time.sleep().
Any ideas?
P.S. CODE:
Curl command:
curl "LINK TO SERVER" -H "Content-Type: audio/x-speex;rate=16000" -H "Accept-Language: ENUS" -H "Transfer-Encoding: chunked" -H "Accept: application/xml" -H "Accept-Topic: Dictation" -k --data-binary #audio_16k16bit.pcm
Python equivalent:
#!/bin/python
import requests
import time
headers = {
'Content-Type': 'audio/x-wav;codec=pcm;bit=16;rate=16000',
'Accept-Language': 'ENUS',
'Transfer-Encoding': 'chunked',
'Accept': 'application/xml',
'Accept-Topic': 'Dictation',
}
params = (
('appId', ''),
('appKey',''),
('id', ''),
)
data = open('audio_16k16bit.pcm', 'rb').read()
r = requests.post('LINK TO SERVER', headers=headers, data=data, verify=False)
print(r.content)
SOLUTION
Should remove and let the server determine the type of file, also since we are sending it as one file chunked is not needed:
'Content-Type': 'audio/x-wav;codec=pcm;bit=16;rate=16000',
'Transfer-Encoding': 'chunked'
I am trying to form a web payload for a particular request body but unable to get it right. What I need is to pass my body data as below
data={'file-data':{"key1": "3","key2": "6","key3": "8"}}
My complete payload request looks like this
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
However, when I pass this, python tries to parse each individual key value and assigns to the 'file-data' key like this
file-data=key1
file-data=key2
file-data=key3
and so on for as many keys I pass within the nested dictionary. The requirement however, is to pass the entire dictionary as a literal content like this(without splitting the values by each key):
file-data={"key1": "3","key2": "6","key3": "8"}
The intended HTTP trace should thus ideally look like this:
POST /sample_URL/ HTTP/1.1
Host: sample_host.com
Authorization: Basic XYZ=
Cache-Control: no-cache
Content-Type: multipart/form-data; boundary=----UVWXXXX
------WebKitFormBoundaryXYZ
Content-Disposition: form-data; name="file-data"
{"key1": "3","key2": "6","key3":"8" }
------WebKitFormBoundaryMANZXC
Content-Disposition: form-data; name="file"; filename=""
Content-Type:
------WebKitFormBoundaryBNM--
As such, I want to use this as part of a payload for a POST request(using python requests library). Any suggestions are appreciated in advance-
Edit1: To provide more clarity, the API definition is this:
Body
Type: multipart/form-data
Form Parameters
file: required (file)
The file to be uploaded
file-data: (string)
Example:
{
"key1": "3",
"key2": "6",
"key3": "8"
}
The python code snippet I used(after checking suggestions) is this:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
url = "https://sample_url/upload"
filepath='mypath'
filename='logo.png'
f=open(filepath+'\\'+filename)
filedata={'file-data':"{'key1': '3','key2': '6','key3': '8'}"}
base64string = encodestring('%s:%s' % ('user', 'password').replace('\n', '')
headers={'Content-type': 'multipart/form-data','Authorization':'Basic %s' % base64string}
r = requests.post(url=url,headers=headers,data=filedata,files={'file':f})
print r.text
The error I get now is still the same as shown below:
{"statusCode":400,"errorMessages":[{"severity":"ERROR","errorMessage":"An exception has occurred"]
It also says that some entries are either missing or incorrect. Note that I have tried passing the file parameter after opening it in binary mode as well but it throws the same error message
I got the HTTP trace printed out via python too and it looks like this:
send: 'POST sample_url HTTP/1.1
Host: abc.com
Connection: keep-alive
Accept-Encoding: gzip,deflate
Accept: */*
python-requests/2.11.1
Content-type: multipart/form-data
Authorization: Basic ABCDXXX=
Content-Length: 342
--CDXXXXYYYYY
Content-Disposition:form-data; name="file-data"
{\'key1\': \'3\',\'key2\': \'6\'
,\'key3\': \'8\'}
--88cdLMNO999999
Content-Disposition: form-data; name="file";
filename="logo.png"\x89PNG\n\r\n--cbCDEXXXNNNN--
If you want to post JSON with python requests, you should NOT use data but json:
r = requests.post('http://httpbin.org/post', json={"key": "value"})
I can only guess that you are using data because of your example
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
Whis is not valid python syntax btw.
I'm sure I'm doing something simple wrong, but I can't for the life of me figure out how to set the "IsRead" property to true. It's the last step of my process that gets a filtered list of messagesa and stores and processes any attachments.
According to the docs "IsRead" is writable: http://msdn.microsoft.com/office%5Coffice365%5CAPi/complex-types-for-mail-contacts-calendar#ResourcesMessage
http://msdn.microsoft.com/office%5Coffice365%5CAPi/mail-rest-operations#MessageoperationsUpdatemessages
I'm using python 2.7 and the requests module:
# once file acquired mark the email as read
params = {'IsRead':'True'}
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, params, auth=(email,pwd))
log.debug( response )
The response that comes back is this:
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}
What's the problem with my request?
At first glance it looks OK. I wonder if the Content-Type header isn't being set to "application/json" or something along those lines. Try getting a network trace and verify that the request looks something like:
PATCH https://outlook.office365.com/api/v1.0/Me/Messages('msgid') HTTP/1.1
Accept: application/json;odata.metadata=full
Authorization: Bearer <token>
Content-Type: application/json;odata.metadata=full
Host: outlook.office365.com
Content-Length: 24
Expect: 100-continue
Connection: Keep-Alive
{
"IsRead": "true"
}
Well I have an answer for myself and it is indeed a simple matter.
It was a mistake to not fully read how PATCH is different from GET or POST.
In short it's important to make sure your headers are set for the right content-type.
Here is the working code:
# once file acquired mark the email as read
changes = {u'IsRead':u'True'}
headers = {'Content-Type': 'application/json'}
json_changes = json.dumps(changes)
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, data=json_changes, auth=__AUTH, headers=headers)
log.debug( response )
I'm trying to use the requests module to send a csv file to an API that uploads data into a database. Since the data is going into a database, the API is configured to reject files that have an unrecognized column name. The accepted columns are "id", "artist", "video". I have a test.csv file with just 1 row of data:
id,artist,video
1,The Shins,Phantom Limb
When I send the file to the api with the following curl request, it goes through successfully.
curl -i -u myUser:myPassword -X POST -T .\test.csv "http://destination.com/api/endpoint/create-or-update-records"
Here's the curl response message:
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Sat, 11 Oct 2014 14:47:51 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 0
Connection: close
However, when I try to send the file using the requests post method like this:
url = "http://destination.com/api/endpoint/create-or-update-records"
files = {'file': open("test.csv", "rb")}
r = requests.post(url, files=files, auth=("myUser","myPassword"))
The response I get back is this:
Unknown fields: '--5a6f03307ed74747904844625f76a82e'. Valid fields are: 'id', 'artist', 'video'
If I send the file again, I get the same message, but the "--lotsofcharacters" is now a difference set of characters.
I'm guessing I'm missing a setting or something, but I have combed the requests API and can't figure out what it is. What is different between the curl request and the requests request that is causing one to fail, and the other to go through?
You're not posting plain text to your server with requests. The documentation explicitly states that you use the files parameter when you wish to perform a multipart/form-data upload to the server. In this case all you need to do is
with open('test.csv', 'rb') as csv_file:
r = requests.post(url, data=csv_file, headers={'Content-Type': 'text/plain'}, auth=('user', 'password'))