Would anybody be able to help me identify where I am going wrong with a very basic AQL I am trying to execute using Python 3 and the 'Requests' library?
No matter what I do I cant seem to get past a 400: Bad Request. It is clearly something to do with the formatting of the data I am trying to Post but just can't see it. I'm assuming I want to pass it as a string as when posting sql queries they have to be in plain text.
Just as an aside, I can execute this query absolutely fine using postman etc.
import requests
from requests.auth import HTTPBasicAuth
import json
HEADERS = {'Content-type': 'text'}
DATA = 'items.find({"type":"folder","repo":{"$eq":"BOS-Release-Builds"}}).sort({"$desc":["created"]}).limit(1)'
response = requests.post('https://local-Artifactory/artifactory/api/search/aql', headers=HEADERS, auth=HTTPBasicAuth('user', 'password'), data = DATA)
print (response.status_code)
##print (response.json())
def jprint(obj):
text = json.dumps(obj, sort_keys=True, indent=4)
print (text)
jprint(response.json())
print(response.url)
print(response.encoding)
Ok, after sitting staring at the code for another few minutes I spotted my mistake.
I shouldnt have defined the Header content type in the request and instead let the 'Requests' library deal with this for you.
So the code should look like:
import requests
from requests.auth import HTTPBasicAuth
import json
DATA = 'items.find({"type":"folder","repo":{"$eq":"BOS-Release-Builds"}}).sort({"$desc":["created"]}).limit(1)'
response = requests.post('https://uk-artifactory.flowbird.group/artifactory/api/search/aql', auth=HTTPBasicAuth('user', 'password!'), data = DATA)
print (response.status_code)
##print (response.json())
def jprint(obj):
text = json.dumps(obj, sort_keys=True, indent=4)
print (text)
jprint(response.json())
print(response.url)
print(response.encoding)
Related
Can someone please let me know how to make requests to Wit.ai message api. I am struggling with my code.
import requests
import json
import sys
from wit import Wit
# Wit speech API endpoint
API_ENDPOINT = 'https://api.wit.ai/message'
q='who are you'
# Wit.ai api access token
wit_access_token = 'B3GHXHLTXIASO7S4KY7UC65LMSTCDEHK'
# defining headers for HTTP request
headers = {'authorization': 'Bearer ' + wit_access_token}
# making an HTTP post request
resp = requests.post(API_ENDPOINT, headers = headers,data = {'q':'who are you'})
# converting response content to JSON format
data = json.loads(resp.content)
print(data)
I am getting back this:
{u'code': u'json-parse', u'error': u'Invalid JSON'}
The /message endpoint of Wit API accepts only GET method and expects the query parameters in URL (not data in request body). The requests library will correct the case on your lowercase Authorization header field, but it's a good practice to write it according to the standard. Also, JSON response can be fetched decoded with json() method on Response.
With all of this:
import requests
API_ENDPOINT = 'https://api.wit.ai/message'
WIT_ACCESS_TOKEN = 'B3GHXHLTXIASO7S4KY7UC65LMSTCDEHK'
headers = {'Authorization': 'Bearer {}'.format(WIT_ACCESS_TOKEN)}
query = {'q': 'who are you'}
resp = requests.get(API_ENDPOINT, headers=headers, params=query)
data = resp.json()
print(data)
Note however, Wit has a Python library which abstracts all of these low-level details, and makes your code much simpler and easier to read. Use it.
It would look like this (example from the docs):
from wit import Wit
client = Wit(access_token=WIT_ACCESS_TOKEN)
resp = client.message('what is the weather in London?')
print('Yay, got Wit.ai response: ' + str(resp))
Trying to output my response from Microsoft's Computer Vision API to a .json file, it works with all of the other APIs I've been using so far. With the code below, directly from Microsoft's documentation, I get an error:
Error: the JSON object must be str, not 'bytes'
Removing the parsed = json.loads(data) and using print(json.dumps(data, sort_keys=True, indent=2)) prints out the information for the image that I want, but also says Error and is prefixed with
b
denoting it's in bytes and ending with
is not JSON serializable
I'm just trying to find out how I can get the response into a .json file like i'm able to do with other APIs and am at a loss for how I can possible convert this in a way that will work.
import http.client, urllib.request, urllib.parse, urllib.error, base64, json
API_KEY = '{API_KEY}'
uri_base = 'westus.api.cognitive.microsoft.com'
headers = {
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': API_KEY,
}
params = urllib.parse.urlencode(
{
'visualFeatures': 'Categories, Description, Color',
'language': 'en',
}
)
body = "{'url': 'http://i.imgur.com/WgPtc53.jpg'}"
try:
conn = http.client.HTTPSConnection(uri_base)
conn.request('POST', '/vision/v1.0/analyze?%s' % params, body, headers)
response = conn.getresponse()
data = response.read()
# 'data' contains the JSON data. The following formats the JSON data for display.
parsed = json.loads(data)
print ("Response:")
print (json.dumps(parsed, sort_keys=True, indent=2))
conn.close()
except Exception as e:
print('Error:')
print(e)
Shortly after posting the question, I realized I had missed looking for something: just converting bytes to a string.
found this Convert bytes to a Python string
and was able to modify my code to:
parsed = json.loads(data.decode('utf-8'))
And it seems to have resolved my issue. Now error-free and able to export to .json file like I needed.
I'm trying to post a json file to influxdb on my local host. This is the code:
import json
import requests
url = 'http://localhost:8086/write?db=mydb'
files ={'file' : open('sample.json', 'rb')}
r = requests.post(url, files=files)
print(r.text)
This is what sample.json looks like:
{
"region" : "eu-west-1",
"instanceType": "m1.small"
}
My response gives the following errors:
{"error":"unable to parse '--1bee44675e8c42d8985e750b2483e0a8\r':
missing fields\nunable to parse 'Content-Disposition: form-data;
name=\"file\"; filename=\"sample.json\"\r': invalid field
format\nunable to parse '\r': missing fields\nunable to parse '{':
missing fields\nunable to parse '\"region\" : \"eu-west-1\",': invalid
field format\nunable to parse '\"instanceType\": \"m1.small\"': invalid
field format\nunable to parse '}': missing fields"}
My json seems to be a valid json file. I am not sure what I am doing wrong.
I think that the fault maybe is that you just open the file but not read it. I mean since you want to post the content of the json object which is stored on the file, and not the file itself, it may be better to do that instead:
import json
import requests
url = 'http://localhost:8086/write?db=mydb'
json_data = open('sample.json', 'rb').read() # read the json data from the file
r = requests.post(url, data=json_data) # post them as data
print(r.text)
which is actually your code modified just a bit...
Writing data with JSON was deprecated for performance reasons and has since been removed.
See GitHub issue comments 107043910.
I am trying to connect to a page that takes in some values and return some data in JSON format in Python 3.4 using urllib. I want to save the values returned from the json into a csv file.
This is what I tried...
import json
import urllib.request
url = 'my_link/select?wt=json&indent=true&f=value'
response = urllib.request.Request(url)
response = urllib.request.urlopen(response)
data = response.read()
I am getting an error below:
urllib.error.HTTPError: HTTP Error 505: HTTP Version Not Supported
EDIT: Found a solution to my problem. I answered it below.
You have found a server that apparently doesn't want to talk HTTP/1.1. You could try lying to it by claiming you are using a HTTP/1.0 client instead, by patching the http.client.HTTPConnection class:
import http.client
http.client.HTTPConnection._http_vsn = 10
http.client.HTTPConnection._http_vsn_str = 'HTTP/1.0'
and re-trying your request.
I used FancyURLopener and it worked. Found this useful: docs.python.org: urllib.request
url_request = urllib.request.FancyURLopener({})
with url_request.open(url) as url_opener:
json_data = url_opener.read().decode('utf-8')
with open(file_output, 'w', encoding ='utf-8') as output:
output.write(json_data)
Hope this helps those having the same problems as mine.
I have been working on solving an HTTP 500 (bad syntax/string) error for far too long, and after doing some searching I cannot find a solution anywhere. I have a nested json PUT request that I have been able to make work using a couple API tools (both browser extensions and stand-alone programs), but when I try to use the json in Python's HTTP Requests module, I keep getting the 500 error code returned.
I have gotten other, simplier jsons (e.g. data={"RequestID": "71865"}) to work using similar code to the following, which leaves me to believe something is not getting formatted correctly, and I am unfortunately too new to this json-python thing to figure it out. I think the issue is because of the way python handles the nested json.
# -*- coding: utf-8 -*-
#!/usr/bin/env python
import requests
import json
USER_NAME=u"myusername"
USER_PASS=u"mypassword"
PUT_URL="https://webservice.url.com/A/Path/To/Create/"
headers = {"Content-Type": "application/json"}
data = {
"ListOfFields": {
"Field": [
{"fieldname": "summary","value": "test summary"},
{"fieldname": "notes","value": "an example json PUT"},
{"fieldname": "user","value": "myuser"}
]
}
}
data_json = json.dumps(data)
payload = {'json_playload': data_json } ## I have tried with and without this line.
r = requests.put('{}'.format(PUT_URL), data=data_json, headers=headers, auth=(USER_NAME, USER_PASS), timeout=10)
# r = requests.put('{}'.format(PUT_URL), data=payload, headers=headers, auth=(USER_NAME, USER_PASS), timeout=10)
I have tried putting the data value into quotes, a single line, and making some other slight tweaks, but I keep getting the 500 error.
print(r.status_code)
>> 500
As mentioned before, I have gotten similar code to work in python using GET and POST and the same web server, but this one is giving me a headache!
The Requests library has a nasty habit of clobbering data when passing in nested JSON to the data param. To avoid this, pass it into the json param instead:
r = requests.put(PUT_URL, json=data_json, headers=headers, auth=(USER_NAME, USER_PASS), timeout=10)
For more details, take a look at this answer to a similar question: Post JSON using Python Requests
Do you want to pretty print your JSON data? Try this:
data_json = json.dumps(data, sort_keys=True, indent=4, separators=(',', ': '))
See https://docs.python.org/2/library/json.html