I was trying to retrieve data from an API and i was receiving
'set' object has no attribute 'items'
This is my api.py and i have import it to my views
import json
import urllib
import urllib2
import pycurl
def get_resources(request, filter, endpoint, lookup):
headers = {'X-Auth-Token:%s' % request.user.token, 'Content-Type:application/json'}
data = urllib.urlencode(filter)
url = endpoint+lookup
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
result = json.loads(response.read())
return result
and my views.py is like this
def indexView(request):
resources = json.dumps(get_resources(request,{}, api_endpoint, '/v2/meters'))
return HttpResponse(resources, mimetype='application/json')
I know that i was doing wrong here, I hope someone who could help me thanks.
The line:
headers = {'X-Auth-Token:%s' % request.user.token,
'Content-Type:application/json'}
defines a set. And it should probably be a dictionary (which has : somewhere in after which follows the value for the key before the :)
Try something like that:
headers = {'X-Auth-Token': request.user.token, 'Content-Type': 'application/json'}
Related
I'm trying to use the http.client library to gain access to EPC data through the use of the API found here;
https://epc.opendatacommunities.org/docs/api/domestic
I cannot get the data in the correct format, I have managed to get a 'bytes' variable, which seems to be html, but cannot get it to a readable dataframe.
from http.client import HTTPSConnection
from base64 import b64encode
import pandas as pd
def basic_auth(username, password):
token = b64encode(f"{username}:{password}".encode('utf-8')).decode("ascii")
return f'Basic {token}'
username = "ethan.mercer#aecom.com"
password = "bc6b6b549004ec082a1b8a3a03bd9d1adde3f02b"
c = HTTPSConnection("epc.opendatacommunities.org")
headers = { 'Authorization' : basic_auth(username, password) }
c.request('GET', '/api/v1/domestic/search', headers=headers)
res = c.getresponse()
data = res.read()
data = pd.read_html(data,skiprows=1)[0]
Output:ValueError: No tables found
Updated code:
import json
import requests
import pandas as pd
from base64 import b64encode
def basic_auth(username, password):
token = b64encode(f"{username}:{password}".encode('utf-8')).decode("ascii")
return f'Basic {token}'
username = "ethan.mercer#aecom.com"
password = "bc6b6b549004ec082a1b8a3a03bd9d1adde3f02b"
headers = {'Accept' : 'application/json', 'Authorization' : basic_auth(username, password) }
EPC_Data = requests.get('https://epc.opendatacommunities.org/api/v1/domestic/search?local-authority=E08000003', headers=headers).text
EPC_Data = json.loads(EPC_Data)
This code now works correctly, and is returning a dictionary of headers.
Because your data is in bytes try decoded it like this
data.decode("utf-8")
this should convert your data to an entire string. This is what you can give to pd.read_html (this only will work if there are <table> tags in it)
I'm new to Python. I'm trying to make a change in the Json body that I get in an exchange response using the requests library.
I want to do something like:
import json
import requests
def request_and_fill_form_in_response() -> requests.Response():
response = requests.get('https://someurl.com')
body_json = response.json()
body_json['some_field'] = 'some_value'
response.content = json.dumps(body_json)
return response
In this particular scenario I'm only interested of updating the response.content object (regardless of if it is a good practice or not).
Is this possible?
(btw, the code above throws 'AttributeError: can't set attribute' error, which is pretty much self-explanatory, but I want to make sure I'm not missing something)
You can rewrite the content in this way:
from json import dumps
from requests import get, Response
def request_and_fill_form_in_response() -> Response:
response = get('https://mocki.io/v1/a9fbda70-f7f3-40bd-971d-c0b066ddae28')
body_json = response.json()
body_json['some_field'] = 'some_value'
response._content = dumps(body_json).encode()
return response
response = request_and_fill_form_in_response()
print(response.json())
and the result is:
{'name': 'Aryan', 'some_field': 'some_value'}
but technically _content is a private variable and there must be a method as a setter to assign a value to it.
Also, you can create your own Response object too. (you can check the response methods here)
Alright, so I'm a little outside of my league on this one I think.
I'm attempting to facilitate custom HTTP headers what is noted here:
API-Key = API key
API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
from https://www.kraken.com/help/api
I'm trying to work solely out of urllib if at all possible.
Below is one of many attempts to get it encoded like required:
import os
import sys
import time
import datetime
import urllib.request
import urllib.parse
import json
import hashlib
import hmac
import base64
APIKey = 'ThisKey'
secret = 'ThisSecret'
data = {}
data['nonce'] = int(time.time()*1000)
data['asset'] = 'ZUSD'
uripath = '/0/private/TradeBalance'
postdata = urllib.parse.urlencode(data)
encoded = (str(data['nonce']) + postdata).encode()
message = uripath.encode() + hashlib.sha256(encoded).digest()
signature = hmac.new(base64.b64decode(secret),
message, hashlib.sha512)
sigdigest = base64.b64encode(signature.digest())
#this is purely for checking how things are coming along.
print(sigdigest.decode())
headers = {
'API-Key': APIKey,
'API-Sign': sigdigest.decode()
}
The above may be working just fine, where I'm struggling now is appropriately getting it to the site.
This is my most recent attempt:
myBalance = urllib.urlopen('https://api.kraken.com/0/private/TradeBalance', urllib.parse.urlencode({'asset': 'ZUSD'}).encode("utf-8"), headers)
Any help is greatly appreciated.
Thanks!
urlopen doesn't support adding headers, so you need to create a Request object and pass it to urlopen:
url = 'https://api.kraken.com/0/private/TradeBalance'
body = urllib.parse.urlencode({'asset': 'ZUSD'}).encode("utf-8")
headers = {
'API-Key': APIKey,
'API-Sign': sigdigest.decode()
}
request = urllib.request.Request(url, data=body, headers=headers)
response = urllib.request.urlopen(request)
I am writing a flask backend with flask sqlalchemy and marshmallow which returns json and kivy as frontend. I am trying to login in my flask api using kivy urlrequest but no data is been passed. I get error from my flask api ('no input data provided') even so i passed valid data.
from kivy.network.urlrequest import UrlRequest
import urllib
import json
def success(req, result):
print('success')
def fail(req, result):
print('fail')
def error(req, result):
print('error')
def progress(req, result, chunk):
print('loading')
values = {'email':'wilson#gmail.com', 'pwdhash':'wilson'}
params = urllib.urlencode(values)
headers = {'Content-type': 'application/x-www-form-urlencoded',
'Accept': 'text/plain'}
req = UrlRequest('http://127.0.0.1:5000/login', on_success=success, on_failure=fail, on_error=error, on_progress=progress, req_body=params, req_headers=headers)
req.wait()
print(req.result)
please how to fix this?
And to answer my question
My API only accepts json data and i never passed one just plain text. To fix this, i called json.dumps to convert the data into json and changed the Content-type from application/x-www-form-urlencoded to application/json
below is the fixed code:
from kivy.network.urlrequest import UrlRequest
import urllib
import json
def success(req, result):
print('success')
def fail(req, result):
print('fail')
def error(req, result):
print('error')
def progress(req, result, chunk):
print('loading')
values = {'email':'wilson#gmail.com', 'pwdhash':'wilson'}
# converted data to json type
params = json.dumps(values)
# changed Content-type from application/x-www-form-urlencoded to application/json
headers = {'Content-type': 'application/json',
'Accept': 'text/plain'}
req = UrlRequest('http://127.0.0.1:5000/login', on_success=success, on_failure=fail, on_error=error, on_progress=progress, req_body=params, req_headers=headers)
req.wait()
print(req.result)
I am trying to parse a response.text that I get when I make a request using the Python Requests library. For example:
def check_user(self):
method = 'POST'
url = 'http://localhost:5000/login'
ck = cookielib.CookieJar()
self.response = requests.request(method,url,data='username=test1&passwd=pass1', cookies=ck)
print self.response.text
When I execute this method, the output is:
{"request":"POST /login","result":"success"}
I would like to check whether "result" equals "success", ignoring whatever comes before.
The manual suggests: if self.response.status_code == requests.codes.ok:
If that doesn't work:
if json.loads(self.response.text)['result'] == 'success':
whatever()
Since the output, response, appears to be a dictionary, you should be able to do
result = self.response.json().get('result')
print(result)
and have it print
'success'
If the response is in json you could do something like (python3):
import json
import requests as reqs
# Make the HTTP request.
response = reqs.get('http://demo.ckan.org/api/3/action/group_list')
# Use the json module to load CKAN's response into a dictionary.
response_dict = json.loads(response.text)
for i in response_dict:
print("key: ", i, "val: ", response_dict[i])
To see everything in the response you can use .__dict__:
print(response.__dict__)
import json
def check_user(self):
method = 'POST'
url = 'http://localhost:5000/login'
ck = cookielib.CookieJar()
response = requests.request(method,url,data='username=test1&passwd=pass1', cookies=ck)
#this line converts the response to a python dict which can then be parsed easily
response_native = json.loads(response.text)
return self.response_native.get('result') == 'success'
I found another solution. It is not necessary to use json module. You can create a dict using dict = eval(whatever) and return, in example, dict["result"]. I think it is more elegant. However, the other solutions also work and are correct
Put in the return of your method like this:
return self.response.json()
If you wanna looking for more details, click this following link:
https://www.w3schools.com/python/ref_requests_response.asp
and search for json() method.
Here is an code example:
import requests
url = 'https://www.w3schools.com/python/demopage.js'
x = requests.get(url)
print(x.json())
In some cases, maybe the response would be as expected. So It'd be great if we can built a mechanism to catch and log the exception.
import requests
import sys
url = "https://stackoverflow.com/questions/26106702/how-do-i-parse-a-json-response-from-python-requests"
response = requests.get(url)
try:
json_data = response.json()
except ValueError as exc:
print(f"Exception: {exc}")
# to find out why you have got this exception, you can see the response content and header
print(str(response.content))
print(str(response.headers))
print(sys.exc_info())
else:
if json_data.get('result') == "success":
# do whatever you want
pass