Alright, so I'm a little outside of my league on this one I think.
I'm attempting to facilitate custom HTTP headers what is noted here:
API-Key = API key
API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
from https://www.kraken.com/help/api
I'm trying to work solely out of urllib if at all possible.
Below is one of many attempts to get it encoded like required:
import os
import sys
import time
import datetime
import urllib.request
import urllib.parse
import json
import hashlib
import hmac
import base64
APIKey = 'ThisKey'
secret = 'ThisSecret'
data = {}
data['nonce'] = int(time.time()*1000)
data['asset'] = 'ZUSD'
uripath = '/0/private/TradeBalance'
postdata = urllib.parse.urlencode(data)
encoded = (str(data['nonce']) + postdata).encode()
message = uripath.encode() + hashlib.sha256(encoded).digest()
signature = hmac.new(base64.b64decode(secret),
message, hashlib.sha512)
sigdigest = base64.b64encode(signature.digest())
#this is purely for checking how things are coming along.
print(sigdigest.decode())
headers = {
'API-Key': APIKey,
'API-Sign': sigdigest.decode()
}
The above may be working just fine, where I'm struggling now is appropriately getting it to the site.
This is my most recent attempt:
myBalance = urllib.urlopen('https://api.kraken.com/0/private/TradeBalance', urllib.parse.urlencode({'asset': 'ZUSD'}).encode("utf-8"), headers)
Any help is greatly appreciated.
Thanks!
urlopen doesn't support adding headers, so you need to create a Request object and pass it to urlopen:
url = 'https://api.kraken.com/0/private/TradeBalance'
body = urllib.parse.urlencode({'asset': 'ZUSD'}).encode("utf-8")
headers = {
'API-Key': APIKey,
'API-Sign': sigdigest.decode()
}
request = urllib.request.Request(url, data=body, headers=headers)
response = urllib.request.urlopen(request)
Related
I'm trying to use the http.client library to gain access to EPC data through the use of the API found here;
https://epc.opendatacommunities.org/docs/api/domestic
I cannot get the data in the correct format, I have managed to get a 'bytes' variable, which seems to be html, but cannot get it to a readable dataframe.
from http.client import HTTPSConnection
from base64 import b64encode
import pandas as pd
def basic_auth(username, password):
token = b64encode(f"{username}:{password}".encode('utf-8')).decode("ascii")
return f'Basic {token}'
username = "ethan.mercer#aecom.com"
password = "bc6b6b549004ec082a1b8a3a03bd9d1adde3f02b"
c = HTTPSConnection("epc.opendatacommunities.org")
headers = { 'Authorization' : basic_auth(username, password) }
c.request('GET', '/api/v1/domestic/search', headers=headers)
res = c.getresponse()
data = res.read()
data = pd.read_html(data,skiprows=1)[0]
Output:ValueError: No tables found
Updated code:
import json
import requests
import pandas as pd
from base64 import b64encode
def basic_auth(username, password):
token = b64encode(f"{username}:{password}".encode('utf-8')).decode("ascii")
return f'Basic {token}'
username = "ethan.mercer#aecom.com"
password = "bc6b6b549004ec082a1b8a3a03bd9d1adde3f02b"
headers = {'Accept' : 'application/json', 'Authorization' : basic_auth(username, password) }
EPC_Data = requests.get('https://epc.opendatacommunities.org/api/v1/domestic/search?local-authority=E08000003', headers=headers).text
EPC_Data = json.loads(EPC_Data)
This code now works correctly, and is returning a dictionary of headers.
Because your data is in bytes try decoded it like this
data.decode("utf-8")
this should convert your data to an entire string. This is what you can give to pd.read_html (this only will work if there are <table> tags in it)
According to the documentation for the Cosmos DB REST API, with every API call, the Authorization header must be set. The value for this is constructed as described here: https://learn.microsoft.com/en-us/rest/api/cosmos-db/access-control-on-cosmosdb-resources
I am implementing this in Python as follows:
def get_authorisation_token(verb, resource_type, resource_id, date, master_key):
key = base64.b64decode(master_key)
text = f"""{verb.lower()}\n{resource_type.lower()}\n{resource_id.lower()}\n{date.lower()}\n\n"""
text_encoded = text.encode('utf-8')
signature_hash = hmac.new(key, text_encoded, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(signature_hash).decode()
key_type = 'master'
version = '1.0'
uri = f'type={key_type}&ver={version}&sig={signature}'
uri_encoded = urllib.parse.quote(uri)
return uri_encoded
Since this is sent with every call, the auth token needs to be re-created to match the request URL. So, for example to get a list of databases, one must provide the resource type to be dbs and the resource link/ID to be an empty string with the URL: https://{databaseaccount}.documents.azure.com/dbs/
The part I cannot figure out, is the correct combination of resource type and resource ID/link to get all users from a particular database. Documentation can be found here: https://learn.microsoft.com/en-us/rest/api/cosmos-db/list-users
I have tried some combinations but nothing returns the users, I just get a 401:
{
"code": "Unauthorized",
"message": "The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'get\nusers\ndbs/<db_name>\nmon, 09 nov 2020 23:37:24 gmt\n\n'\r\nActivityId: 697a4159-f160-4aab-ae90-6cb5eaadb710, Microsoft.Azure.Documents.Common/2.11.0"
}
Regarding the issue, please refer to the following code
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
import base64
from urllib.parse import quote
import hmac
from hashlib import sha256
import requests
from azure.cosmos.auth import GetAuthorizationHeader
from azure.cosmos.cosmos_client import CosmosClientConnection
master_key = ''
database_name = ''
key = base64.b64decode(master_key)
verb = 'GET'
resource_type = 'users'
resource_id = f'dbs/{database_name}'
now = datetime.now()
stamp = mktime(now.timetuple())
date = format_date_time(stamp)
print(date)
text = "{verb}\n{resource_type}\n{resource_id}\n{date}\n{other}\n".format(
verb=(verb.lower() or ''),
resource_type=(resource_type.lower() or ""),
resource_id=(resource_id or ""),
date=date.lower(),
other="".lower())
body = text.encode("utf-8")
digest = hmac.new(key, body, sha256).digest()
signature = base64.encodebytes(digest).decode("utf-8")
key_type = 'master'
version = '1.0'
uri = f'type={key_type}&ver={version}&sig={signature[:-1]}'
uri_encoded = quote(uri)
url = "https://<>.documents.azure.com:443/dbs/<>/users"
payload = {}
headers = {
'Authorization': uri_encoded,
'x-ms-date': date,
'x-ms-version': '2018-12-31'
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
I am trying write a python script using requests package to use an online mongodb query service API hosted within the organization. The API expects the authorization header in the format 'websitename/username:Password' and using the basic authentication base64 encoding. I tried to create the GET request using the requests package which has the authorization header in the following format:
import requests
headers = {'Authorization': 'Basic %s' % 'Base64encoded
websitename/username:Password string here'}
content_res = requests.get(get_url, headers = headers).json()
But I am getting a parse error here for the string as I think the expected string for header in requests package is in form of 'username:password' here and not the desired format i.e. 'websitename/username:password'.
Is there a way in which I could use the base64 encoded sting in the format which the service is expecting i.e. 'websitename/username:password' in requests package?
Any help is highly appreciated.
Thanks.
It sounds to me like you are getting a HTTP response error because the authorization header value you are passing is not base64 encoded. To correct this you can simply encode the string using the base64 python module:
Python 2.7 https://docs.python.org/2/library/base64.html
Python 3.5 https://docs.python.org/3.5/library/base64.html
An example would be something like this:
import base64
import requests
website = 'example.com'
username = 'root'
password = '1234'
auth_str = '%s/%s:%s' % (website, username, password)
b64_auth_str = base64.b64encode(auth_str)
headers = {'Authorization': 'Basic %s' % b64_auth_str}
content_res = requests.get(get_url, headers=headers).json()
import base64
import requests
website = 'example.com'
username = 'root'
password = '1234'
auth_str = '%s/%s:%s' % (website, username, password)
b64_auth_str = base64.b64encode(auth_str.encode('ascii'))
headers = {'Authorization': 'Basic %s' % b64_auth_str}
content_res = requests.get(get_url, headers=headers).json()
First attempt at connecting to a Bitcoin exchange's private API and I am already stuck in trying to do a Test call with my code.
from urllib2 import Request, urlopen
from urllib import urlencode
import datetime
api_key = "myAPIkey"
api_secret = "mySercetKey"
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
values = urlencode({"timestamp" : timestamp})
headers = {"Content-Type": "application/x-www-form-urlencoded", "key": api_key, "sig": api_secret}
request = Request("https://www.fybsg.com/api/SGD/test", data=values, headers=headers)
response_body = urlopen(request).read()
print response_body
Here's what was returned from response_body:
{"error":"An unexpected error occurred, its probably your fault, go read the docs."}
Can a kind soul point out what's wrong with my code? (I've got a feeling it's seriously wrong)
The API documentation for the bitcoin exchange can be found here. (Test function)
You were passing invalid timestamp, in the API documentation they have mentioned that the timestamp have to be Current Unix timestamp which can be achieved like this:-
timestamp = datetime.datetime.now()
timestamp = int(time.mktime(timestamp.timetuple()))
OR just:
import time
timestamp= int(time.time())
So after updating your code
from urllib2 import Request, urlopen
from urllib import urlencode
import datetime
import time
api_key = "myAPIkey"
api_secret = "mySercetKey"
timestamp = datetime.datetime.now() #.strftime('%Y-%m-%d %H:%M:%S')
timestamp = int(time.mktime(timestamp.timetuple()))
print timestamp
values = urlencode({"timestamp" : timestamp})
#sig - HMAC-SHA1 signature of POST Data with Key's Secret
from hashlib import sha1
import hmac
hashed = hmac.new(values, api_secret, sha1)
hashed_value = hashed.digest().encode("base64").rstrip('\n')
headers = {"Content-Type": "application/x-www-form-urlencoded",
"key": api_key, "sig":hashed_value}
request = Request("https://www.fybsg.com/api/SGD/test", data=values, headers=headers)
response_body = urlopen(request).read()
print response_body
I'm getting this response:-
{"error":"Invalid API Key or account number"}
Which I think you can fix by using your valid private key or account number.
I was trying to retrieve data from an API and i was receiving
'set' object has no attribute 'items'
This is my api.py and i have import it to my views
import json
import urllib
import urllib2
import pycurl
def get_resources(request, filter, endpoint, lookup):
headers = {'X-Auth-Token:%s' % request.user.token, 'Content-Type:application/json'}
data = urllib.urlencode(filter)
url = endpoint+lookup
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
result = json.loads(response.read())
return result
and my views.py is like this
def indexView(request):
resources = json.dumps(get_resources(request,{}, api_endpoint, '/v2/meters'))
return HttpResponse(resources, mimetype='application/json')
I know that i was doing wrong here, I hope someone who could help me thanks.
The line:
headers = {'X-Auth-Token:%s' % request.user.token,
'Content-Type:application/json'}
defines a set. And it should probably be a dictionary (which has : somewhere in after which follows the value for the key before the :)
Try something like that:
headers = {'X-Auth-Token': request.user.token, 'Content-Type': 'application/json'}