I need to access an API that uses JSON Web Tokens as their authentication method. Is there a good way to use a python code step to create this token then add that token as a header to a custom request webhook step?
My experience authenticating with APIs has been using the simple API key method. As such I first read your question and didn't fully understand. I decided to do some research and hopefully learn something along the way, and I certainly did. I share my findings and answer below:
For starters I began reading into JSON Web Tokens(JWT) which lead me to the JWT website, which was an excellent resource. It very clearly spells out the components that make up a JWT and how they need to be formatted, I would highly recommend having a look.
From the JWT website I found that a JWT is made up of three components:
A base64 URL safe encoded header.
A base64 URL safe encoded payload.
A base64 URL safe encoded signature.
All three of the above combined form the correctly formatted JWT. Fortunately the JWT website has a list of libraries made for Python. Unfortunately none of these third-party libraries are available in the vanilla Python offered by the Zapier code module. To get this done required reading some source code and leveraging what libraries we do have available. So after a few hours and lots of trial and error I was able to come up with the following solution for generating a correctly formatted JWT:
import hashlib
import hmac
import requests
from base64 import urlsafe_b64encode
def base64url_encode(payload):
if not isinstance(payload, bytes):
payload = payload.encode('utf-8')
encode = urlsafe_b64encode(payload)
return encode.decode('utf-8').rstrip('=')
def generate_JWT(header, payload, secret):
encoded_header = base64url_encode(header)
encoded_payload = base64url_encode(payload)
signature = hmac.new(secret,
encoded_header + "." + encoded_payload,
hashlib.sha256)
encoded_signature = base64url_encode(signature.digest())
return encoded_header + "." + encoded_payload + "." + encoded_signature
def get_request(url, jwt):
headers = {
"Authorization" : "Bearer " + jwt
}
result = requests.get(url, headers=headers)
return result
secret = "yoursecrettoken"
header = '{"alg":"HS256","typ":"JWT"}'
payload = '{"sub":"1234567890","name":"John Doe","iat":1516239022}'
jwt = generate_JWT(header, payload, secret)
response = get_request("https://SomeApiEndpoint.com/api/", jwt)
You can test the output of this against the JWT's debugger here.
Note: For the encoding to work properly for the header and payload objects you have to convert them to a string object. I tried doing this by calling the JSON.dumps() function and passing the dictionary objects, but when I encoded the return values they did not match what was shown on the JWT debugger. The only solution I could find was by wrapping the dictionary objects in quotations and ensuring there were no spaces within it.
And so with the JWT in hand you can use it in your Zapier Webhooks custom get request step, or you could save the zap and send the request in the same code module using Python's request library as I have in my code example.
Thanks for the learning opportunity, and I hope this helps.
I had a similar issue trying to generate and use a JWT in a custom integration. Unfortunately, this code above did not work for my situation. I'm currently using the below javascript and it seems to be functioning perfectly.
const toBase64 = obj => {
const str = JSON.stringify (obj);
return Buffer.from(str).toString ('base64');
};
const replaceSpecialChars = b64string => {
// this will match the special characters and replace them with url-safe substitutes
return b64string.replace (/[=+/]/g, charToBeReplaced => {
switch (charToBeReplaced) {
case '=':
return '';
case '+':
return '-';
case '/':
return '_';
}
});
};
const crypto = require('crypto');
const signatureFunction = crypto.createSign('RSA-SHA256');
const headerObj = {
alg: 'RS256',
typ: 'JWT',
};
const payloadObj = {
iat: Math.round(Date.now() / 1000), // lists the current Epoch time
exp: Math.round(Date.now() / 1000) + 3600, // adds one hour
sub: '1234567890'
name: 'John Doe'
};
const base64Header = toBase64(headerObj);
const base64Payload = toBase64(payloadObj);
const base64UrlHeader = replaceSpecialChars(base64Header);
const base64UrlPayload = replaceSpecialChars(base64Payload);
signatureFunction.write(base64UrlHeader + '.' + base64UrlPayload);
signatureFunction.end();
// The private key without line breaks
const privateKey = `-----BEGIN PRIVATE KEY-----
MIIJQwIBADANBgkqhkiG9w0BAQEFAASCCS0wggkpAgEAAoICAQC5Q+0Je6sZ6BuX
cTsN7pEzAaj4819UE7gM+Tf7U5AKHSKk3hN5UILtp5EuEO7h7H+lyknn/5txltA4
-----END PRIVATE KEY-----`;
const signatureBase64 = signatureFunction.sign(privateKey, 'base64');
const signatureBase64Url = replaceSpecialChars(signatureBase64);
console.log("Your JWT is: " + base64UrlHeader + "." + base64UrlPayload + "." + signatureBase64Url);
I have this code in a Zapier code-step prior to calling the custom integration and pass in the generated token object to authenticate the call.
Hopefully, this helps someone!
Related
I've a slack app that is sending to a service written in typescript that is forwarding the message to my python script where I'm trying to validate the request. However, for some reason, the validation always fails.
The typescript relevant code:
const rp = require('request-promise');
var qs = require('querystring')
export const handler = async (event: any, context: Context, callback: Callback): Promise<any> => {
const options = {
method: method,
uri: some_url,
body: qs.parse(event.body),
headers: {
signature: event.headers['X-Slack-Signature'],
timestamp: event.headers['X-Slack-Request-Timestamp']
},
json: true
};
return rp(options);
The python code (based on this article) :
def authenticate_message(self, request: Request) -> bool:
slack_signing_secret = bytes(SLACK_SIGNING_SECRET, 'utf-8')
slack_signature = request.headers['signature']
slack_timestamp = request.headers['timestamp']
request_body = json.loads(request.body)['payload']
basestring = f"v0:{slack_timestamp}:{request_body}".encode('utf-8')
my_signature = 'v0=' + hmac.new(slack_signing_secret, basestring, hashlib.sha256).hexdigest()
return hmac.compare_digest(my_signature, slack_signature))
I'm pretty sure the issue is the way I'm taking the body but tried several options and still no luck.
Any ideas?
Thanks,
Nir.
I had the same issue. My solution was to parse the payload to replace '/' by %2F and ':' by %3A. It's not explicit in the Slack doc but if you see the example, that's how it's shown:
'v0:1531420618:token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J&team_domain=testteamnow&channel_id=G8PSS9T3V&channel_name=foobar&user_id=U2CERLKJA&user_name=roadrunner&command=%2Fwebhook-collect&text=&response_url=https%3A%2F%2Fhooks.slack.com%2Fcommands%2FT1DC2JH3J%2F397700885554%2F96rGlfmibIGlgcZRskXaIFfN&trigger_id=398738663015.47445629121.803a0bc887a14d10d2c447fce8b6703c'
You see command and response_url are parsed.
I managed to get this working in Python. I see you ask in Typescript, but I hope this python script helps:
#app.route('/slack-validation', methods=['GET', 'POST'])
def slack_secutiry():
headers = request.headers
timestamp = request.headers['X-Slack-Request-Timestamp']
slack_payload = request.form
dict_slack = slack_payload.to_dict()
### This is the key that solved the issue for me, where urllib.parse.quote(val, safe='')] ###
payload= "&".join(['='.join([key, urllib.parse.quote(val, safe='')]) for key, val in dict_slack.items()])
### compose the message:
sig_basestring = 'v0:' + timestamp + ':' + payload
sig_basestring = sig_basestring.encode('utf-8')
### secret
signing_secret = slack_signing_secret.encode('utf-8') # I had an env variable declared with slack_signing_secret
my_signature = 'v0=' + hmac.new(signing_secret, sig_basestring, hashlib.sha256).hexdigest()
print('my signature: ')
print(my_signature)
return '', 200
It might be useful for you to check how the request validation feature is implemented in the Bolt framework:
https://github.com/slackapi/bolt-python/blob/4e0709f0578080833f9aeab984a778be81a30178/slack_bolt/middleware/request_verification/request_verification.py
Note that it is implemented as a middleware, enabled by default when you instantiate the app (see attribute request_verification_enabled).
You can inspect this behaviour and/or change it if you want to validate the requests manually:
app = App(
token=SLACK_BOT_TOKEN,
signing_secret=SLACK_SIGNING_SECRET,
request_verification_enabled=False
)
The following solution solves the problem of verification of signing secret of slack
#!/usr/bin/env python3
import hashlib
import hmac
import base64
def verify_slack_request(event: dict, slack_signing_secret: str) -> bool:
"""Verify slack requests.
Borrowed from https://janikarhunen.fi/verify-slack-requests-in-aws-lambda-and-python.html
- Removed optional args
- Checks isBase64Encoded
:param event: standard event handler
:param slack_signing_secret: slack secret for the slash command
:return: True if verification worked
"""
slack_signature = event['headers']['x-slack-signature']
slack_time = event['headers']['x-slack-request-timestamp']
body = event['body']
if event['isBase64Encoded']:
body = base64.b64decode(body).decode("utf-8")
""" Form the basestring as stated in the Slack API docs. We need to make a bytestring"""
base_string = f'v0:{slack_time}:{body}'.encode('utf-8')
""" Make the Signing Secret a bytestring too. """
slack_signing_secret = bytes(slack_signing_secret, 'utf-8')
""" Create a new HMAC 'signature', and return the string presentation."""
my_signature = 'v0=' + hmac.new(
slack_signing_secret, base_string, hashlib.sha256
).hexdigest()
''' Compare the the Slack provided signature to ours.
If they are equal, the request should be verified successfully.
Log the unsuccessful requests for further analysis
(along with another relevant info about the request).'''
result = hmac.compare_digest(my_signature, slack_signature)
if not result:
logger.error('Verification failed. my_signature: ')
logger.error(f'{my_signature} != {slack_signature}')
return result
if __name__ == '__main__':
# add correct params here
print(verify_slack_request({}, None))
Borrowed From:
https://gist.github.com/nitrocode/288bb104893698011720d108e9841b1f
Credits: https://gist.github.com/nitrocode
sorry for my english.
I'm interacting with the Azure admin rest API and I want to programmatically create a SAS token. In Azure documentation is explained for C # (I attach the code below) but I need to implement it in Python (I'm new) and I can't get the data encoding and signing process correctly, even though I've searched a lot of information and tested in some different ways. Could someone help me to "translate" this code?. Thank you very much.
c#
using System;
using System.Text;
using System.Globalization;
using System.Security.Cryptography;
public class Program
{
public static void Main()
{
var id = "account-name";
var key = "account-key";
var expiry = DateTime.UtcNow.AddDays(10);
using (var encoder = new HMACSHA512(Encoding.UTF8.GetBytes(key)))
{
var dataToSign = id + "\n" + expiry.ToString("O", CultureInfo.InvariantCulture);
var hash = encoder.ComputeHash(Encoding.UTF8.GetBytes(dataToSign));
var signature = Convert.ToBase64String(hash);
var encodedToken = string.Format("SharedAccessSignature uid={0}&ex={1:o}&sn={2}", id, expiry, signature);
Console.WriteLine(encodedToken);
}
}
}
Based on #GaJsu 's solution
Following is my solution:
import base64
import hmac
import hashlib
from datetime import datetime
from dateutil.relativedelta import relativedelta
identifier = 'the "Identifier" value in "management API" tab'
end_date = datetime.now() + relativedelta(months=+6)
expiry = f'{end_date.isoformat()}0Z'
key_azure = 'the key in "management API" tab, primary/secondary key'
string_to_sign = f'{identifier}\n{expiry}'
signature = (
base64.b64encode(
hmac.new(
bytearray(key_azure, "utf-8"),
bytearray(string_to_sign,"utf-8") ,
hashlib.sha512).digest()
)
).decode("utf-8")
auth_sas = f"SharedAccessSignature uid={identifier}&ex={expiry}&sn={signature}"
Finally I can create the SAS token. Here is the code for the signature:
string_to_sign = '{}{}{}'.format(id_azure,'\n',expiry)
signature = (base64.b64encode(hmac.new(bytearray(key_azure, "utf-8") , bytearray(string_to_sign,"utf-8") , hashlib.sha512).digest())).decode("utf-8").replace("\n", "")
I am totally new to python flask and encountered a problem when writing some code using the requests and flask modules.
I am working on a project using the web API offered by the Panther platform. The project provided an example using Apache Java.
The source code is as below (see more for details).
public class TestProject {
public static void main(String args[]) throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpPost httppost = new HttpPost("http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?");
StringBody organism = new StringBody("Homo sapiens", ContentType.TEXT_PLAIN);
FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);
StringBody enrichmentType = new StringBody("process", ContentType.TEXT_PLAIN);
StringBody testType = new StringBody("FISHER", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("FDR", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("BONFERRONI", ContentType.TEXT_PLAIN);
//StringBody cor = new StringBody("NONE", ContentType.TEXT_PLAIN);
StringBody type = new StringBody("enrichment", ContentType.TEXT_PLAIN);
HttpEntity reqEntity = MultipartEntityBuilder.create()
.addPart("organism", organism)
.addPart("geneList", fileData)
.addPart("enrichmentType", enrichmentType)
.addPart("test_type", testType)
.addPart("type", type)
//.addPart("correction", cor)
.build();
httppost.setEntity(reqEntity);
CloseableHttpResponse response = httpclient.execute(httppost);
try {
//System.out.println("----------------------------------------");
//System.out.println(response.getStatusLine());
HttpEntity resEntity = response.getEntity();
if (resEntity != null) {
System.out.println(IOUtils.toString(resEntity.getContent(), StandardCharsets.UTF_8));
}
EntityUtils.consume(resEntity);
} finally {
response.close();
}
} finally {
httpclient.close();
}
}
}
The part I am mostly interested in is .addPart("organism", organism) and all the other code with similar structures. They will help pass the parameters from a third-party website to the web API offered by Panther.
I remade the JAVA code into python3 using requests. The code is as follows:
uploadTemp = {'file':open('./app/static/data_temp/temp.txt','rb')}
url="http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
params = {"organism":organism,"geneList":pantherName,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"}
# or params = {"organism":organism,"geneList":uploadTemp,"enrichmentType":"fullGO_process","test_type":"BINOMIAL","type":"enrichment","correction":"BONFERRONI"}
Pantherpost= requests.post(url, params = params)
print(Pantherpost.text)
I am expecting an XML object from the web API including some basic biological information. However, the result I got was null (or \n\n\rnull\n when I print Pantherpost.content)
It seems that the parameters I have got from my own web were not correctly sent to the web API.
In addition to this getting null problem, as a beginner, I am also not quite sure about whether the "geneList" part should be receiving a plain-text object or a file. The manual says it is expecting a file, however, it may have been reformatted into plain-text by this command
FileBody fileData = new FileBody(new File("c:\\data_files\\gene_expression_files\\7_data\\humanEnsembl"), ContentType.TEXT_PLAIN);
Anyway, I did try both interpretations: pantherName is a list with name correctly formatted in plain-text and uploadTemp is a .txt file generated for the project. There must be some extra bugs in my code since it returned null in both cases.
Can someone please help out? Thank you very much.
I've found the following issues with your python code:
One. If you want to POST a file using requests, you should use keyword files=.
Two. Keys in files object should match respective parameters of the request (you're using file instead).
Three. You put your parameters in the wrong place of the request by writing params=params.
Function annotation from requests source code:
:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:Request.
In example Java code StringBody is used to create parameters, which implies that parameters should be placed inside the body of HTTP request, not query string. So you should use data= keyword instead. If you use params=, output will be null.
SO article on difference between data and params keywords in requests.
So I've spent some time reading thier manual and made a test script:
import requests
url = "http://pantherdb.org/webservices/garuda/tools/enrichment/VER_2/enrichment.jsp?"
filepath = "C:\\data\\YOUR_DATA.txt" # change to your file location
# all required parameters according to manual, except geneList which is a file (see below)
params = { # using defaults from manual
"type": "enrichment",
"organism": "Homo sapiens",
"enrichmentType": "process",
"test_type": "FISHER",
"correction": "FDR",
}
# note that the key here is the name of paramter: geneList
files = {'geneList': open(filepath, 'rb')}
# it outputs null, when 'params=params' is used
r = requests.post(url, data=params, files=files)
print(r.status_code)
print(r.text)
Output:
200
Id Name GeneId raw P-value FDR
i have a function written in Python which calls Robinhood(a stock trading broker) API to get quote data("get_quote(self, symbol)" function in following code snipshot). And it works fine. Correct market data was returned.
import requests
import urllib
class Robinhood(object):
# All known endpoints as of September 5th, 2015
endpoints = {
"quotes": "https://api.robinhood.com/quotes/",
"user": "https://api.robinhood.com/user/",
"user/additional_info": "https://api.robinhood.com/user/additional_info/",
"user/basic_info": "https://api.robinhood.com/user/basic_info/",
"user/employment": "https://api.robinhood.com/user/employment/",
"user/investment_profile": "https://api.robinhood.com/user/investment_profile/",
"watchlists": "https://api.robinhood.com/watchlists/"
}
def get_quote(self, symbol):
''' Returns a qoute object for a given symbol including all data returned by Robinhood's API'''
data = { 'symbols' : symbol }
res = self.session.get(self.endpoints['quotes'], params=data)
if res.status_code == 200:
return res.json()['results']
else:
raise Exception("Could not retrieve quote: " + res.text)
I tried to implement this logic in C++ using Curl library. But it doesn't work. There was no compile or run time error but the program returned a single unreadable character instead of the market price of stock. It looks to me like my URL is not correctly set up but I couldn't figure out how to fix it. Does someone have an idea? Thank you!
std::string RobinhoodAPI::GetQuote(std::string ticker)
{
struct response resStr;
init_string(&resStr);
std::string url = "https://api.robinhood.com/quotes/symbols=AVP/";
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &resStr);
resCode = curl_easy_perform(curl);
std::cout << std::string(resStr.ptr);
return std::string(resStr.ptr);
}
I have created an open api specification for the unofficial documentation of the robinhood api. With this you can generate http client for most languages.
Please visit here https://github.com/sabareeshkkanan/robinhood for the specification. And please visit this repo for how to generate client using this specification https://github.com/swagger-api/swagger-codegen
It's frustrating to deal with the regular Signed URLs (Query String Authentication) for Google Cloud Storage.
Google Cloud Storage Signed URLs Example -> Is this really the only code available in the whole internet for generating Signed URLs for Google Cloud Storage? Should I read it all and adapt it manually for Pure Python GAE if needed?
It's ridiculous when you compare it with AWS S3 getAuthenticatedURL(), already included in any SDK...
Am I missing something obvious or does everyone face the same problem? What's the deal?
Here's how to do it in Go:
func GenerateSignedURLs(c appengine.Context, host, resource string, expiry time.Time, httpVerb, contentMD5, contentType string) (string, error) {
sa, err := appengine.ServiceAccount(c)
if err != nil {
return "", err
}
expUnix := expiry.Unix()
expStr := strconv.FormatInt(expUnix, 10)
sl := []string{
httpVerb,
contentMD5,
contentType,
expStr,
resource,
}
unsigned := strings.Join(sl, "\n")
_, b, err := appengine.SignBytes(c, []byte(unsigned))
if err != nil {
return "", err
}
sig := base64.StdEncoding.EncodeToString(b)
p := url.Values{
"GoogleAccessId": {sa},
"Expires": {expStr},
"Signature": {sig},
}
return fmt.Sprintf("%s%s?%s", host, resource, p.Encode()), err
}
I have no idea why the docs are so bad. The only other comprehensive answer on SO is great but tedious.
Enter the generate_signed_url method. Crawling down the rabbit hole you will notice that the code path when using this method is the same as the solution in the above SO post when executed on GAE. This method however is less tedious, has support for other environments, and has better error messages.
In code:
def sign_url(obj, expires_after_seconds=60):
client = storage.Client()
default_bucket = '%s.appspot.com' % app_identity.get_application_id()
bucket = client.get_bucket(default_bucket)
blob = storage.Blob(obj, bucket)
expiration_time = int(time.time() + expires_after_seconds)
url = blob.generate_signed_url(expiration_time)
return url
I came across this problem recently as well and found a solution to do this in python within GAE using the built-in service account. Use the sign_blob() function in the google.appengine.api.app_identity package to sign the signature string and use get_service_account_name() in the same package to get the the value for GoogleAccessId.
Don't know why this is so poorly documented, even knowing now that this works I can't find any hint using Google search that it should be possible to use the built-in account for this purpose. Very nice that it works though!
Check out https://github.com/GoogleCloudPlatform/gcloud-python/pull/56
In Python, this does...
import base64
import time
import urllib
from datetime import datetime, timedelta
from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from OpenSSL import crypto
method = 'GET'
resource = '/bucket-name/key-name'
content_md5, content_type = None, None
expiration = datetime.utcnow() + timedelta(hours=2)
expiration = int(time.mktime(expiration.timetuple()))
# Generate the string to sign.
signature_string = '\n'.join([
method,
content_md5 or '',
content_type or '',
str(expiration),
resource])
# Take our PKCS12 (.p12) key and make it into a RSA key we can use...
private_key = open('/path/to/your-key.p12', 'rb').read()
pkcs12 = crypto.load_pkcs12(private_key, 'notasecret')
pem = crypto.dump_privatekey(crypto.FILETYPE_PEM, pkcs12.get_privatekey())
pem_key = RSA.importKey(pem)
# Sign the string with the RSA key.
signer = PKCS1_v1_5.new(pem_key)
signature_hash = SHA256.new(signature_string)
signature_bytes = signer.sign(signature_hash)
signature = base64.b64encode(signature_bytes)
# Set the right query parameters.
query_params = {'GoogleAccessId': 'your-service-account#googleapis.com',
'Expires': str(expiration),
'Signature': signature}
# Return the built URL.
return '{endpoint}{resource}?{querystring}'.format(
endpoint=self.API_ACCESS_ENDPOINT, resource=resource,
querystring=urllib.urlencode(query_params))
And if you don't want to write it by your own checkout this class on GitHub.
Really easy to use
GCSSignedUrlGenerator