I am writing a basic TACACS client module. I can get the packets on the wire via struct.pack ok, as I've checked against wireshark. But I am not sure how to handle the byte-wise XOR of the md5 and not even sure if I am creating the hash properly. The RFCs for TACACS say take specific items from the TACACS header and 'secret key' and encrypt it five times, concatenating the previous hash during each run. See below
def encrypt(packet, tac_key):
key = packet[4:8] + bytes(tac_key, 'utf-8') + packet[:1] + packet[2:3]
h1 = md5()
h1.update(key)
hash = h1.digest()
previous = h1.digest()
for i in range(2, 6):
new = md5()
key = key + previous
new.update(key)
hash += new.digest()
previous = new.digest()
return hash
The packet has already been run through struct.pack
body = struct.pack(fmt, TAC_AUTHEN_LOGIN, TAC_PLUS_PRIV_LVL_USER, TAC_PLUS_AUTHEN_TYPE_PAP, TAC_PLUS_AUTHEN_SVC_LOGIN,
user_len, port_len, rem_addr_len, data_len, bytearray(user, 'utf-8'),
bytearray(port, 'utf-8'), bytearray(data, 'utf-8)'))
Is this the correct way to do this or should I not use struct.pack on the body of the packet and just encrypt it?
I was not generating a new pseudo_pad with server's response header. Therefore, the response remained obfuscated.
Related
I'm working with an API provided by a client, who provided sample code on authorizing via C#, but my company works in Python. They authorize via HMAC, and following their sample code (on .net fiddle), I finally got to the point where our byte key and byte message match those of the C# call.
However, when we get to this in their code:
using (HMACSHA256 sha = new HMACSHA256(secretKeyByteArray))
{
// Sign the request
byte[] signatureBytes = sha.ComputeHash(signature);
Where our equivalent is
signature_hmac = hmac.new(
base64.b64decode(secretKey),
bytes(message, "utf8"),
digestmod=hashlib.sha256,
)
The byte value of their signatureBytes doesn't match with the byte value of our signature_hmac.digest(). If using the same hashing libraries, and the byte values of the inputs match, we should get the same result, right?
To make sure, when I say byte values match, the value of base64.b64decode(secretKey) (Python) matched var secretKeyByteArray = Convert.FromBase64String(secretKey); (C#).
Worked for me, make sure your input arrays are EXACTLY the same on both. I think you might be missing a base64 decode or encoding the strings differently.
Here's my 2 tests:
secretKey = "wxyz".encode()
message = "abcd".encode()
signature_hmac = hmac.new(
secretKey,
message,
digestmod=hashlib.sha256,
)
x = signature_hmac.hexdigest()
and C#
byte[] signatureBytes;
byte[] secretKeyByteArray = Encoding.UTF8.GetBytes("wxyz");
byte[] signature = Encoding.UTF8.GetBytes("abcd");
using (HMACSHA256 sha = new HMACSHA256(secretKeyByteArray))
{
signatureBytes = sha.ComputeHash(signature);
};
string x = Convert.ToHexString(signatureBytes);
I'm a little bit confused by what I need to do here for Python, but from the Yubikey API documentation for verifying Yubikeys that have YubiOTP the HMAC signature needs to be generated a specific way - from their documentation:
Generating signatures
The protocol uses HMAC-SHA-1 signatures. The HMAC key to use is the
client API key.
Generate the signature over the parameters in the message. Each
message contains a set of key/value pairs, and the signature is always
over the entire set (excluding the signature itself), and sorted in
alphabetical order of the keys. More precisely, to generate a message
signature do:
Alphabetically sort the set of key/value pairs by key order.
Construct a single line with each ordered key/value pair
concatenated using &, and each key and value contatenated with =. Do
not add any linebreaks. Do not add whitespace. For example:
a=2&b=1&c=3.
Apply the HMAC-SHA-1 algorithm on the line as an octet string using
the API key as key (remember to base64decode the API key obtained from
Yubico).
Base 64 encode the resulting value according to RFC 4648, for
example, t2ZMtKeValdA+H0jVpj3LIichn4=.
Append the value under key h to the message.
Now my understanding of their API from their documentation states the following valid request parameters:
id - the Client ID from Yubico API
otp - the YubiOTP value from the YubiOTP component of a yubikey.
h - the HMAC-SHA1 signature for the request
timestamp - empty does nothing, 1 includes the timestamp in the reply from the server
nonce - A 16 to 40 character long string with random unique data.
sl - a value of 0 to 100 indicating percentage of syncing required by client, or strings "fast" or "Secure" to use server values; if nonexistent server decides
timeout - # of seconds to wait for sync responses; let server decide if absent.
I have a total of two functions I'm trying to use to try and handle all these things and generate the URL. Namely, we the HMAC support function and the verify_url_generate which generates the URL (and API_KEY is statically coded - my API Secret Key from Yubico):
def generate_signature(message, key=base64.b64decode(API_KEY)):
message = bytes(message, 'UTF-8')
digester = hmac.new(key, message, hashlib.sha1)
digest = digester.digest()
signature = base64.urlsafe_b64encode(digest)
return str(signature, 'UTF-8')
def verify_url_generate(otp):
nonce = "".join(secrets.choice(ascii_lowercase) for _ in range(40))
data = OrderedDict(
{
"id": None,
"nonce": None,
"otp": None,
"sl": 50,
"timeout": 10,
"timestamp": 1
}
)
data['otp'] = otp
data['id'] = CLIENT_ID
data['nonce'] = nonce
args = ""
for key, value in data.items():
args += f"{key}={value}&"
sig = generate_signature(args[:-1])
url = YUBICO_API_URL + args + "&h=" + sig
print(url)
return
Any URL generated by this triggers a notice about "BAD_SIGNATURE" from the remote site - any URL generated minus the HMAC sig (h=) parameter works. So we know the issue isn't the URL, it's the HMAC signature.
Does anyone know what I'm doing wrong with my HMAC generation approach, by passing the HMAC sig generator the concatenated args from the ordered dict in key=value separated by & for each parameter format?
Can you try using standard_b64encode and then using urllib.parse.quote(url) in your final URL?
I ask because this page says that "As such, all parameters must be properly URL encoded. In particular, some base64 characters (such as "+") in the value fields needs to be escaped." which means it is expecting +(or %2B) in the args and does a unquote and then normal decode.
I am using Python's socket to respond to Websocket requests and I am facing problems with the Sec-WebSocket-Accept header. There might have been problems like this asked before but no one worked for me
The documentation at https://developer.mozilla.org states that for the connection to be established :
The server takes the value of the Sec-WebSocket-Key sent in the handshake request, appends 258EAFA5-E914-47DA-95CA-C5AB0DC85B11, takes SHA-1 of the new value, and is then base64 encoded.
The response I am currently sending is wrapped in a function that takes in the key from the request
and does what is said in the documentation
def socket101(self,template,key):
key = key.strip()
key += "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" #Add the key
key = sha1(key.encode()).digest() #Take the sha1
key = b64encode(key) #B64 encode it
return (b"HTTP/1.1 101 Switching Protocols\n"
+b"Content-Type: text/html\n"
+b"Connection: Upgrade\n"
+b"Upgrade: websocket\n"
+"Sec-WebSocket-Accept: {}\n".format(key).encode()
+b"\n"
)
Then I am calling the above method as follows
def AwaitMessage(client,address):
while True:
data = client.recv(1024)
try:
if str(data.strip()) != '':
headers = ParseHeaders(data) #Parses the request headers
HTTP_MSG = socket101(headers['Sec-WebSocket-Key']) #Call the function above passing the key
client.send(HTTP_MSG) #Send the message
except Exception as f:
pass
client.close()
But this does not want whatever I do and throws the following error:
Error during WebSocket handshake: Incorrect 'Sec-WebSocket-Accept' header value
The browser does not provide any information on why it is incorrect and I can't guess.
The JavaScript code is just making the connection with the socket.
const host = window.location.host
const PORT = 8000;
const socket = new WebSocket(`ws://${host}:${PORT}/ws/connection`)
Any ideas?
The following code worked for me
from base64 import b64encode
from hashlib import sha1
key = b64encode(sha1((key + GUID).encode()).digest()) #This
return (b"HTTP/1.1 101 Switching Protocols\n"
+b"Content-Type: text/html\n"
+b"Connection: Upgrade\n"
+b"Upgrade: websocket\n"
+b"Sec-WebSocket-Accept: " + key + b"\n" #Maybe this did something?
+b"\n"
)
I am almost sure that it is exactly the same as the thing I have above but the above does not work.
EDIT: The problem was that type(key) is bytes and thus f'{key}' will wrap quotes arround it, but on the snippet above it is concatinated with other bytes.
Legend
I expose an API which requires client to sign requests by sending two headers:
Authorization: MyCompany access_key:<signature>
Unix-TimeStamp: <unix utc timestamp in seconds>
To create a signature part, the client should use a secret key issued by my API service.
In Python (Py3k) it could look like:
import base64
import hmac
from hashlib import sha256
from datetime import datetime
UTF8 = 'utf-8'
AUTH_HEADER_PREFIX = 'MyCompany'
def create_signature(access_key, secret_key, message):
new_hmac = hmac.new(bytes(secret_key, UTF8), digestmod=sha256)
new_hmac.update(bytes(message, UTF8))
signature_base64 = base64.b64encode(new_hmac.digest())
return '{prefix} {access_key}:{signature}'.format(
prefix=AUTH_HEADER_PREFIX,
access_key=access_key,
signature=str(signature_base64, UTF8).strip()
)
if __name__ == '__main__':
message = str(datetime.utcnow().timestamp())
signature = create_signature('my access key', 'my secret key', message)
print(
'Request headers are',
'Authorization: {}'.format(signature),
'Unix-Timestamp: {}'.format(message),
sep='\n'
)
# For message='1457369891.672671',
# access_key='my access key'
# and secret_key='my secret key' will ouput:
#
# Request headers are
# Authorization: MyCompany my access key:CUfIjOFtB43eSire0f5GJ2Q6N4dX3Mw0KMGVaf6plUI=
# Unix-Timestamp: 1457369891.672671
I wondered if I could avoid dealing with encoding digest of bytes to Base64 and just use HMAC.hexdigest() to retrieve a string.
So that my function will change to:
def create_signature(access_key, secret_key, message):
new_hmac = hmac.new(bytes(secret_key, UTF8), digestmod=sha256)
new_hmac.update(bytes(message, UTF8))
signature = new_hmac.hexdigest()
return '{prefix} {access_key}:{signature}'.format(
prefix=AUTH_HEADER_PREFIX,
access_key=access_key,
signature=signature
)
But then I found that Amazon uses similar approach as in my first code snippet:
Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature;
Signature = Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) );
Seeing that Amazon doesn't use hex digest I stopped myself to move forward with it because maybe they know something I don't.
Update
I've measured a performance and found hex digest to be faster:
import base64
import hmac
import string
from hashlib import sha256
UTF8 = 'utf-8'
MESSAGE = '1457369891.672671'
SECRET_KEY = 'my secret key'
NEW_HMAC = create_hmac()
def create_hmac():
new_hmac = hmac.new(bytes(SECRET_KEY, UTF8), digestmod=sha256)
new_hmac.update(bytes(MESSAGE, UTF8))
return new_hmac
def base64_digest():
return base64.b64encode(NEW_HMAC.digest())
def hex_digest():
return NEW_HMAC.hexdigest()
if __name__ == '__main__':
from timeit import timeit
print(timeit('base64_digest()', number=1000000,
setup='from __main__ import base64_digest'))
print(timeit('hex_digest()', number=1000000,
setup='from __main__ import hex_digest'))
Results with:
3.136568891000934
2.3460130329913227
Question #1
Does someone know why do they stick to Base64 of bytes digest and don't use just hex digest? Is there some solid reason to keep using this approach over hex digest?
Question #2
According to RFC2716 the format of Authorization header value when using Basic Authentication
is:
Authorization: Base64(username:password)
So basically you wrap with Base64 two values (user's id and password) seprated by colon.
As you can see in my code snippet and in Amazon's documentation nor me, nor Amazon do that for own custom value of the Authorization header.
Would it be a better style to wrap the whole pair as Base64(access_key:signature) to stick closer to this RFC or it doesn't matter at all?
Amazon does use the hex digest in Signature Version 4.
Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7
http://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html
Your example is from Signature Version 2, the older algorithm, which does use Base-64 encoding for the signature (and which also is not supported in the newest AWS regions).
So, your concern that AWS knows something you don't is misplaced, since their newer algorithm uses it.
In the Authorization: header, it really doesn't make a difference other than a few extra octets.
Where Base-64 gets messy is when the signature is passed in the query string, because + and (depending on who you ask) / and = require special handling -- they need to be url-escaped ("percent-encoded") as %2B, %2F, and %3D respectively... or you have to make accommodations for the possible variations on the server... or you have to require the use of a non-standard Base-64 alphabet, where + / = becomes - ~ _ the way CloudFront does it. (This particular non-standard alphabet is only one of multiple non-standard options, all "solving" the same problem of magic characters in URLs with Base-64).
Go with hex-encoding.
You will almost inevitably find would-be consumers of your API that find Base-64 to be "difficult."
I am trying to validate an incoming webhook and so far the resulting hash is not matching the test hash generated by the api.
The docs list the following example for Ruby however I am using Python/Django so any help to 'convert' this function would be appreciated!
Ruby Function
# request_signature - the signature sent in Webhook-Signature
# request_body - the JSON body of the webhook request
# secret - the secret for the webhook endpoint
require "openssl"
digest = OpenSSL::Digest.new("sha256")
calculated_signature = OpenSSL::HMAC.hexdigest(digest, secret, request_body)
if calculated_signature == request_signature
# Signature ok!
else
# Invalid signature. Ignore the webhook and return 498 Token Invalid
end
This is roughly what I have put together myself so far using https://docs.python.org/3/library/hashlib.html.
Python Attempt
import hashlib
secret = "xxxxxxxxxxxxxxxxxx"
json_data = {json data}
h = hashlib.new('sha256')
h.update(secret)
h.update(str(json_data))
calculated_signature = h.hexdigest()
if calculated_signature == webhook_signature:
do_something()
else:
return 498
When I run the above the hashes never match obviously due to my incorrect Python implementation.
Any help/pointers would be greatly appreciated!
I believe it should be something like this:
import hmac
import hashlib
digester = hmac.new(secret, request_body, hashlib.sha256)
calculated_signature = digester.hexdigest()
A few notes:
Use the actual request body. Don't rely on str(json_data) equalling the request body. This will almost certainly fail as python will print out inner strings using repr which will likely leave a bunch of spurious u"..." that aren't actually in the response. json.dumps won't necessarily do better because there could be whitespace differences that are isignificant to JSON, but are very significant to the hmac signature.
hmac is your friend :-)