We Keep Coding

different utf8 encoding outputs between Python and Haskell

I'm trying to encode a has which is computed by hmac sha256 in Haskell and sometimes my function has different output when compared to it's Python counterpart.
This is the python funciton:
import requests
import json
import hmac
import hashlib
import base64
import time
def strToSign(time, method, endpoint, body):
return time + method + endpoint + body
str_to_sign2 = strToSign('1','GET','/api/v1/position?symbol=XBTUSDM','')
signature2 = base64.b64encode(
hmac.new(api_secret.encode('utf-8'), str_to_sign2.encode('utf-8'), hashlib.sha256).digest())
And this is the Haskell function:
import qualified Data.ByteString.Char8 as BC
import qualified Data.Text as T
import qualified Data.ByteString.Base64.URL as U
import Data.Text.Encoding (encodeUtf8)
import qualified Crypto.Hash.SHA256 as H
apiSignTest :: BC.ByteString -> BC.ByteString -> BC.ByteString -> BC.ByteString -> IO BC.ByteString
apiSignTest time method endpoint body = do
let timeStamp = time
let secret = mconcat [timeStamp,method,endpoint,body]
let hash = H.hmac (BC.pack C.apiSecretFuture) secret
return $ (encodeUtf8 . U.encodeBase64) hash
some examples where the encoded outputs are different
Haskell : b'KbCFw8OYGeGB433L93vQvbsnzSXxG88r_-HR5AGDJmo='
Python : "KbCFw8OYGeGB433L93vQvbsnzSXxG88r/+HR5AGDJmo="
Python : b'dwSmCd75wZToIDt6I0Ik/sX8Vxk4W+RA0Sv1TO+x4WI='
Haskell : "dwSmCd75wZToIDt6I0Ik_sX8Vxk4W-RA0Sv1TO-x4WI="
Python : b'X8SE3ohju6VAu2Dt5nGIQP40+KU9RrhXORAUOdL7rJg='
Haskell : "X8SE3ohju6VAu2Dt5nGIQP40-KU9RrhXORAUOdL7rJg="

Data.ByteString.Base64.URL.encodeBase64 is specifically using the base64url encoding rather than vanilla base64. The purpose, as the name suggests, is that these encodings can be directly embedded in URLs, which the vanilla version cannot because / and + have special meanings in URLs.
To get the same behaviour as Python's b64encode, use Data.ByteString.Base64.encodeBase64 instead. Or, the other way around, in Python you can use urlsafe_b64encode.

Constructing a URI in Python?

I'm trying to convert a Java program to Python, and one thing that I am currently stuck on is working with URI's in Python. I found urllib.response in Python, but I'm struggling to figure out how to utilize it.
What I'm trying to do with this URI is obtain user info (particularly username and password), the host and path. In Java, there are associated methods (getUserInfo(), getHost(), and getPath()) for this, but I'm having trouble finding equivalents for this in Python, even after looking up the urllib.response Python documentation.
The equivalent code in Java is:
URI dbUri = new URI(env);
username = dbUri.getUserInfo().split(":")[0];
password = dbUri.getUserInfo().split(":")[1];
dbUrl = "jdbc:postgresql://" + dbUri.getHost() + dbUri.getPath();
and what would be the appropriate methods that could be used to convert this to Python?

Seems like you'd want to use something like urllib.parse.urlparse.
https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse
from urllib.parse import urlparse
db_url = urlparse(raw_url_string)
username = db_url.username
password = db_url.password
host = db_url.net_loc
path = db_url.path
...
You might need to adjust this a bit. There is a subtle difference between urlparse and urlsplit regarding parameters. Then you can use one of the urlunparse or unsplit.

Code
from urllib import parse
from urllib.parse import urlsplit
url = 'http://localhost:5432/postgres?user=postgres&password=somePassword'
split_url = urlsplit(url)
hostname = split_url.netloc
path = split_url.path
params = dict(parse.parse_qsl(split_url.query))
username = params['user']
password = params['password']
db_url = "jdbc:postgresql://" + hostname + path
print(db_uri)
Output
jdbc:postgresql://localhost:5432/postgres

base64 HMAC with SHA256 in Python

I am having a hard time creating a signature.
I am needing to make a signature using HMAC with SHA256 using a Checkout Request JSON and a secret key. I need to do it by concatenating signature, pipe character (|) and Checkout Request JSON and then encoding it with BASE64.
This is a formula I found in the documentations:
$signed_checkout_request = base64( hmac_sha256( $checkout_request, $private_key ) + "|" + $checkout_request )
I have made this based on some online code:
import hashlib
import hmac
import base64
checkout_request = '{"charge":{"amount":499,"currency":"EUR"}}'.encode('utf-8');
private_key = b'44444444444';
digest = hmac.new(private_key, msg=checkout_request, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(digest).decode()
However I am not sure how to get the "|" into it. I am also not sure if I am even on the right track if I am honest... I don't have much experience in this section and I have failed at googling.

private_key = 'blahblahblah'
checkout_request = json.dumps({"charge":{"amount":4999,"currency":"EUR"}}, sort_keys=True, separators=(",", ":"))
digest = hmac.new(private_key.encode(), msg=checkout_request.encode(), digestmod=hashlib.sha256,).hexdigest()
signature = base64.b64encode((digest + "|" + checkout_request).encode()).decode()
I was able to get it to work with that :)

RSA sign a string with private key in python

I am communicating with our clients server. For an api I need to sign a string with my private key. They have the following condition to follow
User SHA 256 algorithm to calculate the hash of the string
Use the private key and RSA (PKCS1_PADDING) algorithm to sign the Hash Value.
Base64 encode the encrypted Hash Value
and I am doing following
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
import base64
pkey = RSA.importKey(keystring)
message = "Hello world"
h = SHA256.new(message.encode())
signature = PKCS1_v1_5.new(pkey).sign(h)
result = base64.b64encode(signature).decode()
Here I am getting a string as result. But on the server side my signature is not matching.
Is there anything I am going wrong with ?? Can anyone help me on this ?

I came back to this question recently and noticed it was never resolved. I don't know what was going wrong with the OPs setup but the following code worked for me.
First, the python code that generates the signature of "Hello world":
from Cryptodome.Signature import PKCS1_v1_5
from Cryptodome.Hash import SHA256
from Cryptodome.PublicKey import RSA
import base64
def sign(message: str, private_key_str: str) -> str:
priv_key = RSA.importKey(private_key_str)
h = SHA256.new(message.encode('utf-8'))
signature = PKCS1_v1_5.new(priv_key).sign(h)
result = base64.b64encode(signature).decode()
return result
And now the Java code that verifies it:
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.GeneralSecurityException;
import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.Signature;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;
...
...
public static boolean verify(String message, String b64Sig, byte[] pubkey_spki) throws GeneralSecurityException {
var pubKey = (PublicKey) KeyFactory.getInstance("RSA").generatePublic(new X509EncodedKeySpec(pubkey_spki));
var verifier = Signature.getInstance("SHA256withRSA");
verifier.initVerify(pubKey);
verifier.update(message.getBytes(StandardCharsets.UTF_8));
return verifier.verify(Base64.getDecoder().decode(b64Sig));
}
Perhaps the trickiest part of this is specifying the correct padding scheme in each language/library. These signatures use the scheme identified as RSASSA-PKCS1-v1_5 in the PKCS#1 RFC 8017. On the python side this is accomplished by providing the SHA256 hash object to the PKCS1_v1_5 signature object. In Java it is perhaps a little more straightforward in that you ask for Signature object that implements the RSA algorithm with SHA256 as the hash function, but still have to know that this is RSASSA-PKCS1-v1_5 and not some other possibility in RFC 8017.
I think if one is not already something of an expert then understanding that these magic choices in python and Java produce compatible code is going to be difficult.

How to get refresh tokens on Facebook using Python?

I am currently using the following code for accessing the refresh tokens. Bare in mind that the config variable is frequently updated with the new access_token value after it is set.
refresh_token_url = config['facebook']['domain'] + \
'oauth/access_token?' + \
'grant_type=fb_exchange_token&' + \
'client_id=' + config['facebook']['client_id'] + '&' + \
'client_secret=' + config['facebook']['client_secret'] + '&' + \
'fb_exchange_token=' + config['facebook']['access_token']
new_access_token_obj = requests.get(refresh_token_url)
config['facebook']['access_token'] = json.loads(new_access_token_obj.text)['access_token']
But that I have noticed is that the new_access_token_obj has an expiry time that is lower than the old access_token but the two access tokens are different.
So, I am hoping that someone could tell me how to fix this code with a new access token which has a higher ['expires_in'] field.
PS :
Please note that I am aware that long lived tokens can be obtained using a web-app but I am looking to get it through a script (through some work-around).