Translation of a PHP Script to Python3 (Django) - python
I am attempting to convert from scratch the following PHP script into Python for my Django Project:
Note that it is my understanding that this script should handle values sent from a form, sign the data with the Secret_Key, encrypt the data in SHA256 and encode it in Base64
<?php
define ('HMAC_SHA256', 'sha256');
define ('SECRET_KEY', '<REPLACE WITH SECRET KEY>');
function sign ($params) {
return signData(buildDataToSign($params), SECRET_KEY);
}
function signData($data, $secretKey) {
return base64_encode(hash_hmac('sha256', $data, $secretKey, true));
}
function buildDataToSign($params) {
$signedFieldNames = explode(",",$params["signed_field_names"]);
foreach ($signedFieldNames as $field) {
$dataToSign[] = $field . "=" . $params[$field];
}
return commaSeparate($dataToSign);
}
function commaSeparate ($dataToSign) {
return implode(",",$dataToSign);
}
?>
Here is what I have done so far :
def sawb_confirmation(request):
if request.method == "POST":
form = SecureAcceptance(request.POST)
if form.is_valid():
access_key = 'afc10315b6aaxxxxxcfc912xx812b94c'
profile_id = 'E25C4XXX-4622-47E9-9941-1003B7910B3B'
transaction_uuid = str(uuid.uuid4())
signed_field_names = 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency'
signed_date_time = datetime.datetime.now()
signed_date_time = str(signed_date_time.strftime("20%y-%m-%dT%H:%M:%SZ"))
locale = 'en'
transaction_type = str(form.cleaned_data["transaction_type"])
reference_number = str(form.cleaned_data["reference_number"])
amount = str(form.cleaned_data["amount"])
currency = str(form.cleaned_data["currency"])
# Transform the String into a List
signed_field_names = [x.strip() for x in signed_field_names.split(',')]
# Get Values for each of the fields in the form
values = [access_key, profile_id, transaction_uuid,signed_field_names,'',signed_date_time,locale,transaction_type,reference_number,amount,currency]
# Insert the signedfieldnames in their place in the list (MUST BE KEPT)
values[3] = 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency'
# Merge the two lists as one
DataToSign = list(map('='.join, zip(signed_field_names, values)))
# Hash Sha-256
API_SECRET = 'bb588d4f96ac491ebd43cceb18xx149b79291f874f1a41fcbf5bc078bb6c8793af2df5ad4b174f80bd5f24a4e4eec6fdabdxxxxxc6c1410db40252deea613e0b976748539294438694ba08xx4ba831d3d850349cacfa445f9706aa57be7f8e61aab0be2288054dbe88ec6200ccd7c72888bcc0aa373f42059ec248d3c86b0f45'
message = '{} {}'.format(DataToSign, API_SECRET)
signature = hmac.new(bytes(API_SECRET , 'latin-1'), msg = bytes(message , 'latin-1'), digestmod = hashlib.sha256).hexdigest().upper()
base64string = base64.b64encode( bytes(signature, "utf-8") )
When printing the variables as they come, I obtain the following :
VALUES : ['afc10315b6aa3b2a8cfc91253812b94c', 'E25C4FE4-4622-47E9-9941-1003B7910B3B', '0b59b0ae-bd25-4421-a231-bb83dcfc91fa', 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency', '', '2021-03-06T22:07:30Z', 'en', 'authorization', '1615068450109', '100', 'USD']
DATATOSIGN : ['access_key=afc10315b6aa3b2a8cfc91253812b94c', 'profile_id=E25C4FE4-4622-47E9-9941-1003B7910B3B', 'transaction_uuid=0b59b0ae-bd25-4421-a231-bb83dcfc91fa', 'signed_field_names=access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency', 'unsigned_field_names=', 'signed_date_time=2021-03-06T22:07:30Z', 'locale=en', 'transaction_type=authorization', 'reference_number=1615068450109', 'amount=100', 'currency=USD']
SIGNATURE : 953C786EB9884CEC13C24118B00125BDCFE23AFF8AB02E7BEF29A83156C55C16
BASE64STRING : b'OTUzQzc4NkVCOTg4NENFQzEzQzI0MTE4QjAwMTI1QkRDRkUyM0FGRjhBQjAyRTdCRUYyOUE4MzE1NkM1NUMxNg=='
I think I am getting pretty close from the final result I would like to achieve since I would then simply have to post the Base64String to a specific URL.
However, I am unsure of a couple of things which may seem a bit off :
Is my "translation" of the PHP code into Python correct? Am I meant to merge my lists with a result in "DATATOSIGN"? I am not proficient in PHP so I might have misunderstood how to present the data.
The signature in Base64 should be 44 chars AT ALL TIME like "WrXOhTzhBjYMZROwiCug2My3jiZHOqATimcz5EBA07M=" when using the PHP Sample Code but mine way exceeds this limitation.
If you need any additional information, please do not hesitate to ask.
Hope you can give me pointers !
To approach this problem, it might be good to get an idea of what your final PHP result would be for given parameters.
Here are the parameters I'm using for this with your given PHP code:
$params = [
'access_key' => 'afc10315b6aaxxxxxcfc912xx812b94c',
'profile_id' => 'E25C4XXX-4622-47E9-9941-1003B7910B3B',
'transaction_uuid' => '12345',
'signed_field_names' => 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency',
'unsigned_field_names' => '',
'signed_date_time' => '2021-03-06 16:14:00',
'locale' => 'en',
'transaction_type' => 'credit',
'reference_number' => '12345',
'amount' => '50',
'currency' => 'usd'
];
When I run the original PHP code with these parameters, and these lines for outputting the code:
<?php
echo "build data to sign:\n";
print_r(buildDataToSign($params));
echo "\n";
echo "sign data:\n";
echo signData(buildDataToSign($params), 'secret');
?>
I get the following output:
build data to sign:
access_key=afc10315b6aaxxxxxcfc912xx812b94c,profile_id=E25C4XXX-4622-47E9-9941-1003B7910B3B,transaction_uuid=12345,signed_field_names=access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency,unsigned_field_names=,signed_date_time=2021-03-06 16:14:00,locale=en,transaction_type=credit,reference_number=12345,amount=50,currency=usd
sign data:
6V0iIqu3smGmadPK4KvRuHm1nNkuIVLBPbLg7VkA7M8=
So with your new Python version of this PHP code, you'll probably want to have a similar sign data value of 6V0iIqu3smGmadPK4KvRuHm1nNkuIVLBPbLg7VkA7M8= at the end with these parameters!
Because your Python example does not seem to get the same result as-is, after adding a return base64string to the end of your Python def, I get the following output:
sign_data:
b'NThFMjU4QTQyRjU2MkVDRDgzM0RCOEIwM0VDODczQTExNjc3MUNDMEM2OURGMDFGMjdFQkU3MEMzMDAyNjA3RQ=='
In order to match the PHP version of your code, I wanted to try to find out what was going on between the PHP and Python approaches in regard to the hmac and base64 parts.
When I broke down your PHP code example into steps relating to the hmac value and then later the base64 value, here is what I found (using a data message of 'hello' and a key of 'secret' to keep it simple):
Example PHP Code:
<?php
$hash_value = hash_hmac('sha256', 'hello', 'secret', true);
$base64_value = base64_encode($hash_value);
echo "hash value:\n";
echo $hash_value;
echo "\n";
echo "base64 value:\n";
echo $base64_value;
echo "\n";
?>
Example PHP Code Output:
;▒▒▒▒▒C▒|
base64 value:
iKqz7ejTrflNJquQ07r9SiCDBww7zOnAFO4EpEOEfAs=
That looks like some crazy binary-type data! Then, I wanted to try to make sure that the base64 value could be reproduced in Python. To do that, I used a simple approach in Python using the same values as earlier.
Example Python Code:
import base64
import hashlib
import hmac
# Based on your Python code example
hash_value = hmac.new(bytes('secret' , 'latin-1'), msg = bytes('hello', 'latin-1'), digestmod = hashlib.sha256).hexdigest().upper()
base64_value = base64.b64encode(bytes(hash_value, 'utf-8'))
print("hash value:")
print(hash_value)
print("base64 value:")
print(base64_value)
Example Python Code Output:
hash value:
88AAB3EDE8D3ADF94D26AB90D3BAFD4A2083070C3BCCE9C014EE04A443847C0B
base64 value:
b'ODhBQUIzRURFOEQzQURGOTREMjZBQjkwRDNCQUZENEEyMDgzMDcwQzNCQ0NFOUMwMTRFRTA0QTQ0Mzg0N0MwQg=='
So, like you found out earlier, something is causing this base64 value result on the Python side to be longer than the PHP version.
After looking into things more (especially seeing the strange data result in the PHP test code above), I found out that the hash_hmac() function in PHP has the option to return a result in binary form (with the true value at the end of the hash_hmac() in your PHP code example). On the Python side, it looks like you decided to use hmac.hexdigest() which I think I've used before in the past when I wanted a string-like value. For this case, however, I think you might want to get the value back as a binary value. To do this, it looks like you'll want to use hmac.digest() instead.
Modified Example Python Code:
import base64
import hashlib
import hmac
# Based on your Python code example
hash_value = hmac.new(bytes('secret' , 'latin-1'), msg = bytes('hello', 'latin-1'), digestmod = hashlib.sha256).digest()
base64_value = base64.b64encode(bytes(hash_value))
print("hash value:")
print(hash_value)
print("base64 value:")
print(base64_value)
Modified Example Python Code Output:
hash value:
b'\x88\xaa\xb3\xed\xe8\xd3\xad\xf9M&\xab\x90\xd3\xba\xfdJ \x83\x07\x0c;\xcc\xe9\xc0\x14\xee\x04\xa4C\x84|\x0b'
base64 value:
b'iKqz7ejTrflNJquQ07r9SiCDBww7zOnAFO4EpEOEfAs='
Now, the final base64 results appear to match between the example PHP and Python code.
In order for me to better understand what was different between the PHP and Python code, I ended up writing a simple translation of your PHP code into Python (and partly based on your Python code as well).
Here is what the related Python code looks like on my side (with example params):
import base64
import hmac
params = {
'access_key': 'afc10315b6aaxxxxxcfc912xx812b94c',
'profile_id': 'E25C4XXX-4622-47E9-9941-1003B7910B3B',
'transaction_uuid': "12345",
'signed_field_names': 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency',
'unsigned_field_names': '',
'signed_date_time': "2021-03-06 16:14:00",
'locale': 'en',
'transaction_type': "credit",
'reference_number': "12345",
'amount': "50",
'currency': "usd"
}
SECRET_KEY = 'secret'
def sign(params):
return sign_data(build_data_to_sign(params), SECRET_KEY)
def sign_data(data, secret_key):
return base64.b64encode(bytes(hmac.new(bytes(secret_key, 'latin-1'), msg=bytes(data, 'latin-1'), digestmod='sha256').digest()))
def build_data_to_sign(params):
data_to_sign = []
signed_field_names = params['signed_field_names'].split(',')
for field in signed_field_names:
data_to_sign.append(field + "=" + params[field])
return comma_separate(data_to_sign)
def comma_separate(data_to_sign):
return ','.join(data_to_sign)
When I use my code translation to check your Python code, I checked the values for the variables signed_field_names and DataToSign in your code, and I got the following results:
signed_field_names:
['access_key', 'profile_id', 'transaction_uuid', 'signed_field_names', 'unsigned_field_names', 'signed_date_time', 'locale', 'transaction_type', 'reference_number', 'amount', 'currency']
DataToSign:
['access_key=afc10315b6aaxxxxxcfc912xx812b94c', 'profile_id=E25C4XXX-4622-47E9-9941-1003B7910B3B', 'transaction_uuid=12345', 'signed_field_names=access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency', 'unsigned_field_names=', 'signed_date_time=2021-03-06 16:14:00', 'locale=en', 'transaction_type=credit', 'reference_number=12345', 'amount=50', 'currency=usd']
When I check the values with my code translation attempt, I get these values:
signed_field_names:
['access_key', 'profile_id', 'transaction_uuid', 'signed_field_names', 'unsigned_field_names', 'signed_date_time', 'locale', 'transaction_type', 'reference_number', 'amount', 'currency']
DataToSign:
access_key=afc10315b6aaxxxxxcfc912xx812b94c,profile_id=E25C4XXX-4622-47E9-9941-1003B7910B3B,transaction_uuid=12345,signed_field_names=access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency,unsigned_field_names=,signed_date_time=2021-03-06 16:14:00,locale=en,transaction_type=credit,reference_number=12345,amount=50,currency=usd
So it looks like your DataToSign = list(map('='.join, zip(signed_field_names, values))) line is specifying a list whereas my code attempt is specifying a string based on your original PHP example.
Because of this, I think you'll want to turn the result back into a string like this (though the variable name could also be written differently if you so choose):
DataToSignString = ','.join(DataToSign)
To save time in this long post, I also found that your message variable was different than my translation of your PHP code. To work around this, I made the message variable in your Python code set to the previously mentioned DataToSignString:
# Commenting out previous message line for now
# message = '{} {}'.format(DataToSignString, API_SECRET)
message = DataToSignString
Also, the following changes seem to be needed for your Python example:
signature = hmac.new(bytes(API_SECRET , 'latin-1'), msg = bytes(message , 'latin-1'), digestmod = hashlib.sha256).digest()
base64string = base64.b64encode(bytes(signature))
This way, you have a binary version of the hmac object. Also, it looks like the utf-8 part might not be needed for now in the base64encode part.
Finally, I added a return to return the calculated base64string while also converting it to a string before base64string is returned:
return str(base64string, 'utf-8')
When put together, here is what the modified code from your Python example looks like:
import base64
import datetime
import hashlib
import hmac
import pprint
import uuid
def sign():
access_key = 'afc10315b6aaxxxxxcfc912xx812b94c'
profile_id = 'E25C4XXX-4622-47E9-9941-1003B7910B3B'
transaction_uuid = "12345"
signed_field_names = 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency'
signed_date_time = "2021-03-06 16:14:00"
locale = 'en'
transaction_type = "credit"
reference_number = "12345"
amount = "50"
currency = "usd"
# Transform the String into a List
signed_field_names = [x.strip() for x in signed_field_names.split(',')]
# Get Values for each of the fields in the form
values = [access_key, profile_id, transaction_uuid,signed_field_names,'',signed_date_time,locale,transaction_type,reference_number,amount,currency]
# Insert the signedfieldnames in their place in the list (MUST BE KEPT)
values[3] = 'access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency'
# Merge the two lists as one
DataToSign = list(map('='.join, zip(signed_field_names, values)))
DataToSignString = ','.join(DataToSign)
# Hash Sha-256
API_SECRET = 'secret'
message = DataToSignString
signature = hmac.new(bytes(API_SECRET , 'latin-1'), msg = bytes(message , 'latin-1'), digestmod = hashlib.sha256).digest()
base64string = base64.b64encode(bytes(signature))
return str(base64string, 'utf-8')
result = sign()
print("sign_data:")
print(result)
The output for this code (with the given parameters) is:
sign_data:
6V0iIqu3smGmadPK4KvRuHm1nNkuIVLBPbLg7VkA7M8=
The value part of this output should be the same as the PHP output from earlier in this post. The earlier value was 6V0iIqu3smGmadPK4KvRuHm1nNkuIVLBPbLg7VkA7M8= and the latest output showed a result of 6V0iIqu3smGmadPK4KvRuHm1nNkuIVLBPbLg7VkA7M8=.
#summea You are god sent ! Thanks a ton ! I cannot believe how much of an effort you made, I am baffled !
If anyone is attempted to Implement Secure Acceptance / CyberSource and see this message, note that you would not be able to pass value="{{ signed_field_names }} in your front end as it is since the data looks like ['access_key','profile_id'].
You would need to either tweak it before sending it (which somehow gives out a Signature Mismatched on CYBS end) or simply hardcode the input field in payment_confirmation like so : value="access_key,profile_id,transaction_uuid,signed_field_names,unsigned_field_names,signed_date_time,locale,transaction_type,reference_number,amount,currency"/>
Related
Why am I getting a Runtime.MarshalError when using this code in Zapier?
The following code is giving me: Runtime.MarshalError: Unable to marshal response: {'Yes'} is not JSON serializable from calendar import monthrange def time_remaining_less_than_fourteen(year, month, day): a_year = int(input['year']) b_month = int(input['month']) c_day = int(input['day']) days_in_month = monthrange(int(a_year), int(b_month))[1] time_remaining = "" if (days_in_month - c_day) < 14: time_remaining = "No" return time_remaining else: time_remaining = "Yes" return time_remaining output = {time_remaining_less_than_fourteen((input['year']), (input['month']), (input['day']))} #print(output) When I remove {...} it then throws: 'unicode' object has no attribute 'copy'
I encountered this issue when working with lambda transformation blueprint kinesis-firehose-process-record-python for Kinesis Firehose which led me here. Thus I will post a solution to anyone who also finds this questions when having issues with the lambda. The blueprint is: from __future__ import print_function import base64 print('Loading function') def lambda_handler(event, context): output = [] for record in event['records']: print(record['recordId']) payload = base64.b64decode(record['data']) # Do custom processing on the payload here output_record = { 'recordId': record['recordId'], 'result': 'Ok', 'data': base64.b64encode(payload) } output.append(output_record) print('Successfully processed {} records.'.format(len(event['records']))) return {'records': output} The thing to note is that the Firehose lambda blueprints for python provided by AWS are for Python 2.7, and they don't work with Python 3. The reason is that in Python 3, strings and byte arrays are different. The key change to make it work with lambda powered by Python 3.x runtime was: changing 'data': base64.b64encode(payload) into 'data': base64.b64encode(payload).decode("utf-8") Otherwise, the lambda had an error due to inability to serialize JSON with byte array returned from base64.b64encode.
David here, from the Zapier Platform team. Per the docs: output: A dictionary or list of dictionaries that will be the "return value" of this code. You can explicitly return early if you like. This must be JSON serializable! In your case, output is a set: >>> output = {'Yes'} >>> type(output) <class 'set'> >>> json.dumps(output) Object of type set is not JSON serializable To be serializable, you need a dict (which has keys and values). Change your last line to include a key and it'll work like you expect: # \ here / output = {'result': time_remaining_less_than_fourteen((input['year']), (input['month']), (input['day']))}
Shopify HMAC parameter verification failing in Python
I'm having some trouble verifying the HMAC parameter coming from Shopify. The code I'm using per the Shopify documentation is returning an incorrect result. Here's my annotated code: import urllib import hmac import hashlib qs = "hmac=96d0a58213b6aa5ca5ef6295023a90694cf21655cf301975978a9aa30e2d3e48&locale=en&protocol=https%3A%2F%2F&shop=myshopname.myshopify.com×tamp=1520883022" Parse the querystring params = urllib.parse.parse_qs(qs) Extract the hmac value value = params['hmac'][0] Remove parameters from the querystring per documentation del params['hmac'] del params['signature'] Recombine the parameters new_qs = urllib.parse.urlencode(params) Calculate the digest h = hmac.new(SECRET.encode("utf8"), msg=new_qs.encode("utf8"), digestmod=hashlib.sha256) Returns False! hmac.compare_digest(h.hexdigest(), value) That last step should, ostensibly, return true. Every step followed here is outlined as commented in the Shopify docs.
At some point, recently, Shopify started including the protocol parameter in the querystring payload. This itself wouldn't be a problem, except for the fact that Shopify doesn't document that : and / are not to be URL-encoded when checking the signature. This is unexpected, given that they themselves do URL-encode these characters in the query string that is provided. To fix the issue, provide the safe parameter to urllib.parse.urlencode with the value :/ (fitting, right?). The full working code looks like this: params = urllib.parse.parse_qsl(qs) cleaned_params = [] hmac_value = dict(params)['hmac'] # Sort parameters for (k, v) in sorted(params): if k in ['hmac', 'signature']: continue cleaned_params.append((k, v)) new_qs = urllib.parse.urlencode(cleaned_params, safe=":/") secret = SECRET.encode("utf8") h = hmac.new(secret, msg=new_qs.encode("utf8"), digestmod=hashlib.sha256) # Compare digests hmac.compare_digest(h.hexdigest(), hmac_value) Hope this is helpful for others running into this issue!
import hmac import hashlib ... # Inside your view in Django's views.py params = request.GET.dict() # myhmac = params.pop('hmac') params['state'] = int(params['state']) line = '&'.join([ '%s=%s' % (key, value) for key, value in sorted(params.items()) ]) print(line) h = hmac.new( key=SHARED_SECRET.encode('utf-8'), msg=line.encode('utf-8'), digestmod=hashlib.sha256 ) # Cinderella ? print(hmac.compare_digest(h.hexdigest(), myhmac))
Transform Python HTML Inputs
I am writing a Python CGI that gets certain field values from a HTML page, I am at a point where I need to transform those input values to make them match table values of a database and get data from it. For example, if the field value of my UI is 'critic' then it should be transformed to '8_critic' to match the database table value. How can I do this please ? Something like this maybe ? #!C:\Python27\python.exe -u import cgi import json import cgitb print "Content-type: application/json\n\n" fs = cgi.FieldStorage() def transform(fs): for key in fs.keys(): if key == 'critic' : #key = '8_critic' elif key == 'somthing_else' : #key = 'another_thing' Any help would be appreciated.
The best way for me would me to have a translation table: tr = { 'critic' : '8_critic', 'foo' : 'bar' } and to make a new dictionnary by comprehension: nfs = { tr[k]:fs[k] for k in fs } Here, I assumed that you did not care about the FieldStorage at this point. If it isn't so, you can still do something like that: nfs = cgi.FieldStorage() for k in fs: nfs[ tr[k] ] = fs[k]
Python to ruby conversion
Hi guys I have a python script that posts some data to google and gets back response. The script is below net, cid, lac = 24005, 40242, 62211 import urllib a = '000E00000000000000000000000000001B0000000000000000000000030000' b = hex(cid)[2:].zfill(8) + hex(lac)[2:].zfill(8) c = hex(divmod(net,100)[1])[2:].zfill(8) + hex(divmod(net,100)[0])[2:].zfill(8) string = (a + b + c + 'FFFFFFFF00000000').decode('hex') try: data = urllib.urlopen('http://www.google.com/glm/mmap',string) r = data.read().encode('hex') print r except: print 'connect error' I want to get the same response with a ruby script. I am not able to form the request properly and I always get the badimplementation error or http 501 error. Could you tell me where the mistake is at? (The ruby script is attached below). require 'net/http' def fact(mnc,mcc,cid,lac) a = '000E00000000000000000000000000001B0000000000000000000000030000' b = cid.to_s(16).rjust(8,'0') + lac.to_s(16).rjust(8,'0') c = mnc.to_s(16).rjust(8,'0') + mcc.to_s(16).rjust(8,'0') string = [a + b + c + 'FFFFFFFF00000000'].pack('H*') url = URI.parse('http://www.google.com/glm/mmap') resp = Net::HTTP.post_form(url,string) print resp end puts fact(5,240,40242,62211)
From the documentation: Posts HTML form data to the specified URI object. The form data must be provided as a Hash mapping from String to String. You have to pass the parameters, if I understood that correctly, on the form: {"param1" => "value1", "param2"=>"value2"} I just didn't understand what are the names of the parameters you are passing on your request. Here are some usage examples for the method Net::HTTP::post_form, also from the official doc: Ex 1: uri = URI('http://www.example.com/search.cgi') res = Net::HTTP.post_form(uri, 'q' => 'ruby', 'max' => '50') puts res.body Ex2: uri = URI('http://www.example.com/search.cgi') res = Net::HTTP.post_form(uri, 'q' => ['ruby', 'perl'], 'max' => '50') puts res.body Link to the examples Hope it helps edit: function that accepts a String as a parameter to the post request: Net::HTTP::request_post
trying to automate translation on babelfish with python
I have modified a python babelizer to help me to translate english to chinese. ## {{{ http://code.activestate.com/recipes/64937/ (r4) # babelizer.py - API for simple access to babelfish.altavista.com. # Requires python 2.0 or better. # # See it in use at http://babel.MrFeinberg.com/ """API for simple access to babelfish.altavista.com. Summary: import babelizer print ' '.join(babelizer.available_languages) print babelizer.translate( 'How much is that doggie in the window?', 'English', 'French' ) def babel_callback(phrase): print phrase sys.stdout.flush() babelizer.babelize( 'I love a reigning knight.', 'English', 'German', callback = babel_callback ) available_languages A list of languages available for use with babelfish. translate( phrase, from_lang, to_lang ) Uses babelfish to translate phrase from from_lang to to_lang. babelize(phrase, from_lang, through_lang, limit = 12, callback = None) Uses babelfish to translate back and forth between from_lang and through_lang until either no more changes occur in translation or limit iterations have been reached, whichever comes first. Takes an optional callback function which should receive a single parameter, being the next translation. Without the callback returns a list of successive translations. It's only guaranteed to work if 'english' is one of the two languages given to either of the translation methods. Both translation methods throw exceptions which are all subclasses of BabelizerError. They include LanguageNotAvailableError Thrown on an attempt to use an unknown language. BabelfishChangedError Thrown when babelfish.altavista.com changes some detail of their layout, and babelizer can no longer parse the results or submit the correct form (a not infrequent occurance). BabelizerIOError Thrown for various networking and IO errors. Version: $Id: babelizer.py,v 1.4 2001/06/04 21:25:09 Administrator Exp $ Author: Jonathan Feinberg <jdf#pobox.com> """ import re, string, urllib import httplib, urllib import sys """ Various patterns I have encountered in looking for the babelfish result. We try each of them in turn, based on the relative number of times I've seen each of these patterns. $1.00 to anyone who can provide a heuristic for knowing which one to use. This includes AltaVista employees. """ __where = [ re.compile(r'name=\"q\">([^<]*)'), re.compile(r'td bgcolor=white>([^<]*)'), re.compile(r'<\/strong><br>([^<]*)') ] # <div id="result"><div style="padding:0.6em;">??</div></div> __where = [ re.compile(r'<div id=\"result\"><div style=\"padding\:0\.6em\;\">(.*)<\/div><\/div>', re.U) ] __languages = { 'english' : 'en', 'french' : 'fr', 'spanish' : 'es', 'german' : 'de', 'italian' : 'it', 'portugese' : 'pt', 'chinese' : 'zh' } """ All of the available language names. """ available_languages = [ x.title() for x in __languages.keys() ] """ Calling translate() or babelize() can raise a BabelizerError """ class BabelizerError(Exception): pass class LanguageNotAvailableError(BabelizerError): pass class BabelfishChangedError(BabelizerError): pass class BabelizerIOError(BabelizerError): pass def saveHTML(txt): f = open('page.html', 'wb') f.write(txt) f.close() def clean(text): return ' '.join(string.replace(text.strip(), "\n", ' ').split()) def translate(phrase, from_lang, to_lang): phrase = clean(phrase) try: from_code = __languages[from_lang.lower()] to_code = __languages[to_lang.lower()] except KeyError, lang: raise LanguageNotAvailableError(lang) html = "" try: params = urllib.urlencode({'ei':'UTF-8', 'doit':'done', 'fr':'bf-res', 'intl':'1' , 'tt':'urltext', 'trtext':phrase, 'lp' : from_code + '_' + to_code , 'btnTrTxt':'Translate'}) headers = {"Content-type": "application/x-www-form-urlencoded","Accept": "text/plain"} conn = httplib.HTTPConnection("babelfish.yahoo.com") conn.request("POST", "http://babelfish.yahoo.com/translate_txt", params, headers) response = conn.getresponse() html = response.read() saveHTML(html) conn.close() #response = urllib.urlopen('http://babelfish.yahoo.com/translate_txt', params) except IOError, what: raise BabelizerIOError("Couldn't talk to server: %s" % what) #print html for regex in __where: match = regex.search(html) if match: break if not match: raise BabelfishChangedError("Can't recognize translated string.") return match.group(1) #return clean(match.group(1)) def babelize(phrase, from_language, through_language, limit = 12, callback = None): phrase = clean(phrase) seen = { phrase: 1 } if callback: callback(phrase) else: results = [ phrase ] flip = { from_language: through_language, through_language: from_language } next = from_language for i in range(limit): phrase = translate(phrase, next, flip[next]) if seen.has_key(phrase): break seen[phrase] = 1 if callback: callback(phrase) else: results.append(phrase) next = flip[next] if not callback: return results if __name__ == '__main__': import sys def printer(x): print x sys.stdout.flush(); babelize("I won't take that sort of treatment from you, or from your doggie!", 'english', 'french', callback = printer) ## end of http://code.activestate.com/recipes/64937/ }}} and the test code is import babelizer print ' '.join(babelizer.available_languages) result = babelizer.translate( 'How much is that dog in the window?', 'English', 'chinese' ) f = open('result.txt', 'wb') f.write(result) f.close() print result The result is to be expected inside a div block . I modded the script to save the html response . What I found is that all utf8 characters are turned to nul . Do I need take special care in treating the utf8 response ?
I think you need to use: import codecs codecs.open instead of plain open, in your: saveHTML method, to handle utf-8 docs. See the Python Unicode Howto for a complete explanation.