We Keep Coding

How can I read back from a buffer using ctypes?

I have a third-party library, and I need to use one function in python script from it. Here it is:
ReadFromBlob(PVOID blob, INT blob_size, PCSTR section, PCSTR key, const void **buffer, UINT * size)
blob - some pointer? to bytes to read from
blob_size - blob size in bytes
section and key - string values like "Image"
buffer - bytes to read to
size - buffer size
The documentation gives an example of how to use it:
UINT size = 0;
PVOID buffer = NULL;
ReadFromBlob(<blob>, <blob_size>, "MainImage", "Image", &buffer, &size);
I'm not familiar with C, so argument types confusing me. I need to be able to read values from the buffer in python.
This is what I have so far:
from ctypes import *
lib = cdll.LoadLibrary(path_to_lib)
with open(filepath, 'rb') as file:
data = file.read()
blob_size = c_int(len(data))
blob = cast(c_char_p(data), POINTER(c_char * blob_size.value))
b = bytes()
size = c_uint(len(b))
buffer = cast(cast(b, c_void_p), POINTER(c_char * size.value))
lib.ReadFromBlob(blob, blob_size, b"MainImage", b"Image", buffer, pointer(size))
But I still get an empty buffer in the end. Please help me.

It looks like the function searches the blob for data based on the section and key and returns a pointer into the blob data and a size, so I made a test function that just echoes back the blob and size as the output parameters:
#include <windows.h>
#include <stdio.h>
__declspec(dllexport)
void ReadFromBlob(PVOID blob, INT blob_size, PCSTR section, PCSTR key, const void **buffer, UINT * size) {
printf("section=\"%s\" key=\"%s\"\n",section,key);
*buffer = blob; // just echo back input data for example
*size = (UINT)blob_size;
}
The types look like Windows types, and ctypes has a submodule wintypes with Windows definitions that help get the types right. Make sure to set the .argtypes and .restype correctly with parallel ctypes types for the Windows types. This helps ctypes check that arguments are passed correctly.
import ctypes as ct
from ctypes import wintypes as w
dll = ct.CDLL('./test')
# Note the parallels between C types and ctypes types.
# PVOID is just "pointer to void" and LPVOID mean the same thing, etc.
dll.ReadFromBlob.argtypes = w.LPVOID,w.INT,w.LPCSTR,w.LPCSTR,ct.POINTER(w.LPCVOID),w.LPUINT
dll.ReadFromBlob.restype = None
# storage for returned values, passed by reference as output parameters
buffer = w.LPCVOID()
size = w.UINT()
dll.ReadFromBlob(b'filedata',8,b'Section',b'Key',ct.byref(buffer),ct.byref(size))
print(ct.cast(buffer,ct.c_char_p).value,size.value)
Output showing the received section and key, and printing the returned blob data and size:
section="Section" key="Key"
b'filedata' 8

How to turn python AES function into C# code

I have a part of python code that I want to use in my C# project, but I can't find a right way to achieve it.
python code:
def getCiphertext(plaintext, key = key_, cfb_iv = iv_, size = 128):
message = plaintext.encode('utf-8')
cfb_cipher_encrypt = AES.new(key, AES.MODE_CFB, cfb_iv, segment_size = size)
mid = cfb_cipher_encrypt.encrypt(message)
return hexlify(mid).decode()
I have tried the C# code below, but the result is different:
using System.Security.Cryptography;
public static string AesEncrypt(string str, string key, string IVString)
{
Encoding encoder = Encoding.UTF8;
byte[] toEncryptArray = Encoding.UTF8.GetBytes(str);
RijndaelManaged rm = new RijndaelManaged
{
Key = encoder.GetBytes(key),
Mode = CipherMode.CFB,
BlockSize = 128,
Padding = PaddingMode.PKCS7,
IV = encoder.GetBytes(IVString),
};
ICryptoTransform cTransform = rm.CreateEncryptor();
byte[] resultArray = cTransform.TransformFinalBlock(toEncryptArray, 0, toEncryptArray.Length);
return ToBCDStringLower(resultArray);//result
}
public static string ToBCDStringLower(byte[] buffer)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < buffer.Length; i++)
{
sb.Append(buffer[i].ToString("x2"));
}
return sb.ToString();
}
Thanks guys all!

.NET's CFB implementation:
CFB in .NET is problematic. In .NET Framework it is supported, in .NET Core only from .NET 5.0.
In addition, .NET Framework and .NET Core allow different segment sizes, but both support 8-bit and 128-bit, which corresponds to the most common variants, namely CFB8 and CFB128 (or full block CFB). The segment size is an additional parameter in CFB which corresponds to the bits encrypted per encryption step, see CFB.
Another peculiarity is that in .NET the plaintext size must be an integer multiple of the segment size. This is remarkable (actually already a bug), since CFB is a stream cipher mode that does not require padding.
Therefore, for CFB, with the exception of CFB8, padding is generally required. In the case of CFB128 to the full block, allowing the default padding PKCS7 to be applied.
Thus, to obtain the ciphertext that corresponds to the unpadded plaintext, the ciphertext must be truncated to the plaintext size.
Comparison of Python and C# code:
In the posted Python code, the segment size in the argument list defaults to 128 bits (the PyCryptodome default is 8 bits). In the C# code, the segment size (here denoted as FeedbackSize) is not specified, so the default value of 128 bits is used.
Thus, unless a segment size other than 128 bits is explicitly specified in the Python code, both codes apply the same segment size.
Also, in the C# code, padding (PKCS7) is done as required by the C# implementation. Therefore, when the ciphertext of the C# code is truncated to the plaintext size, it matches the ciphertext of the Python code.
The following example uses the code you posted unchanged:
string plaintext = "The quick brown fox jumps over the lazy dog";
string key = "01234567890123456789012345678901";
string cfb_iv = "0123456789012345";
string ciphertext = AesEncrypt(plaintext, key, cfb_iv);
string ciphertextTrunc = ciphertext.Substring(0, plaintext.Length * 2); // *2 since AesEncryptOP returns the ciphertext hex encoded
Console.WriteLine(ciphertext);
Console.WriteLine(ciphertextTrunc);
Output:
09f1e464983a7d25305d5b865386e477d97b34b9a6365372ef83b78e495692489c1848a124345eb808eb66d268c6d1ad
09f1e464983a7d25305d5b865386e477d97b34b9a6365372ef83b78e495692489c1848a124345eb808eb66
As you can verify, the shortened ciphertext corresponds to the output of the Python code.
Note that as explained in the 1st section, padding is required for CFB128. Changing the padding to PaddingMode.None will result in a CryptographicException: The input data is not a complete block. However, with CFB8 this would be possible.
BouncyCastle:
An alternative to the .NET built-in implementation is BouncyCastle, which implements CFB as stream cipher mode so that no padding is needed. The following code:
using Org.BouncyCastle.Crypto.Engines;
using Org.BouncyCastle.Crypto.Modes;
using Org.BouncyCastle.Crypto.Parameters;
using Org.BouncyCastle.Crypto;
...
public static string Encrypt(string str, string keyString, string IVString)
{
byte[] inputBytes = Encoding.UTF8.GetBytes(str);
byte[] IV = Encoding.UTF8.GetBytes(IVString);
byte[] key = Encoding.UTF8.GetBytes(keyString);
AesEngine engine = new AesEngine();
CfbBlockCipher blockCipher = new CfbBlockCipher(engine, 128);
BufferedBlockCipher cipher = new BufferedBlockCipher(blockCipher);
KeyParameter keyParam = new KeyParameter(key);
ParametersWithIV keyParamWithIv = new ParametersWithIV(keyParam, IV);
cipher.Init(true, keyParamWithIv);
byte[] outputBytes = new byte[cipher.GetOutputSize(inputBytes.Length)];
int length = cipher.ProcessBytes(inputBytes, outputBytes, 0);
cipher.DoFinal(outputBytes, length);
string encryptedInput = ToBCDStringLower(outputBytes);
return encryptedInput;
}
directly (i.e. without truncation) returns the result of the Python code.

How to read from pointer address in Python?

I want to read in a Python script a number of bytes starting from a specific address. E.g., I want to read 40000 bytes starting from 0x561124456.
The pointer is given from a C# app. I want to use this method to pass data between the app and script. I've used a TCP socket via localhost, but I want to try this method also.
How can I do this?

If you really want to, enjoy:
import ctypes
g = (ctypes.c_char*40000).from_address(0x561124456)
Looks like segfault fun. There are good socket-connection libraries on both languages (sockets, RPC etc...), so I would think about this again if this is for some large project.

Once I got a pointer of memory location from C, I found "list(listSize * listDataType).from_address(memoryPointer)" created a internal copy of C memeory. If the data in memory is huge, Python takes a long time to create a list object by using internal copy. To avoid internal copy, I used the ctypelib.as_array in python:
import ctypes
import binascii
import numpy as np
myCfunslib.getData.restype = ctypes.c_void_p
#myCfunslib.getData.restype=ctypes.POINTER(ctypes.c_ubyte)#no need to cast
dataSize = 1092 * 1208
#call the c function to get the data memory pointer
cMemoryPointer = myCfunslib.getData();
newpnt = ctypes.cast(cMemoryPointer, ctypes.POINTER(ctypes.c_ubyte))
# and construct an array using this data
DataBytes = np.ctypeslib.as_array(newpnt, (dataSize,)) #no internal copy
print "the mid byte of the data in python side is ", DataBytes[dataSize/2]

I happened to work on the similar issue. My python script load .so library to get an image buffer address from c++ .so. After I got the buffer address, I need to be able to read each byte in the buffer. I used "from_address" to create a list object:
imageBytes = list(c_ubyte * dataSize).from_address(pointer)
The following shows the details how to get memory address passed from c++ to pyth and how to access the memory data on python side too. In c++ code frameprovider.cpp:
dataPackPtr = new DataPack();
DataPack * getFrame(){
uint32_t width = 1920;
uint32_t height = 1208;
const size_t buffersize = width * height * 4;//rgba, each color is one byte
unsigned char* rgbaImage = (unsigned char * )malloc(buffersize);
memset(rgbaImage, 0, buffersize); // set all the buffer data to 0.
dataPackPtr->width = width;
dataPackPtr->height = height;
dataPackPtr->buffersize = buffersize;
dataPackPtr->bufferPtr = rgbaImage;
return dataPackPtr;
}
extern "C" {
DataPack* getFrame_wrapper(){
return getFrame();
}
}
My python:
import ctypes
import binascii
lib = ctypes.cdll.LoadLibrary('/libpath/frameprovider.so')
print vars(lib)
class dataPack(ctypes.Structure):
_fields_ = [("width",ctypes.c_int),
("height",ctypes.c_int),
("buffersize",ctypes.c_int),
("bufferAddress", ctypes.c_void_p)]
lib.getFrame_wrapper.restype = ctypes.POINTER(dataPack)
data = lib.getFrame_wrapper()
print "in python the w= ", data.contents.width, "h=",data.contents.height
print "the buffersize=",data.contents.height
imageBytes = list(
(data.contents.buffersize * ctypes.c_ubyte).
from_address(data.contents.bufferAddress))
print "the len of imageBytes are ", len(imageBytes)
print imageBytes[data.contents.buffersize -1] #print the last byte in the buffer
print "in python, the hex value of element 12 is ", hex(imageBytes[12])

What is the equivalent of python's file.read() in objective C / iOS?

Say that I have the Python code:
someString = file("filename.jpg").read()
How can I replicate this line of code in Objective C on iOS? I'm trying to convert a jpg into a string, for eventual url encoding so I can pass it on to the Tumblr API. I tried base64 encoding to convert the image to a string initially, but that doesn't seem to get a valid response from tumblr. I also tried NSUTF8Encoding, but those attempts return nil strings (see below).
Why am I doing this? Someone from Tumblr posted a Python example of how to submit multiple photos for a photoset in response to this thread, and the process appears to start with converting the images to strings using file.read(). The API unfortunately doesn't take a simple ‘multipart/form-data’ request, only ‘application/x-www-form-urlencoded’ format.
What sort of encoding is Python using to do this read(), and how can I replicate it in Objective C on the iPhone?
I have tried the following:
NSString *utf8String = [NSString stringWithContentsOfFile:[[NSBundle mainBundle] pathForResource:#"filename" ofType:#"jpg"] encoding:NSUTF8StringEncoding error:&error];
This gives a nil string and error 261 The operation couldn’t be completed.
Using base64 encoding gives me a string, but the Tumblr API doesn't accept it after url encoding it, so I assume it's not doing what Python is doing with read() (though I could just be missing a step). The Tubmlr API responds with "status":400,"msg":"Bad Request"},"response":{"errors":["Error uploading photo."]
There is a similar question on stackoverflow about uploading photos to photosets on Tumblr here, but this question is more specific to the posted Python code.

I figured the answer out by simply logging the string -- it looks like it's a hexadecimal format, with \xXX where XX is the hexadecimal code for each byte, with ASCII substitutions for certain easily-printed characters.
\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00 \x00\x00\x00\x1d\x08\x06\x00\x00\x00\xcb\x9en\x87\x00\x00\x01\xc9IDATH\r\xc5\x971K\x03A\x10\x85s\nB \x10\xb0\n\xa4\xb2J\x95* \x08\x82\x95\xbfA\x10\xfc\x01V\xb6\x82\x95?\xc0J\x10\x04[[\xc16U#\xb0\x8d`/(\x8a\x82\x08Z\t\x12\x89\xdf$\xd9p\x9c\xae\xfb&\x9c8\xf0\xd8\xbd\xdb\xb7o^\xe6\x96\xb9K6\x1c\x0e+\xb3D\x96e\xf3\xeck\xb3\xffj\x96\xfda\xcf\\\x98\xa8#\x89\xdb\xe0\x1c\xfe\x00\xf4\x99\x0f#\x17\xec\x80\xba\xaa3\xe5Y\x05\x14\xb0\xa1\x06\x0e\xc0=\xb0\xb2\xfd\x84\x17\xee\xaf(z\x81#%72\xb1\x1bIZ4r\x03\xaf\x16\x12\xa4F\xc9\x00\x82\xf6\xbcM\xb8\x98,v\xbd\x9fJ\x1c\xd6\xd53\xd0 \xf9\x12Pc]%\xaa\x06\x9e\x11\xfcTE\xe1\xd9y\x91B2#\xb9>P{\x90\x14\xc7$\xe3K!\x19\x98(Y\x15\xd4xR\x89\x1e\x03\xef\xaa(\xbc7\x95\xeb1\xd0RE\xe1-\xab\\\x8f\x01\xcf\x19\xb8\xfb\x0b\x03\x9e\x9e\x7f\xad\x1a\x90\x1a\x915\rb\x15\xdc\x82X\xf3\t\xf7\x1f\xe1tB\xa3I\x8d\xb2\x81\x89\x89=\xc1\xc0v*i~\xdds\x06\xc8]QJ\xabpLk\x1cy7\xa99;\x16#\x1f\x84r\x17\xc7K\xe3\xa4t\xf2\xeb\xde\n\xd8;\xe1\xb7\x13n\xcd\xca^\\z\xe4\xdd\xa4\xe6\xa8^\x80\xe2\xaf.^\x9f\xa4t\xf2\xeb\xd2!$i\x13\x1c\n\xc9\x83\x993\xb8\xad|\xa2\xd8<j\x00\x01{\xde\x9b\xa0\x0b\x82\xb0w\xec\xb1w\x0bTe\x03\x90;\xc0~\xad}^y\x13\xc6\xf8\xafh\x1d\x81o\xfdaZ\x01\x167\x80\xf2\x8ccI\xd4\xfb=\xf2\xac\x85\x8a\x8c\x0cp\xe3\x14\xa8\x02e\xf1\x8e\xcd\x84\x85}>\x95%\xea\xd5iX\x1f\xf0|\xeb\x99\xe12\xa3i\x06\xfc\x7f&\xca\xb30\xaa\xc0byzn\xa5\xbaU\xe0?\rT\xbf\x00\x87\x89 \xa8s3+7\x00\x00\x00\x00IEND\xaeB`\x82
The above can be replicated in Objective C by reading in the image to NSData, and then getting the description (which is a list of 2 digit hexadecimal-formatted byte values). After that, you could loop through each byte and decide how to represent it based on its value:
NSData *data = [NSData dataWithContentsOfFile:[[NSBundle mainBundle] pathForResource:#"anImage" ofType:#"jpg"]];
NSString *hexFormatString = [data describe];
hexFormatString = [[hexFormatString stringByTrimmingCharactersInSet:[NSCharacterSet characterSetWithCharactersInString:#"<>"]] stringByReplacingOccurrencesOfString:#" " withString:#""];
NSMutableString * newString = [NSMutableString string];
for (int x=0; x<[hexFormatString length]; x+=2) {
NSString *component = [hexFormatString substringWithRange:NSMakeRange(x, 2)];
int value = 0;
sscanf([component cStringUsingEncoding:NSASCIIStringEncoding], "%x", &value);
if ((value <=46 && value >= 45) || (value <=57 && value >= 48) || (value <=90 && value >= 65) || (value == 95) || (value <=122 && value >= 97)) { //48-57, 65-90, 97-122
[newString appendFormat:#"%c", (char)value];
}
else {
[newString appendFormat:#"%%%#", [component uppercaseString]];
}
}

Is that what you're looking for ?
NSData *data = [NSData dataWithContentsOfFile:#"filename.jpg"];
NSString *string = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]

String Concatenation appends Junk Characters with ctypes

I am appending "00000000" to a string and it works fine for the first time. However, when run second time "Junk Characters" are appended instead of "000000". This is the sample code how I am doing it in the actual program.
File one.py
# File One.py
from two import *
def One():
while(1):
key = Two()
key = key + "00000000"
print key
def main():
One()
if __name__ == "__main__":
main()
File two.py
from ctypes import *
import binascii
handle = None
def Two():
global handle
libc = CDLL('libthree.so', DEFAULT_MODE, handle)
if not handle:
handle = libc._handle
buffer = create_string_buffer(16)
libc.Three(buffer)
return binascii.b2a_hex(buffer)
File three.c - Generates libthree.so
#include "stdio.h"
#include "stdlib.h"
void Three(char * buffer)
{
long value = 0x78563412;
memcpy(buffer,&value,4);
memcpy(buffer + 4,&value,4);
memcpy(buffer+ 8,&value,4);
memcpy(buffer+ 12,&value,4);
return;
}
int main()
{
return 0;
}

create_string_buffer can be initialized with a string or a length. If initialized with a string s, it allocates space for len(s)+1 chars, so that the terminating null can be appended. But if initialized with an integer value, create_string_buffer assumes that since you are the human, you must know what you are doing, and allocates just that much space. Unfortunately, your C code is writing into the full 16 characters of space, so there is no room for a null terminator. When this works for you, it is purely by accident that the byte after the allocated storage happens to be 0 (null), terminating the string. Later on, that memory gets used for something else, and then you get the garbage. Try using create_string_buffer(16+1) instead, and see if things improve for you.
The docs also suggest using the .string() method of the returned string buffer object, so that you explicitly apply null-terminated semantics - the alternative is .raw() which will read past nulls up to the defined buffer size. Ironically, if you specify key = key.raw() + "000000", this may give you exactly the 16-character sized buffer you originally specified, and bypass the junk characters that way.
So here are two things you can try:
In One, do:
key = key.raw() + "00000000"
Or in Two, change to:
buffer = create_string_buffer(16+1)
But please don't do both.