Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 9 years ago.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Questions concerning problems with code you've written must describe the specific problem — and include valid code to reproduce it — in the question itself. See SSCCE.org for guidance.
Improve this question
I'm at a loose end here, it seems like such a simple problem so I'm hoping there is a simple answer!
I have a binary (approx 35m) which has 64 bytes of padded data every 2048 bytes starting at offset 1536 - I just want to remove this padding.
The first occurrence is 1536, then 3648,5760,7872,etc
(2112 bytes - 64 bytes of dummy data = 2048)
I've tried bvi,bbe,hexdump+sed+xxd and I'm clearly missing something.
Thanks in advance,
You didn't show any code, so I presume you need help wrapping your head around the algorithm. It's actually quite simple:
While you haven't reached the EOF of STDIN,
Read 2112 bytes from STDIN
From the bytes read, remove the 64 bytes starting at position 1536.
Print the remaining 2048 bytes to STDOUT.
In Perl,
binmode(STDIN);
binmode(STDOUT);
while (1) {
my $rv = read(STDIN, my $rec, 2112);
die $! if !defined($rv);
last if !$rv;
substr($rec, 1536, 64, '');
print($rec)
or die $!;
}
If you want to use Perl:
Open the file with the :raw layer. We don't want :utf8 or :crlf translation.
Then, we can seek to the positions we are interested in, and can read a few bytes
my $size = -s $filename;
open my $fh, "<:raw", $filename;
for (seek($fh, 1536, 0) ; tell($fh) + 2048 < $size ; seek($fh, 2048 - 64, 1)) {
read $fh, my $buffer, 64;
...;
}
Read
perldoc -f tell
perldoc -f seek
perldoc -f read
for further information
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 4 years ago.
Improve this question
I'm using struct to parse fixed width strings. However, I'm having some trouble with dealing fixed width strings larger than 1000 bytes.
For example, when I execute the following code:
import struct
fmt = '2s25s16s1s40s2s1s1s2s9s1s6s10s25s2s2s9s8s2s2s4x8s2x2s2s2s2s2s1x13s6s2s2s2s2s1x3s4s6s4s12x1s2s1x7s1s2s2s2s2s2s1x3s6x2s2x2s2x2s2x2s2x2s2x2s6s2x1s4x4s2s2s2s2s2s2s2s8x3s3s3s3s3s3s3s3s3s2s2s2s2s2s2s8s2x2s2s2s2s2s150s50s4x1s2s8s15x30s30s10s15s15s10s10s10s10s12s3s3s3s3s1s3s3s1x15s2s3s8s2s2s2s2s2s16s2s3s2x2s3s2x1s1s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s10x2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s2s2s3s3s42s6s21s3s3s3s3s3s3s3s3s3s3s3s3s7s'
parse = struct.Struct(fmt).unpack_from
line = '1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ123456'
parse(line.encode())
I get the following error:
Traceback (most recent call last):
File "so.py", line 6, in <module>
parse(line.encode())
struct.error: unpack_from requires a buffer of at least 1164 bytes
I've been searching for ways to set the buffer to 1164 bytes with no success.
Let's take a look to https://docs.python.org/3.0/library/struct.html#struct.unpack_from first, it says:
Unpack the buffer according to tthe given format. The result is a
tuple even if it contains exactly one item. The buffer must contain at
least the amount of data required by the format (len(buffer[offset:])
must be at least calcsize(fmt)).
Let's try first calcsize on your fmt, print(struct.calcsize(fmt)) says 1164.
Now let's see the len of your buffer len(line), it says 1050.
So the error is because you're not following the guidelines provided by the docs...
PS: parse((line+line[0:struct.calcsize(fmt)-len(line)]).encode())
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I'm making a program that encrypts a file using keys.
It can encrypt only numbers, letters, spaces, some symbols.
Etc.
This is text >>> h5D#I2%%&12s
My program can encrypt a file, too. (At least I'm working on it)
What if file contains characters like this ? - uún‰3«°Ø and also NULL, CAN or SOH characters.
I have an idea: I want to leave these and all other non-ascii characters unencrypted. But I don't know if Python can work with them.
P.S. Here is link to the project: (And It's unfinished, not working)
https://www.dropbox.com/sh/lq8j4vmci5c2vmh/AADeSTPVYeV13z5HRHp-NlWPa?dl=0
Python byte strings (type str in Python 2, bytes in Python 3) are just opaque sequences of bytes, where each byte has an integer value between 0 and 255.
How you treat those bytes is up to you. You could treat them as text; printing the text, splitting on whitespace, changing case, etc. Or you can just treat it as binary data, your choice. If you chose to treat the contents as text, then yes, some bytes are 'unprintable' because the ASCII codec hasn't assigned a printable glyph to those codepoints. Python, however, doesn't care.
Open your files in binary mode ('rb', 'wb', etc.) to make sure that line separators (\n, or \r or \r\n characters) are not translated from and to the platform native form.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 9 years ago.
Improve this question
I am trying to translate a Perl function into a Python function, but I am having trouble figuring out what some of the Perl to Python function equivalents.
Perl function:
sub reverse_hex {
my $HEXDATE = shift;
my #bytearry=();
my $byte_cnt = 0;
my $max_byte_cnt = 8;
my $byte_offset = 0;
while($byte_cnt < $max_byte_cnt) {
my $tmp_str = substr($HEXDATE,$byte_offset,2);
push(#bytearry,$tmp_str);
$byte_cnt++;
$byte_offset+=2;
}
return join('',reverse(#bytearry));
}
I am not sure what "push", "shift", and "substr" are doing here that would be the same in Python.
Any help will be much appreciated.
The Perl subroutine seems rather complicated for what it does, viz., taking chunks of two chars at a time (the first 16 chars) from the sent string and then reverses it. Another Perl option is:
sub reverse_hex {
return join '', reverse unpack 'A2' x 8, $_[0];
}
First, unpack here takes two characters at a time (eight times) and produces a list. That list is reversed and joined to produce the final string.
Here's a Python subroutine to accomplish this:
def reverse_hex(HEXDATE):
hexVals = [HEXDATE[i:i + 2] for i in xrange(0, 16, 2)]
reversedHexVals = hexVals[::-1]
return ''.join(reversedHexVals)
The list comprehension produces eight elements of two characters each. [::-1] reverses the list's elements and the result is joined and returned.
Hope this helps!
I realize that you are asking about the perl to python translation, but if you have any control over the perl, I would like to point out that this function is a lot more complicated than it needs to be.
The entire thing could be replaced with:
sub reverse_hex
{
my $hexdate = shift;
my #bytes = $hexdate =~ /../g; # break $hexdate into array of character pairs
return join '', reverse(#bytes);
}
Not only is this shorter, it is much easier to get your head around.
Of course, if you have no control over the perl, you are stuck with what you were dealt.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have this long unicode string in python. Of this unicode string I want to read first 1000 bytes.
Use case: I'm trying to send the email body content on a mobile number using the plivo API as a text message. This text message take maximum of 1000 bytes.
So I need to truncate first 1000 bytes from the email body content.
How can this be done ?
If you need the first 1000 bytes then you need to encode the Unicode value first, as the number of bytes varies with the encoding picked.
Then just slice the first 1000 bytes:
encoded = unicodevalue.encode('utf8')
sliced = encoded[:1000]
As it happens, the Plivo Send Message API requires exactly that; 1000 bytes of UTF-8 encoded data. You probably want to truncate the data further to not cut off multi-byte UTF-8 characters:
encoded = unicodevalue.encode('utf8')
sliced = encoded[:1000]
while True:
try:
sliced.decode('utf8')
except UnicodeDecodeError:
sliced = sliced[:-1] # remove one invalid byte
else:
break
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I am trying to convert this RFID Tag number got from this code;
import serial
ser = serial.Serial()
ser.port = "COM1"
ser.baudrate = 9600
ser.timeout = 3
ser.open()
if ser.open is True:
print "Port Not open"
while ser.isOpen():
#ser.timeout = 7
response = ser.read(17)
response = response.encode('hex')
print response
I am getting this 0000000000000000000213780510015dff which is a hexadecimal number, but I want to convert it to decimal or string. When I try to do that, I am getting a token error. How can I fix that?
You say you want to "convert… to string".
You can use unhexlify to do that, or decode('hex').
However, in your case, the only reason you have hex in the first place is that you called encode('hex'), so just… don't do that.
If you want to decode it to an int or a Decimal or something, you can do that by using the appropriate constructor, as Maxime's answer shows. However, rather than converting to hex just to decode as an int, you might want to just decode it directly. Or maybe you want to decode the hex string into a decimal string? Or maybe this is some UUID-style structure, and you want to use struct.unpack to decode it into pieces? Or…? Without knowing exactly what you're trying to do, it's hard to give an exact answer…
You can use int to convert a hexadecimal number into an integer.
>>> int("0000000000000000000213780510015dff", 16)
149595175772052991