We Keep Coding

Why is this first Byte printed as (b'c\x..) instead of (b'x63\x..)?

I am writing a program for padding oracle attacks and need bytearrays,
but if I define a new bytearray the first byte 0x63 gets printed different.
I have to XOR 2 bytearrays bytewise.
test = bytearray( [99,203,00] )
print(test)
print(hex(99))
Output:
bytearray(b'c\xcb\x00')
0x63
This is my first question here. Thanks for your help!

For string output python replaces printable hexcodes by the chr(hexcode) character for display purposes:
print('c', ord('c'),hex(ord('c'))) # c 99 '0x63'
t = bytearray([99,203,0])
print(t) # bytearray(b'c\xcb\x00')
print(t[0],t[1],t[2]) # 99 203 0
They are equivalent - but shorter to print out. You can get a all-hex representation like so:
t = bytearray([99,203,0])
t_hex = '\\'+'\\'.join( ( hex(i) for i in t) )
print(t_hex)
Output:
\0x63\0xcb\0x0

Parsing a string in Python with variable field sizes

I have This string of length 66.
RP000729SP001CT087ET02367EL048TP020DS042MF0220LT9.300000LN4.500000. Two alphabets (keyword like RP) show next is value until the next keyword.
Now I am parsing the string keeping the number of bytes constant between two keywords i.e. 000729 between RP and SP. And following is the code to parse like that.
msgStr = "RP000729SP001CT087ET02367EL048TP020DS042MF0220LT9.300000LN4.500000"
Ppm = msgStr[msgStr.find("RP")+2:msgStr.find("SP")]
Speed = msgStr[msgStr.find("SP")+2:msgStr.find("CT")]
Coolent_temp = msgStr[msgStr.find("CT")+2:msgStr.find("ET")]
ETime = msgStr[msgStr.find("ET")+2:msgStr.find("EL")]
E_load = msgStr[msgStr.find("EL")+2:msgStr.find("TP")]
Throttle_pos = msgStr[msgStr.find("TP")+2:msgStr.find("DS")]
Distance = msgStr[msgStr.find("DS")+2:msgStr.find("MF")]
MAF = msgStr[msgStr.find("MF")+2:msgStr.find("LT")]
Lat = msgStr[msgStr.find("LT")+2:msgStr.find("LN")]
Lon = msgStr[msgStr.find("LN")+2:]
print Ppm, Speed, Coolent_temp, ETime, E_load, Throttle_pos, Distance, MAF, Lat, Lon
Output:
000729 001 087 02367 048 020 042 0220 9.300000 4.500000
Now I want to collect if there are any number of bytes between two keywords. examples are given below
Example No. 1:
Example1_msgStr= "RP729SP14CT087ET2367EL48TP20DS42MF0220LT0.000000LN0.000000"
Expected Ouput 1:
729 14 087 2367 48 20 42 0220 0.000000 0.000000
Example No. 2:
Example2_msgStr = "RP72956SP134CT874ET02367EL458TP20DS042MF0220LT53.000LN45.00"
Expected Ouput 2:
72956 134 874 02367 458 20 042 0220 53.000 45.00

You should use a regular expression to find variable length matches between two strings:
import re
regex = r'RP(\d+)SP'
strings = ['RP729SP14CT087ET2367EL48TP20DS42MF0220LT0.000000LN0.000000',
'RP72956SP134CT874ET02367EL458TP20DS042MF0220LT53.000LN45.00']
for string in strings:
match = re.search(regex,string)
print('Matched:',match.group(1))
In the regex the brackets () specify a group to store and \d+ means 1 or more number characters. So the whole regex RP(\d+)SP will find a variable length string of numbers between RP and SP.
This shows you how to do one case, you'll need to loop through your delimiters (RP, SP, CT etc. etc.) to capture all the information you want. If the delimiters always come in the same order you can build one enormous regex to capture all the groups at once ...

If you want to check for characters between your delimiters, you can use your code and then turn any of your variables into bool type. If the string is not empty it means that there is something in that string and thus it returns True. If the string is empty, it returns False:
msgStr = 'RP000729SP001CT087ET02367EL048TP020DS042MF0220LT9.300000LN4.500000'
rpm = msgStr[msgStr.find("RP")+2:msgStr.find("SP")] # Outputs '000729'
Speed = msgStr[msgStr.find("SP")+2:msgStr.find("CT")] # Outputs '001'
coolent_temp = msgStr[msgStr.find("CT")+2:msgStr.find("ET")] # Outputs '087'
ETime = msgStr[msgStr.find("ET")+2:msgStr.find("EL")] # '02367'
e_load = msgStr[msgStr.find("EL")+2:msgStr.find("TP")] # '048'
throttle_pos = msgStr[msgStr.find("TP")+2:msgStr.find("DS")] # '020'
Distance = msgStr[msgStr.find("DS")+2:msgStr.find("MF")] # Outputs '042'
MAF = msgStr[msgStr.find("MF")+2:msgStr.find("LT")] # Outputs '0220'
Lat = msgStr[msgStr.find("LT")+2:msgStr.find("LN")] # Outputs '9.300000'
Lon = msgStr[msgStr.find("LN")+2:] # Outputs '4.500000'
bool(rpm) # Outputs True
bool(Speed) # Outputs True
bool(coolent_temp) # Outputs True
bool(ETime) # Outputs True
bool(e_load) # Outputs True
bool(throttle_pos) # Outputs True
bool(Distance) # Outputs True
bool(MAF) # Outputs True
bool(Lat) # Outputs True
bool(Lon) # Outputs True
You could check several fields being not empty at the same time:
all_filled = bool(rpm) and bool(Speed) and bool(coolent_temp) and \
bool(ETime) and bool(e_load) and bool(throttle_pos) and bool(Distance) \
and bool(MAF) and bool(Lat) and bool(Lon)
The way your code is set, if you try it with msgStr1 you get already the separation you desired:
Example1_msgStr= "RP729SP14CT087ET2367EL48TP20DS42MF0220LT0.000000LN0.000000"
# ... your code ...
print (rpm, Speed, coolent_temp, ETime, e_load, throttle_pos, Distance, MAF, Lat, Lon)
#> 729 14 087 2367 48 20 42 0220 0.000000 0.000000
Example2_msgStr= "RP72956SP134CT874ET02367EL458TP20DS042MF0220LT53.000LN45.00"
# ... your code ...
print (rpm, Speed, coolent_temp, ETime, e_load, throttle_pos, Distance, MAF, Lat, Lon)
#> 72956 134 874 02367 458 20 042 0220 53.000 45.00

issue with formatting characters

having a bit of an issue displaying characters
i have a payload recieved from a protocol request :
538cb9350404521a6c44020404563b152606102001085800020002aabb0000563b1526000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
the length of that is 509
what i want to display is the first 4 bytes, then 1 byte, then 1 byte
538cb935
04
04
currently to view the payload i am doing the following :
tm = struct.unpack(">L", payload[0:4])
print "Time : ", tm
ouroripv = struct.unpack(">b", payload[5])
print "Our onion address : "
print "Ip version : ", ouroripv
ouroraddrlen = struct.unpack(">b", payload[6]) # Giving a length of 82 etc atm
print "Ip length : ", ouroraddrlen
i get the result :
Time : (1401731381,)
Our onion address :
Ip version : (4,)
Ip length : (82,)
as you can see the Ip length, the 6th byte in on the payload is displaying 82 rather than the 4 it should be, what is the correct struct.unpack command that is needed to display this ?
how can i do this ?
Thanks guys

in python, the slicing doesn't include the last value, so payload[0:4] takes the first 4 bytes, from 0 to 3.
payload[3] is the fourth byte
payload[4] is the fifth byte

How to sort output data into columns and rows

I have an output that looks like this, where the first number corresponds to the count of the type below (e.g. 72 for Type 4, etc)
72
Type
4
51
Type
5
66
Type
6
78
Type
7
..etc
Is there a way to organize this data to look something like this:
Type 4 = 72 times
Type 5 = 51 times
Type 6 = 66 times
etc..
Essentially, the question is how to take a single column of data and sort /organize it into something more readable using bash, awk, python, etc. (Ideally, in bash, but interested to know how to do in Python).
Thank you.

Use paste to join 3 consecutive lines from stdin, then just rearrange the fields.
paste - - - < file | awk '{print $2, $3, "=", $1, "times"}'

It's simple enough with Python to read three lines of data at a time:
def perthree(iterable):
return zip(*[iter(iterable)] * 3)
with open(inputfile) as infile:
for count, type_, type_num in perthree(infile):
print('{} {} = {} times'.format(type_.strip(), type_num.strip(), count.strip()))
The .strip() calls remove any extra whitespace, including the newline at the end of each line of input text.
Demo:
>>> with open(inputfile) as infile:
... for count, type_, type_num in perthree(infile):
... print('{} {} = {} times'.format(type_.strip(), type_num.strip(), count.strip()))
...
Type 4 = 72 times
Type 5 = 51 times
Type 6 = 66 times
Type 7 = 78 times

In Bash:
#!/bin/bash
A=() I=0
while read -r LINE; do
if (( (M = ++I % 3) )); then
A[M]=$LINE
else
printf "%s %s = %s times\n" "${A[2]}" "$LINE" "${A[1]}"
fi
done
Running bash script.sh < file creates:
Type 4 = 72 times
Type 5 = 51 times
Type 6 = 66 times
Type 7 = 78 times
Note: With a default IFS ($' \t\n'), read would remove leading and trailing spaces by default.

Try this awk one liner:
$ awk 'NR%3==1{n=$1}NR%3==2{t=$1}NR%3==0{print t,$1,"=",n,"times"}' file
Type 4 = 72 times
Type 5 = 51 times
Type 6 = 66 times
Type 7 = 78 times
How it works?
awk '
NR%3==1{ # if we are on lines 1,4,7, etc (NR is the record number (or the line number)
n=$1 # set the variable n to the first (and only) word
}
NR%3==2{ # if we are on lines 2,5,7, etc
t=$1 # set the variable t to the first (and only) word
}
NR%3==0{ # if we are on lines 3,6,9, etc
print t,$1,"=",n,"times" # print the desired output
}' file

Read integers from file using struct.unpack in Python

Suppose I have a file name num.txt as below:
1 2 3 4 5
6 7 8 9 0
I want to read 3 integers from this file, that is 1 2 3.
I know that struct.unpack might do the trick, but I just cannot get it right.
Here is how I did it:
fp = open('num.txt', 'rb')
print struct.unpack('iii', fp.read(12)) #right?
Anyone can help me with this?
PS
This is how I got file num.txt:
fp = open('num.txt', 'wb')
fp.write('1 2 3 4 5\n6 7 8 9 0')
fp.close()

You don't use struct to read numbers from a text file. It is for reading data from a binary file -- where the first byte is actually 0x01 rather than a byte order mark or the encoded value of the character '1'.
You just want
three_ints = [int(x) for x in numfile.readline().strip().split(' ')[:3]]
If you're only interested in the first three numbers, or
all_ints = [[int(x) for x in line.split()] for line in numfile]
if you want a list of lists of the ints on each line.

struct is used for C-style binary representations of numbers. If you have text representations instead then you should just pass them to int().
>>> [int(x) for x in '1 2 3 4 5'.split()]
[1, 2, 3, 4, 5]