We Keep Coding

Bytes Command Function Not Working for PySerial

I have a function which takes a set of values and converts it into a bytes string. For example, I need:
input_array = ['E9', '01','06','57','4A','01','F4','01','01','EF']
## needs to become b'\xE9\x01\x06\x57\x4A\x01\xF4\x01\x01\xEF'
The function I have is:
def string_to_command(inp):
new_string = ''
for i in inp:
new_string += r'\x' + i
return new_string.encode('latin-1')
When I print both commands:
print(string_to_command(input_array))
print(b'\xE9\x01\x06\x57\x4A\x01\xF4\x01\x01\xEF')
# OUTPUT b'\\xE9\\x01\\x06\\x57\\x4A\\x01\\xF4\\x01\\x01\\xEF'
# OUTPUT b'\xe9\x01\x06WJ\x01\xf4\x01\x01\xef'
I am not sure what is going on here. The last one with b"" actually commands my output device properly, the other does not. How do I fix this problem?

You can convert to int first, then to bytes.
>>> a = ['E9', '01','06','57','4A','01','F4','01','01','EF']
>>> bytes(int(x, base=16) for x in a)
b'\xe9\x01\x06WJ\x01\xf4\x01\x01\xef'

The last one with b"" actually commands my output device properly, the other does not. How do I fix this problem?
I think what is going on here is that you are trying to add \x to every value and then encoding it from there. The problem with that is 'E9' and \x'E9' are not the same:
>>> 'E9'.encode()
b'E9'
>>> '\xE9'.encode()
b'\xc3\xa9' <-- Not the same
>>>
One method I like is bytes.fromhex(s), where s is your 'hex string'.
input_array = ['E9', '01','06','57','4A','01','F4','01','01','EF']
# use "".join(input_array) to get all values combined
# into one string
res = bytes.fromhex( "".join(input_array) )
Outputs :
b'\xe9\x01\x06WJ\x01\xf4\x01\x01\xef'

You can join a list of bytes converted from the original array with latin-1 encoding. So adding \x as join sequence, it will output the desired bytes object:
input_array = ['E9', '01','06','57','4A','01','F4','01','01','EF']
print(b''.join([bytes.fromhex(x) for x in input_array]))
Output:
b'\xe9\x01\x06WJ\x01\xf4\x01\x01\xef'

modify data in struct in python

I would like to know if it's possible to modify an already made struct.
What I want is to calculate the CRC of header and then write the CRC value to the 3rd position in header.
import struct
import crcmod
crcfunc = crcmod.mkCrcFun(0x112, 0x00, 0x00)
header = struct.pack('!hii', 1, 2, 0)
crcvalue = crcfunc(header)

You can not do it directly: struct.pack returns a bytes object, which like str, is immutable. However, you can very easily create the updated object:
header = header[:-4] + c.to_bytes(4, byteorder=sys.byteorder)
If you really meant to compute the CRC value on everything excluding the last element, you can do something like this instead:
header = struct.pack('!hi', 1, 2)
header += crcfunc(header).to_bytes(4, byteorder=sys.byteorder)

Since the result of calling struct.pack is to create a byte string, it is immutable.
You could either splice the crc into the byte string by slicing it, or create a new byte string:
header2 = struct.pack('!hii', 1, 2, crcvalue)
Alternatively, if you want to create a mutable byte string, you could create a buffer, and use struct.pack_into instead of struct.pack.

Separating parts of a <class 'byte'> and printing it out to screen

so I am capturing packets with Pydivert. I can print out the full packet payload by using
print(packet.tcp.payload)
OR
print(packet.payload)
output was
b'\x03\x00\x34\xe2\xd1' //continued like this
same output in both cases. I printed out the type by using
print(type(packet.payload))
This showed the type to be
<class 'byte'>
I would like to take say the first 10 byte positions from the output and type it out and also save it into a variable so when I'm modifying the payload, I exclude the initial bytes and then modify the remaining parts. So I can somehow attach the separated out bytes to my newly created bytes to create a final byte stream like for example:
TotalByteStream = (initial bytes which I separated out) + b'\x03\x00\x34\xe2\xd1\x78\x23\x45\x79' //continued like this as needed
//And then do
packet.payload = TotalByteStream
Is this possible?

I'm not sure I understand your question, but you can manipulate bytes in a manner similar to strings.
If you have your original payload:
>>> payload_1 = b'\x03\x00\xf4\xe2\xd1'
>>> type(payload_1)
<class 'bytes'>
>>> payload_1
b'\x03\x00\xf4\xe2\xd1'
You can slice of the first few bytes
>>> part = payload_1[:2]
>>> part
b'\x03\x00'
And later create a new payload where you prepend the part variable
>>> payload_2 = part + b'\xf5\xe5\xd5'
>>> payload_2
b'\x03\x00\xf5\xe5\xd5'
>>> payload_1
b'\x03\x00\xf4\xe2\xd1'
So you get a new payload with the same starting bytes. Does this answer your question? Or did I misunderstand your issue?

redis get the value from the result set

I have result returned as from ZrangebyScore function as [b'101']. I would like to extract only 101 value and discard other additional characters. It is in byte form. How to convert it in Integer format using Python.

If you are using Py3 try this:
mylist = [b'101']
val = int(mylist[0].decode())

How to address indicies in an array using HEX

I have a file that I read lines from and manipulate the strings.
Here is an example of a few of the lines (the file is Intel's HEX format if you're interested):
:10DE50003EDE179280DB0338D2C32202023CD2D3CB
:10DE600022021792A0DB0338E2C32202023CE2D373
:10DE7000220292533EDEB0906400C4FF022082432F
:10DE80003EDE3741324190C3B8240013FDDBFF056D
:10DE900057494D453420D0D88CDEFDDB8FDEFF03A3
A buddy suggested I create an array with the first 4:7 bytes as the index, EG, DE50, then use the remaining 16 bytes as the data (00 after DE50 is not used, and last byte is not used). He said I could use HEX and add let's say, 10 to the DE50 to get DE5A and therefore locate the byte associated with that index. Problem is, I can't figure out a way to do that. Is it even possible? This would allow me to then address any byte I want by knowing the HEX index which would be really powerful.
Thank you!

There is an Intel Hex package in pypi perhaps you should look at that first
Here are some examples copied from the docs.
Once created, an IntelHex object can be loaded with data. This is only
necessary if “source” was unspecified in the constructor. You can also
load data several times (but if addresses in those files overlap you
get exception AddressOverlapError). This error is only raised when
reading from hex files. When reading from other formats, without
explicitly calling merge, the data will be overwritten. E.g.:
>>> from intelhex import IntelHex
>>> ih = IntelHex() # create empty object
>>> ih.loadhex('foo.hex') # load from hex
>>> ih.loadfile('bar.hex',format='hex') # also load from hex
>>> ih.fromfile('bar.hex',format='hex') # also load from hex
NOTE: using IntelHex.fromfile is recommended way.
All of the above examples will read from HEX files. IntelHex also
supports reading straight binary files. For example:
>>> from intelhex import IntelHex
>>> ih = IntelHex() # create empty object
>>> ih.loadbin('foo.bin') # load from bin
>>> ih.fromfile('bar.bin',format='bin') # also load from bin
>>> ih.loadbin('baz.bin',offset=0x1000) # load binary data and place them
>>> # starting with specified offset
Finally, data can be loaded from an appropriate Python dictionary.
This will permit you to store the data in an IntelHex object to a
builtin dictionary and restore the object at a later time. For
example:
>>> from intelhex import IntelHex
>>> ih = IntelHex('foo.hex') # create empty object
>>> pydict = ih.todict() # dump contents to pydict
...do something with the dictionary...
>>> newIH = IntelHex(pydict) # recreate object with dict
>>> another = IntelHex() # make a blank instance
>>> another.fromdict(pydict) # now another is the same as newIH

You're on the right track here, but you can't have an "array" indexed by hex characters. Arrays, and lists, are always indexed by integers, starting with 0.
If you know the initial offset (which you do, from the first line), you can make an index very easily. For example, everything from 'DE50' to 'DE5F' should be line #0, right? So, convert that DE50 to an integer, divide by 16 (truncating fractions), and subtract 0xDE50. Like this:
with open('hexfile.txt') as f:
lines = list(f)
offset = int(lines[0][4:7], 16) // 16
def get_line(hex_index):
index = int(hex_index, 16) // 16
return lines[index - offset]
Alternatively, you could use a dict keyed off the hex indices, instead of a list, and then do what your friend suggested:
with open('hexfile.txt') as f:
lines = {line[4:7]: line for line in f}
def get_line(hex_index):
base_hex_index = hex_index[:3] + '0'
return lines[base_hex_index]
However, this seems to be just adding extra complexity to your data structure for no benefit. If you've got sequential lines, just treat them sequentially. And if you've got numbers as hex strings, just convert them to numbers to treat them as indices.