We Keep Coding

How to create bytes list in python?

I am facing a challenge to create a bytes list in python. I just want to convert the int list into bytes list as mentioned in expected result. The problem statement is that I want to send the expected output to the serial device connected to the com port and with current output the serial device is not encouraging the '\\' as a separator. Please suggest me the correct way to handle the '\' in a list of bytes.
cmdlist = [2, 12, 1, 1, 1, 0, 0, 1, 3, 7, 42, 101, 85, 18]
#Convert CMD list to Hex List
for i in range(len(cmdlist)):
cmdlist[i] = hex(cmdlist[i])
f_cmdList = ''
#Convert hex CMD list to string List
for i in range(len(cmdlist)):
f_cmdList += '\\' + (cmdlist[i])
Final_cmdlist = (bytes(f_cmdList,'utf-8'))
print(Final_cmdlist)
Current output : b'\\0x2\\0xc\\0x1\\0x1\\0x1\\0x0\\0x0\\0x1\\0x3\\0x7\\0x2a\\0x65\\0x55\\0x12'
Expected output : b'\0x2\0xc\0x1\0x1\0x1\0x0\0x0\0x1\0x3\0x7\0x2a\0x65\0x55\0x12'
Thank You !

You can convert a list to bytes simply by using the bytes constructor. Your method is trying to create a string that contains the string representation of a byte array, which won't work when sent to the serial device:
>>> cmdlist = [2, 12, 1, 1, 1, 0, 0, 1, 3, 7, 42, 101, 85, 18]
>>> bytes(cmdlist)
b'\x02\x0c\x01\x01\x01\x00\x00\x01\x03\x07*eU\x12'

You get what you say you expect if you replace your
f_cmdList += '\\' + (cmdlist[i])
with this:
f_cmdList += '\0' + cmdlist[i][1:]
Still not convinced that you really want that, though.

'numpy.ndarray' object has no attribute 'D'

import numpy as np
rowlist = np.array([[0, 2, 3, 4, 5], [0, 0, 0, 3, 2], [1, 2, 3, 4, 5], [0, 0, 0, 6, 7], [0, 0, 0, 9, 9]])
new_rowlist = []
rows_left = set(range(len(rowlist)))
col_label_list = sorted(rowlist[0].D, key=hash)
for c in col_label_list:
rows_with_non_zero = [ r for r in rows_left if rowlist[r][c] != 0 ]
if rows_with_non_zero != []:
pivot = rows_with_non_zero[0]
new_rowlist.append(rowlist[pivot])
rows_left.remove(pivot)
for r in new_rowlist:
print(r)
So i'm following the Coding the Matrix by Philip Klein book lessons and on one of the chapter on Gaussian Elimination, this keeps erroring 'numpy.ndarray' object has no attribute 'D'
What I wanted was to be able to sort the matrix called rowlist. Any idea how to overcome this ? im doing this on jupyter notebook if it is of any help
Thanks in advance!

There is a typo. There is no rowlist[0].D, change the line to the following
col_label_list = sorted(rowlist[0], key=hash)
Also, maybe from next time as a learning process, try and debug a code yourself by locating which line is causing the error (if you read the error message properly).

Reading a text document containing python list into a python program

I have a text file(dummy.txt) which reads as below:
['abc',1,1,3,3,0,0]
['sdf',3,2,5,1,3,1]
['xyz',0,3,4,1,1,1]
I expect this to be in lists in python as below:
article1 = ['abc',1,1,3,3,0,0]
article2 = ['sdf',3,2,5,1,3,1]
article3 = ['xyz',0,3,4,1,1,1]
That many articles have to be created as many lines present in dummy.txt
I was trying the following things:
Opened the file, split it by '\n' and appended it to an empty list in python, it had extra quotes and square brackets hence tried to use 'ast.literal_eval' which did not work as well.
my_list = []
fvt = open("dummy.txt","r")
for line in fvt.read():
my_list.append(line.split('\n'))
my_list = ast.literal_eval(my_list)
I also tried to manually remove additional quotes and extra square brackets using replace, that did not help me either. Any leads much appreciated.

This should help.
import ast
myLists = []
with open(filename) as infile:
for line in infile: #Iterate Each line
myLists.append(ast.literal_eval(line)) #Convert to python object and append.
print(myLists)
Output:
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

fvt.read() will produce the entire file string, so that means line will contain a single character string. So this will not work very well, you also use literal_eval(..) with the entire list of strings, and not a single string.
You can obtain the results by iterating over the file handler, and each time call literal_eval(..) on a single line:
from ast import literal_eval
with open("dummy.txt","r") as f:
my_list = [literal_eval(line) for line in f]
or by using map:
from ast import literal_eval
with open("dummy.txt","r") as f:
my_list = list(map(literal_eval, f))
We then obtain:
>>> my_list
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]

ast.literal_eval is the right approach. Note that creating a variable number of variables like article1, article2, ... is not a good idea. Use a dictionary instead if your names are meaningful, a list otherwise.
As Willem mentioned in his answer fvt.read() will give you the whole file as one string. It is much easier to exploit the fact that files are iterable line-by-line. Keep the for loop, but get rid of the call to read.
Additionally,
my_list = ast.literal_eval(my_list)
is problematic because a) you evaluate the wrong data structure - you want to evaluate the line, not the list my_list to which you append and b) because you reassign the name my_list, at this point the old my_list is gone.
Consider the following demo. (Replace fake_file with the actual file you are opening.)
>>> from io import StringIO
>>> from ast import literal_eval
>>>
>>> fake_file = StringIO('''['abc',1,1,3,3,0,0]
... ['sdf',3,2,5,1,3,1]
... ['xyz',0,3,4,1,1,1]''')
>>> result = [literal_eval(line) for line in fake_file]
>>> result
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]
Of course, you could also use a dictionary to hold the evaluated lines:
>>> result = {'article{}'.format(i):literal_eval(line) for i, line in enumerate(fake_file, 1)}
>>> result
{'article2': ['sdf', 3, 2, 5, 1, 3, 1], 'article1': ['abc', 1, 1, 3, 3, 0, 0], 'article3': ['xyz', 0, 3, 4, 1, 1, 1]}
where now you can issue
>>> result['article2']
['sdf', 3, 2, 5, 1, 3, 1]
... but as these names are not very meaningful, I'd just go for the list instead which you can index with 0, 1, 2, ...

When I do this:
import ast
x = '[ "A", 1]'
x = ast.literal_eval(x)
print(x)
I get:
["A", 1]
So, your code should be:
for line in fvt.read():
my_list.append(ast.literal_eval(line))

Try this split (no imports needed) (i recommend):
with open('dummy.txt','r') as f:
l=[i[1:-1].strip().replace("'",'').split(',') for i in f]
Now:
print(l)
Is:
[['abc', 1, 1, 3, 3, 0, 0], ['sdf', 3, 2, 5, 1, 3, 1], ['xyz', 0, 3, 4, 1, 1, 1]]
As expected!!!

Python Socket, MAC address format

I am new to python, and trying to create a packet parser. Then I stripped off the ethernet packet from the socket from the following commands:
>>raw=socket.socket(socket.PF_PACKET,socket.SOCK_RAW,socket.htons(0x800)) # Raw Packed Created
>>raw=raw.recvfrom(2048) #Received data from socket
>>raw
('\x01\x00^\x00\x00\x01T\xe6\xfc\xd0\x93\x10\x08\x00F\xc0\x00 \x00\x00#\x00\x01\x02Bm\xc0\xa8\x01\x01\xe0\x00\x00\x01\x94\x04\x00\x00\x11d\xee\x9b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', ('eth0', 2048, 2, 1, 'T\xe6\xfc\xd0\x93\x10'))
>> ether=raw[0][0:14] #Ethernet header
>>ether_unpack=struct.unpack("!6s6s2s",ether) #Unpack ethernet header into string form
>>ether_unpack #print unpacked data
('\x01\x00^\x00\x00\x01', 'T\xe6\xfc\xd0\x93\x10', '\x08\x00')
>>ether_hex=binascii.hexlify(ether_unpack[0]) #converted data into hexadecimal format
.
.
.
tcpHeader=raw[0][34:54] #strippin TCP packet
tcp_hdr=struct.unpack("!HH16s", tcpHeader) // TCP header unpack
First Question : what is the format of '\x01\x00^\x00\x00\x01'; What is the format of numerics in my first output
Second question: ether=raw[0][0:14]; [0:14] that takes 14 bytes from first tuple? Requires confirmation
Third Question: tcp_hdr=struct.unpack("!HH16s", tcpHeader) What does the first argument do? I took this command from somewhere, and cannot figure out why the there are 'double H' in the first argument.
Thanks in advance!

1) The first element of raw is an hex format string, you can convert it to an int list using:
>> payload = [int(x.encode('hex'), 16) for x in raw[0]]
>> [1, 0, 94, 0, 0, 1, 84, 230, 252, 208, 147, 16, 8, 0, 70, 192, 0, 32, 0, 0, 64, 0, 1, 2, 66, 109, 192, 168, 1, 1, 224, 0, 0, 1, 148, 4, 0, 0, 17, 100, 238, 155, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2) Yes, raw[0][0:14] takes the first 14 bytes of the first element of the raw tuple.
3) The first argument for the unpack function is the format in which the given string should be unpacked (https://docs.python.org/2/library/struct.html). The first character of the format string can be used to indicate the byte order, size and alignment of the packed data; the ! character in this case is for big-endian or little-endian. The 'double H' means that two unsigned short integers should be unpacked (2 bytes each), followed by a 16 characters string defined by 16s (16 bytes), accordingly tcpHeader is a 20 bytes string and tcp_hdr stores a (int, int, string) tuple.

How to convert a keycode to an char in python?

I'm writing a program in Linux which reads and distinguish inputs from two USB devices(two barcode readers) which simulates a keyboard.
I've already can read inputs from USB, but it happens before OS translate keycode in a charactere.
For example, when I read 'a' i got 24, 'b' 25, etc....
For example, when I read 'a' i got 4, 'b' 5, etc....
Is there any way to convert that code in a char without manual mapping?
Some output exemples:
KEYPRESS = a output = array('B', [0, 0, 4, 0, 0, 0, 0, 0])
KEYPRESS = SHIFT + a output = array('B', [2, 0, 4, 0, 0, 0, 0, 0])
KEYPRESS = 1 output = array('B', [0, 0, 30, 0, 0, 0, 0, 0])
KEYPRESS = ENTER output = array('B', [0, 0, 81, 0, 0, 0, 0, 0])
thx!

Use the chr function. Python uses a different character mapping (ASCII) from whatever you're receiving though, so you will have to add 73 to your key values to fix the offset.
>>> chr(24 + 73)
'a'
>>> chr(25 + 73)
'b'

I've already can read inputs from USB, but it happens before OS
translate keycode in a charactere.
The problem seems to me in your interface or the driver program.
In ASCII 'a' is supposed to have ordinal value 97 whose binary representation is 0b1100001, where as what you are receiving is 27 whose binary representation is 0b11000, similarly for 'b' you were supposed to received '0b1100010' instead you received 25 which is 0b11001. Check your hardware to determine if the 1st and the 3rd bit is dropped from the input.
What you are receiving is USB scan code. I do not think there is a third party python library to do the conversion for you. I would suggest you to refer any of the USB Scan Code Table and from it, create a dictionary of USB Scan Code vs the corresponding ASCII.