Is there a Python function that will respond like the Wire.available function in arduino to get all the data on the wire rather than having to specify how many bytes to grab?
This is what I have now, and it works fine, but I have to know how much data is coming down the wire, or it will provide unexpected results.
for i in range(0, 13):
data += chr(bus.read_byte(address));
Thanks!
Not a perfect solution, but I found a way around knowing exactly how many bytes are on the way.
On the Arduino, I specified the max size of the buffer, (128), add my data, then zero out the rest, and then send the whole thing. On the Pi, I receive the whole buffer, and then the first thing that happens is to filter the \x00 characters. It's not perfect, but it works for now.
for i in range(0, 128):
data += chr(bus.read_byte(address))
print repr(data)
#prints the whole string as it is received
data = filter(lambda a: a != '\x00')
print repr(data)
#prints the string without any '\x00' characters.
I use the PIGPIO library for i2c commands on the raspberrypi, it has functions much closer to wire.
http://abyz.co.uk/rpi/pigpio/python.html#i2c_read_device
I think that's the function you're looking for.
Related
I've been playing around with scapy and want to read through and analyse every hex byte. So far I've been using scapy simply because I don't know another way currently. Before just writing tools myself to go through the pcap files I was wondering if there was an easy way to do it. Here's what I've done so far.
packets = rdpcap('file.pcap')
tcpPackets = []
for packet in packets:
if packet.haslayer(TCP):
tcpPackets.append(packet)
When I run type(tcpPackets[0]) the type I get is:
<class 'scapy.layers.l2.Ether'>
Then when I try to covert the Ether object into a string it gives me a mix of hex and ascii (as noted by the random parenthesis and brackets).
str(tcpPackets[0])
"b'$\\xa2\\xe1\\xe6\\xee\\x9b(\\xcf\\xe9!\\x14\\x8f\\x08\\x00E\\x00\\x00[:\\xc6#\\x00#\\x06\\x0f\\xb9\\n\\x00\\x01\\x04\\xc6)\\x1e\\xf1\\xc0\\xaf\\x07[\\xc1\\xe1\\xff0y<\\x11\\xe3\\x80\\x18 1(\\xb8\\x00\\x00\\x01\\x01\\x08\\n8!\\xd1\\x888\\xac\\xc2\\x9c\\x10%\\x00\\x06MQIsdp\\x03\\x02\\x00\\x05\\x00\\x17paho/34AAE54A75D839566E'"
I have also tried using hexdump but I can't find a way to parse through it.
I can't find the proper dupe now, but this is just a miss-use/miss-understanding of str(). The original data is in a bytes format, for instance x = b'moo'.
When str() retrieves your bytes string, it will do so by calling the __str__ function of the bytes class/object. That will return a representation of itself. The representation will keep b at the beginning because it's believed to distinguish and make it easier for humans to understand that it's a bytes object, as well as avoid encoding issues I guess (alltho that's speculations).
Same as if you tried accessing tcpPackets[0] from a terminal, it would call __repr__ and show you something like <class 'scapy.layers.l2.Ether'> most likely.
As an example code you can experiment with, try this out:
class YourEther(bytes):
def __str__(self):
return '<Made Up Representation>'
print(YourEther())
Obviously scapy's returns another representation, not just a static string that says "made up representation". But you probably get the idea.
So in the case of <class 'scapy.layers.l2.Ether'> it's __repr__ or __str__ function probably returns b'$\\xa2\\....... instead of just it's default class representation (some correction here might be in place tho as I don't remember/know all the technical namification of the behaviors).
As a workaround, this might fix your issue:
hexlify(str(tcpPackets[0]))
All tho you probably have to account for the prepended b' as well as trailing ' and remove those accordingly. (Note: " won't be added in the beginning or end, those are just a second representation in your console when printing. They're not actually there in terms of data)
Scapy is probably more intended to use tcpPackets[0].dst rather than grabing the raw data. But I've got very little experience with Scapy, but it's an abstraction layer for a reason and it's probably hiding the raw data or it's in the core docs some where which I can't find right now.
More info on the __str__ description: Does python `str()` function call `__str__()` function of a class?
Last note, and that is if you actually want to access the raw data, it seams like you can access it with the Raw class: Raw load found, how to access?
You can put all the bytes of a packet into a numpy array as follows:
for p in tcpPackets:
raw_pack_data = np.frombuffer(p.load, dtype = np.uint8)
# Manipulate the bytes stored in raw_pack_data as you like.
This is fast. In my case, rdpcap takes ~20 times longer than putting all the packets into a big array in a similar for loop for a 1.5GB file.
I am trying to read serial data from a device that outputs in a mix of ASCII and binary, using Python 3.
The message format is: "$PASHR,msg type,binary payload,checksum,\r\n" (minus the quotes)
To make it more interesting, there are several different message types, and they have different payload lengths, so I can't just read X bytes (I can infer the payload length based on the message type). The sequence of a variable number of messages of each type (around 15 in total) is sent every 20 seconds, at 115200 baud.
I haven't been able to read this with serial.readline(), probably because of newlines embedded in the payload.
I think that if I could set the line-end character to the sequence "$PASHR" that would give me a way to frame the messages -- ie, everything between one $PASHR and the next is one message, and the likelihood of seeing the sequence in the binary payload is nil. But I have not been able to make it work either using serial.newline = b'$PASHR' or readuntil(b'$PASHR') -- I still get variable length reads. I suspect that the serial.timeout setting enters into the solution, but I am not sure how.
Here's the last version I ended up with last night:
delimiter = b'$PASHR'
while True:
if self.serial.in_waiting:
message = self.serial.read_until(delimiter)
Every method I've tried gives a variable response length, when each record of a give should be the same length. Is there a way to set the newline to that multi-character string, or is there a different/better approach I should use?
Thanks!
I think I figured it out. This seems to work with a timeout of 1 second:
self.serial.timeout = 1
delimiter = b'$PASHR'
if self.serial.in_waiting:
message = self.serial.read_until(delimiter)
if len(message) > 6: # ignore carryover PASHR'
***process message***
When I looked carefully at the variable length returns, I discovered that the message sequence after the first incomplete cycle was:
b'$PASHR' (6 bytes)
b',MPC,*data**checksum*\r\n$PASHR' (108 bytes)
...
b',MPC,*data**checksum*\r\n' (102 bytes)
(delay)
b'$PASHR' (6 bytes)
The delimiter from the last message in the sequence is chopped off and output at the beginning of the next sequence. I can deal with that.
I will try Deepstop's idea of the inter_byte timeout as it is probably more elegant than using the raw timeout, and might get around the partial message problem.
Introduction my problem:
I am trying to write a C++ program that receives data over UDP using WinSock2.
For that, I have a prewritten Python script that sends 10-byte packets to a specified port, where the C++ program then receives them. I have gotten the data transfer to work, however I am confused about the data that is being sent.
My problem:
I am running the Python script from cmd, which prints the sent text on console. I have also added a line into the script, which converts the string to bytes to verify, what exactly is being sent. The first line is the one I added, fairly simple:
logger.debug("Sending packet len %s, data %s", sizeof(packet), bytes(packet))
logger.debug("Sending packet len %s, data %s", sizeof(packet), packet)
And this is the output in my terminal (from 3 different packets sent):
What confuses me, is I would expect the hex code to contain only hexadecimal symbols, but instead there are some quite seemingly random symbols/letters there as well, even though the actual printed text looks just fine. Can someone explain to me, where do these symbols come from, as I am not sure how am I supposed to interpret this information on the receiving end in my C++ code.
Your script does not print plain hex, but the interpreted hex string. Some of your hex chars are interpretable and displayable in ASCII, others are not. I tested the string replacement in my interpreter and you see the same results:
>>> "%s".encode() % b"\xfa\x3f\x00"
b'\xfa?\x00'
With \x3f being the ASCII hex code for ?.
Python tries to be helpful (whether it actually helps is debatable) by printing bytes which correspond to an ASCII symbol as the ASCII symbol.
>>> t1 = b'test'
>>> t2 = b'\x74\x65\x73\x74'
>>> t1 == t2
True
>>> print(t1, t2)
b'test' b'test'
I'm writing a program in Python 3.4.1 that uses PySerial to test some hardware.
Bytes are read from the serial port one at a time, and then appended to list. When the list reaches a certain size, it is sent for processing.
Depending on the incoming data, the data sometimes has to be processed before the list is full, hence byte-by-byte operation.
The list then comes back as:
[b'F', b'o', b'o']
For part of the test script, I need to be able to convert this to a string, so that I can just print:
Foo
My solution is:
b''.join([b'F', b'o', b'o']).decode("ascii")
But it just feels wrong. Is there a better approach to this?
IMO, this is slightly more readable, but I wouldn't complain if I came across your code in review. Tested in Python 2.7:
>>> bytearray([b'F', b'o', b'o']).decode('ascii')
u'Foo'
If you don't like how join looks, you can do following:
bytes.join(b'', [b'F', b'o', b'o']).decode('ascii')
It's almost the same thing as in your code. I don't think you'll find any better approach.
I've been stuck on this one for a while now. I need to send a serial command to a device in python. Here's an example of the format:
ser = serial.Serial('/dev/ttyO1',baudrate=115200)
ser.write('\xAA\x04\x05\x00\x00\x00')
I'm able to communicate just fine if I communicate using the format .write(~hex bytes~)
However, when I need to receive larger chunks of data, the device's communication protocol splits things into packets and I have to confirm receiving each packet. To avoid getting carpal tunnel syndrome while typing out individual ser.write() commands for each packet I want to write a loop that does the packet counting/confirmation for me. here's what I have:
for n in range(0,num_packets,1):
x = '\\x'"%0.2x"%(n)
print('Picture Ready Host Ack')
ser.write('\xAA\x0E\x00\x00' + x + '\x00')
time.sleep(.1)
response = ser.read(ser.inWaiting())
image_write(response,base_path, filename)
However, the double slashes ('\x'..) gives me two slashes:
'\xaa\x0e\x00\x00\\x05\x00'
While a single slash ('\x'...) returns a problem when I define the variable x:
ValueError: invalid \x escape
...Help?
If instead of defining x as '\\x'"%0.2x"%(n), you do x = chr(n), everything should work.
\xij just expands to the character chr(int(ij, 16))
Since you are anyway sending bytes, I would say in a loop, send your first chunk. Do not add anything to it, then send your delimiter by chr(int(n),16) and subsequently the next chunks. This should solve your problem.