I have raspberry pi connected with xbees and a motion sensor, receiving data from the motion sensor connected to one of the xbee. It would then send the data to my raspberry pi. Is there any way I could manipulate or split the output such as the status being only True/False and the address =\x00\x13\xa2\x00#\xbbJ as I wanted to store the address into database if status=="True".
So if I do
if status[0]['dio-0'] == True :
print "Yes"
cur = con.cursor()
cur.execute("INSERT ignore into sensor(sensor_id, status) VALUES(%s,True)",(add[0]))
con.commit()
But the address stored into the database is weird characters instead of \x00\x13\xa2\x00#\xbbJ. or should I do any other ways?
This is the codes.
from xbee import XBee
import serial
PORT = '/dev/ttyUSBXBEE'
BAUD_RATE = 9600
# Open serial port
ser = serial.Serial(PORT, BAUD_RATE)
# Create API object
xbee = XBee(ser)
def decodeReceivedFrame(response):
add = str(response['source_addr_long'])
status = response['samples']
return [add, status]
# Continuously read and print packets
while True:
try:
response = xbee.wait_read_frame()
decodedData = decodeReceivedFrame(response)
status = decodeReceivedFrame(response)[1]
print status
print decodedData
add= decodedData[0]
except KeyboardInterrupt:
break
ser.close()
And this is the output.
[{'dio-0': True}]
['\x00\x13\xa2\x00#\xbbJ}', [{'dio-0': True}]]
[{'dio-0': False}]
['\x00\x13\xa2\x00#\xbbJ}', [{'dio-0': False}]]
In the database
+------------+--------+
| sensor_id | status |
+------------+--------+
| ¢ #»J} | 1 |
+------------+--------+
The variable sensor_id is an array of bytes, and it sounds like you want to store it in a human-readable format.
One way is to convert it to a formatted string before storing it in the database.
sensor_id = ':'.join("%02X" % ord(b) for b in add)
That statement loops through the bytes in the address (for b in add), formats each as a two-character hex string ("%02X" % ord(b)), and then joins each of those strings together with a colon in between (':'.join()).
Related
I wrote a small pyserial interface to read the data from the COM port after issuing a command. For eg : in my case my system has a lot of network interface so i need to validate whether all the interfaces are up using ifconfig command. But when i gave this command , the output of the command is getting truncated at the last few lines. The approximate size of the output in bytes would be 6500-7000 bytes but i am receiving only around 6000-6150 bytes all the time. Please find my code below
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
data_all = b" "
time.sleep(5)
while True:
bytetoread = com_serial.inWaiting()
time.sleep(2)
print ("Bytetoread: " , bytetoread)
data = com_serial.read(bytetoread)
data_all += data
if bytetoread < 1:
break
print ("Data:", data_all)
com_serial.close()
'''
**Output:
Bytetoread: 3967
Bytetoread: 179
Bytetoread: 2049
Bytetoread: 0
**
Data: *********with missing data at the end.
I am not sure why the logs are missing?
I have tried another approach.
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
time.sleep(5)
data_all = b" "
data_all = com_serial.read(100000000)
print (data_all)
com_serial.close()
'''
Here also the last few logs are getting truncated.
The root cause seems to be inadequate buffer size of the Tx and Rx serial buffer. By increasing the buffer size using .set_buffer_size() resolved the issue.
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.set_buffer_size(rx_size = 12800, tx_size = 12800)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
data_all = b" "
data_all = com_serial.read(100000000)
print (data_all)
com_serial.close()
'''
In l3_learning.py, there is a method in class l3_switch named _handle_PacketIn.
Now I understand that this event is when a switch contacts controller when it receives a packet corresponding to which it has no entries in its table.
What I don't understand is that here
packet = event.parsed
Now what does packet.next mean in isinstance(packet.next, ipv4)?
def _handle_PacketIn (self, event):
dpid = event.connection.dpid
inport = event.port
packet = event.parsed
if not packet.parsed:
log.warning("%i %i ignoring unparsed packet", dpid, inport)
return
if dpid not in self.arpTable:
# New switch -- create an empty table
self.arpTable[dpid] = {}
for fake in self.fakeways:
self.arpTable[dpid][IPAddr(fake)] = Entry(of.OFPP_NONE,
dpid_to_mac(dpid))
if packet.type == ethernet.LLDP_TYPE:
# Ignore LLDP packets
return
if isinstance(packet.next, ipv4):
log.debug("%i %i IP %s => %s", dpid,inport,
packet.next.srcip,packet.next.dstip)
# Send any waiting packets...
self._send_lost_buffers(dpid, packet.next.srcip, packet.src, inport)
# Learn or update port/MAC info
if packet.next.srcip in self.arpTable[dpid]:
if self.arpTable[dpid][packet.next.srcip] != (inport, packet.src):
log.info("%i %i RE-learned %s", dpid,inport,packet.next.srcip)
else:
log.debug("%i %i learned %s", dpid,inport,str(packet.next.srcip))
self.arpTable[dpid][packet.next.srcip] = Entry(inport, packet.src)
# Try to forward
dstaddr = packet.next.dstip
if dstaddr in self.arpTable[dpid]:
# We have info about what port to send it out on...
prt = self.arpTable[dpid][dstaddr].port
mac = self.arpTable[dpid][dstaddr].mac
I think I have figured it out.
packet is entire packet that data-link layer sends to the physical layer. packet.next removes the encapsulation of data-link layer and reveals the IP packet (the packet sent by IP layer to data-link layer). So to get the source MAC address we use packet.src and to get the IP address of the source we use packet.next.srcip
I am trying to measure the responses back from DNS servers. Making a sniffer for a typical DNS response that is less than 512 bytes is no big deal. My issue is receiving large 3000+ byte responses - in some cases 5000+ bytes. I haven't been able to get a socket working that can receive that data reliably. Is there a way with Python sockets to receive from a specific source address?
Here is what I have so far:
import socket
import struct
def craft_dns(Qdns):
iden = struct.pack('!H', randint(0, 65535))
QR_thru_RD = chr(int('00000001', 2)) # '\x01'
RA_thru_RCode = chr(int('00100000', 2)) # '\x00'
Qcount = '\x00\x01' # question count is 1
ANcount = '\x00\x00'
NScount = '\x00\x00'
ARcount = '\x00\x01' # additional resource count is 1
pad = '\x00' #
Rtype_ANY = '\x00\xff' # Request ANY record
PROtype = '\x00\x01' # Protocol IN || '\x00\xff' # Protocol ANY
DNSsec_do = chr(int('10000000', 2)) # flips DNSsec bit to enable
edns0 = '\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x00' # DNSsec disabled
domain = Qdns.split('.')
quest = ''
for x in domain:
quest += struct.pack('!B', len(x)) + x
packet = (iden+QR_thru_RD+RA_thru_RCode+Qcount+ANcount+NScount+ARcount+
quest+pad+Rtype_ANY+PROtype+edns0) # remove pad if asking <root>
return packet
def craft_ip(target, resolv):
ip_ver_len = int('01000101', 2) # IPvers: 4, 0100 | IP_hdr len: 5, 0101 = 69
ipvers = 4
ip_tos = 0
ip_len = 0 # socket will put in the right length
iden = randint(0, 65535)
ip_frag = 0 # off
ttl = 255
ip_proto = socket.IPPROTO_UDP # dns, brah
chksm = 0 # socket will do the checksum
s_addr = socket.inet_aton(target)
d_addr = socket.inet_aton(resolv)
ip_hdr = struct.pack('!BBHHHBBH4s4s', ip_ver_len, ip_tos, ip_len, iden,
ip_frag, ttl, ip_proto, chksm, s_addr, d_addr)
return ip_hdr
def craft_udp(sport, dest_port, packet):
#sport = randint(0, 65535) # not recommended to do a random port generation
udp_len = 8 + len(packet) # calculate length of UDP frame in bytes.
chksm = 0 # socket fills in
udp_hdr = struct.pack('!HHHH', sport, dest_port, udp_len, chksm)
return udp_hdr
def get_len(resolv, domain):
target = "10.0.0.3"
d_port = 53
s_port = 5353
ip_hdr = craft_ip(target, resolv)
dns_payload = craft_dns(domain) # '\x00' for root
udp_hdr = craft_udp(s_port, d_port, dns_payload)
packet = ip_hdr + udp_hdr + dns_payload
buf = bytearray("-" * 60000)
recvSock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
recvSock.settimeout(1)
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
sendSock.settimeout(1)
sendSock.connect((resolv, d_port))
sendSock.send(packet)
msglen = 0
while True:
try:
pkt = recvSock.recvfrom(65535)
msglen += len(pkt[0])
print repr(pkt[0])
except socket.timeout as e:
break
sendSock.close()
recvSock.close()
return msglen
result = get_len('75.75.75.75', 'isc.org')
print result
For some reason doing
pkt = sendSock.recvfrom(65535)
Recieves nothing at all. Since I'm using SOCK_RAW the above code is less than ideal, but it works - sort of. If the socket is extremely noisy (like on a WLAN), I could end up receiving well beyond the DNS packets, because I have no way to know when to stop receiving packets when receiving a multipacket DNS answer. For a quiet network, like a lab VM, it works.
Is there a better way to use a receiving socket in this case?
Obviously from the code, I'm not that strong with Python sockets.
I have to send with SOCK_RAW because I am constructing the packet in a raw format. If I use SOCK_DGRAM the custom packet will be malformed when sending to a DNS resolver.
The only way I could see is to use the raw sockets receiver (recvSock.recv or recvfrom) and unpack each packet, look if the source and dest address match within what is supplied in get_len(), then look to see if the fragment bit is flipped. Then record the byte length of each packet with len(). I'd rather not do that. It just seems there is a better way.
Ok I was stupid and didn't look at the protocol for the receiving socket. Socket gets kind of flaky when you try to receive packets on a IPPROTO_RAW protocol, so we do need two sockets. By changing to IPPROTO_UDP and then binding it, the socket was able to follow the complete DNS response over multiple requests. I got rid of the try/catch and the while loop, as it was no longer necessary and I'm able to pull the response length with this block:
recvSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_UDP)
recvSock.settimeout(.3)
recvSock.bind((target, s_port))
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
#sendSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sendSock.settimeout(.3)
sendSock.bind((target, s_port))
sendSock.connect((resolv, d_port))
sendSock.send(packet)
pkt = recvSock.recvfrom(65535)
msglen = len(pkt[0])
Now the method will return the exact bytes received from a DNS query. I'll leave this up in case anyone else needs to do something similar :)
At sender side I have the following code using processing language (portion code):
udp = new UDP( this, 6002 ); // create a new datagram connection on port 6000
//udp.log( true ); // <-- printout the connection activity
udp.listen( true ); // and wait for incoming message
escribeUDPLog3(1,TRANSMIT,getTime()); //call function
int[] getTime(){
int year = year();
int month = month()
int day = day();
int hour = hour();
int minute = minute();
int second = second();
int[] time_constructed = {year, month,day,hour,minute,second};
return time_constructed;
}
void escribeUDPLog3(int pkg_type, int state, int[] time){
short year = (short)(time[0]); //>> 8;
byte year_msb = byte(year >> 8);
byte year_lsb = byte(year & 0x00FF);
byte month = byte(time[1]);
byte day = byte(time[2]);
byte hour = byte(time[3]);
byte minute = byte(time[4]);
byte second = byte(time[5]);
byte[] payload = {byte(pkg_type), byte(state), year_msb, year_lsb, month, day, hour, minute,second};
try {
if (UDP5_flag) {udp.send(payload, UDP5_IP, UDP5_PORT);}
}
catch (Exception e) {
e.printStackTrace();
}
}
At receiver side I'm using SocketServer python structure to set up a server listening for udp datagrams, as following.
from datetime import datetime
import csv
import SocketServer
def nodeStateCheckout(nodeid, state, nodeState):
if (state == ord(nodeState)):
return "OK"
else:
return "FAIL"
def timeConstructor(time):
year = str(ord(time[0]) << 8 | ord(time[1]))
month = str(ord(time[2]))
day = str(ord(time[3]))
hour = str(ord(time[4]))
minute = str(ord(time[5]))
second = str(ord(time[6]))
time_formatted = year + "-" + month + "-" + day \
+ " " + hour + ":" + minute + ":" + second
return time_formatted
class MyUDPHandler(SocketServer.BaseRequestHandler):
"""
This class works similar to the TCP handler class, except that
self.request consists of a pair of data and client socket, and since
there is no connection the client address must be given explicitly
when sending data back via sendto().
"""
def handle(self):
try:
data = self.request[0].strip()
socket = self.request[1]
#print "{} wrote:".format(self.client_address[0])
pkg_type = ord(data[0])
if pkg_type == 1: # log 3
state = ord(data[1])
csvfile = open("log3.csv", "a+")
csvwriter = csv.writer(csvfile, delimiter=',')
time_reconstructed = timeConstructor(data[2:9])
if state == 3:
csvwriter.writerow(["STOP",time_reconstructed])
elif state == 2:
csvwriter.writerow(["START",time_reconstructed])
else:
print "unknown state"
csvfile.close()
else:
print "packet not known"
except IndexError:
print "Bad parsed byte"
if __name__ == "__main__":
HOST, PORT = "localhost", 8892
server = SocketServer.UDPServer((HOST, PORT), MyUDPHandler)
server.serve_forever()
Edited:
I have problem specifically when using timeConstructor(data[2:9]), because I'm accessing out of index data, sometimes (with the help of print) I can't received second byte from data, and one time it get me out of index because I didn't received minute and second. Most of the time the code works well, but this type of error get me curious.
Old:
The problem is when reading the payload, sometimes its seems that some bytes doesn't arrive, even when I captured the whole payload using Wireshark (but Wireshark didn't tell me if this is the sent packet or received packet because I'm using loopback interfaces, maybe duplicated info?). If the datagram has 16 bytes payload long, sometimes I received 15 because when parsing from data I get out of index error.
I think that there are some buffer problems. Isn't it? How to configured it properly? I know that I can get packet loss because of connectionless protocol but I dont think that bytes get lost. It is supposed that "data" has all payload data from one udp datagram.
I believe your problem is that socket.sendto() does not always send all the bytes that you give it. It returns the number of bytes sent and you may have to call it again. You might be better off with opening the socket yourself and calling socket.sendall()
I have a Digi zigbee device which is configured (Python code) to send data serially through a COM port. On the other end of the serial communication there is an embedded board which receives data.
After I connect digi (sending data) to the embedded board, the digi after sending few data reboots (COM PORT closes). But the embedded board remains alive throughout the whole period.
This embedded board has a software through which I can see its logs. When I checked the logs, I received some data (three sensor values) and the digi device dies. I don't know where the problem is occuring. Is it with the digi-zigbee device which is sending data or PYTHON CODE(used in digi device) or the embedded board which is receiving the data?
Here is just a part of python source code :
open()
flushInput()
flushOutput()
while True:
# Retrieve a sample reading from the LT
io_sample = xbee.ddo_get_param(sensor_address, "is")
light = parseIS(io_sample)["AI1"]
temp = parseIS(io_sample)["AI2"]
hum = parseIS(io_sample)["AI3"]
mVanalog = (float(temp) / 1023.0) * 1200.0
temp_C = (mVanalog - 500.0)/ 10.0 # - 4.0
lux = (float(light) / 1023.0) * 1200.0
print "hum:%f" %hum
sync=254
temp=round(temp_C,2)
light=round(lux,2)
no_temp = len(str(temp))
no_light = len(str(light))
total_length=no_temp+no_light+3+3
if total_length<256:
low_byte=total_length
high_byte=0
else:
low_byte=total_length%256
high_byte=high_byte/256
msg_id_low=0
msg_id_high=0
checksum=low_byte+high_byte+msg_id_low+msg_id_high+ord('#')+ord(':')+ord(',')
t=str(temp)
for c in t:
if c == '.':
checksum+=ord('.')
else:
checksum+=ord(c)
t=str(light)
for c in t:
if c == '.':
checksum+=ord('.')
else:
checksum+=ord(c)
checksum=256-(checksum%256)
if checksum == 256:
checksum=0
print "Checksum Value after applying mod operator:%d" %checksum
packet=str(chr(254))+str(chr(low_byte))+str(chr(high_byte))+str(chr(msg_id_low))+str(chr(msg_id_high))+str('#')+str(temp)+str(':')+str(light)+str(',')+str(chr(checksum))
print "packet:%s" %packet
bytes_written = write(packet)
print "bytes_written : %s value : %s" %(bytes_written,packet)
time.sleep(5)
This Code gets the temperature and light value from sensors and converts it into a packet(sync,checksum etc) to be send to the embedded board .