I have the following Simulink model:
Simulink Model with UDP
With this model and a joystick I want to provide inputs to the UDP port which I will read from another application; in my case Blender.
So at Blender I have the following code that works perfectly for 1 provided input from Simulink.
import bpy
import math
import socket
import struct
port = 12009
address = "127.0.0.1"
base = 0
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind((address, port))
cube = bpy.data.objects['Cube']
cube.location = (base, 0, 0)
class ModalTimerOperator(bpy.types.Operator):
"""Operator which runs its self from a timer"""
bl_idname = "wm.modal_timer_operator"
bl_label = "Modal Timer Operator"
_timer = None
def modal(self, context, event):
if event.type in {'RIGHTMOUSE', 'ESC'}:
self.cancel(context)
s.close()
cube.location = (base, 0, 0)
return {'CANCELLED'}
if event.type == 'TIMER':
data, addr = s.recvfrom(1024)
data = struct.unpack('!d', data)
x = data[0]
print(data)
cube.location = (x, 0, 0)
print("X:", x)
return {'PASS_THROUGH'}
def execute(self, context):
wm = context.window_manager
self._timer = wm.event_timer_add(0.001, window = context.window)
wm.modal_handler_add(self)
return {'RUNNING_MODAL'}
def cancel(self, context):
wm = context.window_manager
wm.event_timer_remove(self._timer)
def register():
bpy.utils.register_class(ModalTimerOperator)
def unregister():
bpy.utils.unregister_class(ModalTimerOperator)
if __name__ == "__main__":
register()
# test call
bpy.ops.wm.modal_timer_operator()
When I try to run this, I get the following error:
struct.error: unpack requires a buffer of 8 bytes
How can I unpack an array instead of just a double using struct.unpack?
What changes should I make in the code to achieve this?
I found the answer myself.
There is no need to use matrix concatenation; just simply connect the mux singal of your desired sent signals via UDP. Then in the code just change the line of the struct.unpack to the following (if you have 3 signals):
data = struct.unpack('!ddd', data)
x = data[0]*100
y = data[1]*100
z = data[2]*100
Related
I am writing a simple threaded server that will send a message to all clients. I have an object that is reset after posting the change message, however I am having a hard time figuring out how to reset that object only after all threads have posted the change message.
To add some context to the problem. I am building a multi user Tkinter python app which connects to a remote database to retrieve information and the application needs to know when data changes so that when a user updates data, all other running instances of the app will get the update. From what I understand, MySQL does not support asynchronous application updates. Instead of running a query every 5 seconds on the database to see if there is a change, I am putting this code server side so that it will send a message to a socket on the client that a change has occurred on the database.
The main loop is just a dummy that will simulate a change
Here is my code:
import socket, threading, time, select, os
class dbMonitor:
isDBAltered = False
def postChange(self):
self.isDBAltered = True
def __str__(self):
return str(self.isDBAltered)
class ThreadedServer(object):
def __init__(self, port,dbMon):
self.port = port
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.setblocking(0)
self.sock.bind((socket.gethostname(), self.port))
self.dbMon = dbMon
def listen(self):
self.sock.listen(100)
read_list = [self.sock]
while True:
read,write,error = select.select(read_list,[],[],1)
for s in read:
if s is self.sock:
client, address = self.sock.accept()
client.settimeout(60)
threading.Thread(target = self.listenToClient, args = (client,address)).start()
def listenToClient(self, client, address):
read_list = [client]
size = 1024
while True:
response = b'Ack'
if self.dbMon.isDBAltered:
response = b'CHANGE'
try:
client.send(response)
except:
client.close()
return False
self.dbMon.isDBAltered = False
read,write,error = select.select(read_list,[],[],1)
for s in read:
if s is client:
try:
data = client.recv(size)
print(data)
if data:
client.send(response)
else:
raise error('Client disconnected')
except:
client.close()
return False
def mainLoop():
while True:
time.sleep(15)
print(dbMon)
dbMon.postChange()
dbMon = dbMonitor()
server = ThreadedServer(5005,dbMon)
threading.Thread(target = mainLoop, args=()).start()
threading.Thread(target = server.listen(), args=()).start()
How do I get self.dbMon.isDBAltered = False to execute only after all threads have executed:
response = b'CHANGE'
try:
client.send(response)
You're trying to synchronize something that's asynchronous... This is massively more complicated than it should be. Your dbmon is only storing a boolean flag... why not just asynchronously modify the "database" instead? For example, if the "database" was a thread-safe buffer, you could just append to that buffer or modify that buffer without synchronizing each thread individually, pull the information written to that buffer and write it to the client socket they belong to in another event loop (this is pretty much what asyncore does)
That said, I have some (probably nonworking, but I hope you get the idea) reference modified code for you to go off of if you want to continue pursing this avenue.
Basically, dbmon will keep a mapping of thread ids to [creation time, modified flag]
Our predicate returns true iff all threads created before a certain threshold have ALL set the modified flag. We set the modified flag when we send the response in the data = client.recv(size) portion of your code. And then we wait on that condition in the server send. We keep notifying all waiting threads on each client receive so that when the condition is finally met, our waiting server threads will all unblock and send the subsequent response.
import socket, threading, time, select, os
import collections
class dbMonitor:
def __init__(self):
self.isDBAltered = {}
self.lock = threading.Lock()
def newThread(self, tid):
self.lock.acquire()
# time of creation, boolean whether that thread has sent response
self.isDBAltered[tid] = [time.time(), False]
self.lock.release()
def threadDone(self, tid):
self.lock.acquire()
self.isDBAltered.pop(tid, None)
self.lock.release()
def altered(self, tid):
self.lock.acquire()
self.isDBAltered[tid][1] = True
self.lock.release()
def reset(self, tid):
self.lock.acquire()
self.isDBAltered(tid)[1] = False
self.lock.release()
def __str__(self):
return str(self.isDBAltered)
class ThreadedServer(object):
def __init__(self, port,dbMon):
self.port = port
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.setblocking(0)
self.sock.bind((socket.gethostname(), self.port))
self.dbMon = dbMon
self.lock = threading.lock()
self.cv = threading.Condition()
self.thresh = 2000
def condition_pred(self):
# unblock if threads currently running for longer than self.thresh have all set their flags
return all([timecreate[1] if time.time() - timecreate[0] > self.thresh else True for tid,timecreate in self.dbMon.isDBAltered])
def listen(self):
self.sock.listen(100)
read_list = [self.sock]
while True:
read,write,error = select.select(read_list,[],[],1)
for s in read:
if s is self.sock:
self.lock.acquire()
client, address = self.sock.accept()
client.settimeout(60)
T = threading.Thread(target = self.listenToClient, args = (client,address)).start()
self.dbmon.newThread(T.ident)
self.lock.release()
def listenToClient(self, client, address):
read_list = [client]
size = 1024
while True:
response = b'Ack'
with self.cv:
self.cv.wait_for(self.condition_pred)
self.dbMon.reset(threading.get_ident())
response = b'CHANGE'
try:
client.send(response)
except:
client.close()
self.dbmon.threadDone(threading.get_ident())
return False
read,write,error = select.select(read_list,[],[],1)
for s in read:
if s is client:
with self.cv:
try:
data = client.recv(size)
print(data)
if data:
client.send(response)
self.dbMon.altered(threading.get_ident())
self.cv.notifyAll()
else:
raise error('Client disconnected')
except:
client.close()
self.dbmon.threadDone(threading.get_ident())
return False
I have the following in my run.py script.
import time
NUM_PACKETS = 500
import random
import argparse
import threading
from scapy.all import sniff
from scapy.all import Ether, IP, IPv6, TCP
parser = argparse.ArgumentParser(description='run_test.py')
parser.add_argument('--random-dport',
help='Use a random TCP dest port for each packet',
action="store_true", default=False)
args = parser.parse_args()
class PacketQueue:
def __init__(self):
self.pkts = []
self.lock = threading.Lock()
self.ifaces = set()
def add_iface(self, iface):
self.ifaces.add(iface)
def get(self):
self.lock.acquire()
if not self.pkts:
self.lock.release()
return None, None
pkt = self.pkts.pop(0)
self.lock.release()
return pkt
def add(self, iface, pkt):
if iface not in self.ifaces:
return
self.lock.acquire()
self.pkts.append( (iface, pkt) )
self.lock.release()
queue = PacketQueue()
def pkt_handler(pkt, iface):
if IPv6 in pkt:
return
queue.add(iface, pkt)
class SnifferThread(threading.Thread):
def __init__(self, iface, handler = pkt_handler):
threading.Thread.__init__(self)
self.iface = iface
self.handler = handler
def run(self):
sniff(
iface = self.iface,
prn = lambda x: self.handler(x, self.iface)
)
class PacketDelay:
def __init__(self, bsize, bdelay, imin, imax, num_pkts = 100):
self.bsize = bsize
self.bdelay = bdelay
self.imin = imin
self.imax = imax
self.num_pkts = num_pkts
self.current = 1
def __iter__(self):
return self
def next(self):
if self.num_pkts <= 0:
raise StopIteration
self.num_pkts -= 1
if self.current == self.bsize:
self.current = 1
return random.randint(self.imin, self.imax)
else:
self.current += 1
return self.bdelay
pkt = Ether()/IP(dst='10.0.0.1', ttl=64)/TCP()
port_map = {
1: "veth3",
2: "veth5",
3: "veth7"
}
iface_map = {}
for p, i in port_map.items():
iface_map[i] = p
queue.add_iface("veth3")
queue.add_iface("veth5")
for p, iface in port_map.items():
t = SnifferThread(iface)
t.daemon = True
t.start()
import socket
send_socket = socket.socket(socket.AF_PACKET, socket.SOCK_RAW,
socket.htons(0x03))
send_socket.bind((port_map[3], 0))
delays = PacketDelay(10, 5, 25, 100, NUM_PACKETS)
ports = []
print "Sending", NUM_PACKETS, "packets ..."
for d in delays:
# sendp is too slow...
# sendp(pkt, iface=port_map[3], verbose=0)
if args.random_dport:
pkt["TCP"].dport = random.randint(1025, 65535)
send_socket.send(str(pkt))
time.sleep(d / 1000.)
time.sleep(1)
iface, pkt = queue.get()
while pkt:
ports.append(iface_map[iface])
iface, pkt = queue.get()
print ports
print "DISTRIBUTION..."
for p in port_map:
c = ports.count(p)
print "port {}: {:>3} [ {:>5}% ]".format(p, c, 100. * c / NUM_PACKETS)
I tried running the script using the below command
./run_test.py '--random-dport' 2
It is throwing an unrecognized arguments error with the following message.
ubuntu#workshop:~/p4lang/tutorials/workshop_/mp$ sudo ./run.py 2
WARNING: No route found for IPv6 destination :: (no default route?)
usage: run.py [-h] [--random-dport]
run_test.py: error: unrecognized arguments: 2
ubuntu#workshop:~/p4lang/tutorials/workshop/mp$
What could be the problem here. I assume the rest of the run.py code is fine and the problem is in the above lines. I can add the rest of the code if asked for. I am stuck with this for a long time. Any inputs will help me!
If you want to specify a specific port (e.g. 2)
I'm not sure why you're using action="store_true".
Just do (instead of your add_argument, the rest of the code is the same):
parser.add_argument("--random-dport", type=int,
help='Use a random TCP dest port for each packet',
default=1) #default to port 1
Then you can access the port by doing:
./run_test.py '--random-dport' 2
args.random-dport
>> 2
If you want to choose a random port
If you really want a true/false flag for a random port, as it seems your code is trying to achieve, you can use your original code, but then passing the 2 is pointless, as the presence of the random-dport flag will store true into the variable:
./run_test.py
args.random-dport
>>False
It was not present->false
./run_test.py '--random-dport'
args.random-dport
>>True
It was present->true
./run_test.py '--random-dport' 2
>>error
It had an extra value->error.
The 2 is meaningless here, you are specifying you want a RANDOM dport, which means you can't pick a specific one. The previous section let's you pick a specific port.)
Either way check out the argparse tutorial. It is very helpful for giving you info on what you want to do.
i am trying to receive about 1000 connections to my server but it cannot receive more than 512. What can i do to increase the amount of open connections? I am running windows 8.1
Not: I am very new to this stuff so, thanks for help
Here is my code;
import asyncore
import socket
import uuid
import time
import threading
class statistics(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
while True:
entry = raw_input("")
zaman = int(time.time())
cmd = receivedCmd
print "calculating.."
time.sleep(1)
if entry == 'istatistik':
print str(receivedCmd-cmd) + " command/second"
print "total received commands: " + str(receivedCmd)
entry = ""
class tcpClient:
def __init__(self):
self.clientid = uuid.uuid1(int(time.time()))
self.buffer = ""
self.buffer_size = 0
self.conn_time = time.time()
self.overflow = 0
#print str(self.clientid) + " assingned"
def recv_msg(self, msg):
global receivedCmd
self.buffer = msg
self.buffer_size = len(self.buffer)
receivedCmd = receivedCmd + 1
if self.buffer_size >= 1024:
self.overflow = 1
def __del__(self):
print str(self.clientid) + " has left."
class TCPHandler(asyncore.dispatcher_with_send):
global clist
def handle_read(self):
data = self.recv(1024)
if data:
if clist[self].overflow:
self.send("overflow")
self.handle_close()
else:
self.send(data)
clist[self].recv_msg(data)
def handle_close(self):
del clist[self]
self.close()
def handle_error(self):
del clist[self]
self.close()
class TCPServer(asyncore.dispatcher):
global clist
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.set_reuse_addr()
self.bind((host, port))
self.listen(5)
def handle_accept(self):
self.clist = clist
pair = self.accept()
if pair is None:
pass
else:
sock, addr = pair
#print 'Connection : %s' % repr(addr)
clist[TCPHandler(sock)] = tcpClient()
if __name__ == '__main__':
clist = {}
receivedCmd = 0
server = TCPServer('', 5000)
server2 = TCPServer('',5001)
StaticsThread = statistics()
StaticsThread.start()
asyncore.loop()
Note: I still cannot receive more than 512 connections with the Twisted Framework, i don't know what to do. There have to be thousands of connected clients. Please help.
The asyncore module relies in the select OS function, which only supports a limited number of file descriptors.
As an alternative use a multi-threading server (I won't recommend this) or, better, the Twisted framework which is event-driven (highly recommended!).
Hope this helps!
Since Twisted's default reactor under Windows is also select-based then you should consider using the IOCP reactor instead.
from twisted.internet import iocpreactor
iocpreactor.install()
from twisted.internet import reactor
But also take into account that Twisted prefers Linux systems (where the default reactor is epoll-based) rather than Windows. Maybe switching to Linux is a better choice.
I have the following class (very basic) for communication with 3G modem through AT commands:
import serial
import time
class ATCommands(object):
def __init__(self, port):
self.ser = None
self.port = port
self.open()
def open(self):
self.ser = serial.Serial('/dev/'+ self.port, 115200, timeout=1)
def sendCommand(self,command):
self.ser.write(command.encode())
data = self.ser.readline().strip()
return data
def getIMEI(self):
IMEI = self.sendCommand("AT+CGSN\r")
IMEI = self.sendCommand("AT+CGSN\r")
return IMEI
def getIMEIErro(self):
IMEI = self.sendCommand("AT+CGSN\r")
return IMEI
def getIMEIErro2(self):
self.ser.write("AT+CGSN\r".encode())
data = self.ser.readline().strip()
return data
def __del__(self):
self.ser.close()
if __name__ == '__main__':
print(ATCommands('ttyUSB1').getIMEI()) #OK
print(ATCommands('ttyUSB1').getIMEIErro()) #erro
print(ATCommands('ttyUSB1').getIMEIErro2()) #erro
In the code above I have two strange things:
getIMEI() only works if I execute the statement self.sendCommand("AT+CGSN\r") twice in a row. getIMEIErro() shows that the IMEI is not returned a single command is sent.
If I run the command self.ser.readline() outside the method sendCommand() the code also does not work. getIMEIErro2() shows this error
Anyone know the reason for the errors?
PS: I'm using python 3 e pySerial 2.7
Try this
clear the buffer
put sleeps to wait the baudrate to apply the commands to the modem
always end your commands with \r\n
Something like (based of in my class in https://github.com/vhpanisa/misc pyatapi.py):
def sendCommand(self,command):
from time import sleep
while self.ser.inWaiting() > 0:
self.ser.read()
sleep(0.1)
# Maybe ascii encode not needed, just bytes convert
self.ser.write(bytes(command+"\r\n", encoding='ascii'))
sleep(0.5)
data = []
while self.ser.inWaiting() > 0:
msg = self.ser.readline().strip()
sleep(0.1) # Wait for buffer
msg = msg.replace(b"\r",b"")
msg = msg.replace(b"\n",b"")
if msg != b"":
data.append(str(msg, encoding='ascii'))
return data
Please help me to improve this code:
import base64
import hashlib
import threading
import socket
class WebSocketServer:
def __init__(self, host, port, limit, **kwargs):
"""
Initialize websocket server.
:param host: Host name as IP address or text definition.
:param port: Port number, which server will listen.
:param limit: Limit of connections in queue.
:param kwargs: A dict of key/value pairs. It MAY contains:<br>
<b>onconnect</b> - function, called after client connected.
<b>handshake</b> - string, containing the handshake pattern.
<b>magic</b> - string, containing "magic" key, required for "handshake".
:type host: str
:type port: int
:type limit: int
:type kwargs: dict
"""
self.host = host
self.port = port
self.limit = limit
self.running = False
self.clients = []
self.args = kwargs
def start(self):
"""
Start websocket server.
"""
self.root = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.root.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.root.bind((self.host, self.port))
self.root.listen(self.limit)
self.running = True
while self.running:
client, address = self.root.accept()
if not self.running: break
self.handshake(client)
self.clients.append((client, address))
onconnect = self.args.get("onconnect")
if callable(onconnect): onconnect(self, client, address)
threading.Thread(target=self.loop, args=(client, address)).start()
self.root.close()
def stop(self):
"""
Stop websocket server.
"""
self.running = False
def handshake(self, client):
handshake = 'HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Accept: %s\r\n\r\n'
handshake = self.args.get('handshake', handshake)
magic = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
magic = self.args.get('magic', magic)
header = str(client.recv(1000))
try:
res = header.index("Sec-WebSocket-Key")
except ValueError:
return False
key = header[res + 19: res + 19 + 24]
key += magic
key = hashlib.sha1(key.encode())
key = base64.b64encode(key.digest())
client.send(bytes((handshake % str(key,'utf-8')), 'utf-8'))
return True
def loop(self, client, address):
"""
:type client: socket
"""
while True:
message = ''
m = client.recv(1)
while m != '':
message += m
m = client.recv(1)
fin, text = self.decodeFrame(message)
if not fin:
onmessage = self.args.get('onmessage')
if callable(onmessage): onmessage(self, client, text)
else:
self.clients.remove((client, address))
ondisconnect = self.args.get('ondisconnect')
if callable(ondisconnect): ondisconnect(self, client, address)
client.close()
break
def decodeFrame(self, data):
if (len(data) == 0) or (data is None):
return True, None
fin = not(data[0] & 1)
if fin:
return fin, None
masked = not(data[1] & 1)
plen = data[1] - (128 if masked else 0)
mask_start = 2
if plen == 126:
mask_start = 4
plen = int.from_bytes(data[2:4], byteorder='sys.byteorder')
elif plen == 127:
mask_start = 10
plen = int.from_bytes(data[2:10], byteorder='sys.byteorder')
mask = data[mask_start:mask_start+4]
data = data[mask_start+4:mask_start+4+plen]
decoded = []
i = 0
while i < len(data):
decoded.append(data[i] ^ mask[i%4])
i+=1
text = str(bytearray(decoded), "utf-8")
return fin, text
def sendto(self, client, data, **kwargs):
"""
Send <b>data</b> to <b>client</b>. <b>data</b> can be of type <i>str</i>, <i>bytes</i>, <i>bytearray</i>, <i>int</i>.
:param client: Client socket for data exchange.
:param data: Data, which will be sent to the client via <i>socket</i>.
:type client: socket
:type data: str|bytes|bytearray|int|float
"""
if type(data) == bytes or type(data) == bytearray:
frame = data
elif type(data) == str:
frame = bytes(data, kwargs.get('encoding', 'utf-8'))
elif type(data) == int or type(data) == float:
frame = bytes(str(data), kwargs.get('encoding', 'utf-8'))
else:
return None
framelen = len(frame)
head = bytes([0x81])
if framelen < 126:
head += bytes(int.to_bytes(framelen, 1, 'big'))
elif 126 <= framelen < 0x10000:
head += bytes(126)
head += bytes(int.to_bytes(framelen, 2, 'big'))
else:
head += bytes(127)
head += bytes(int.to_bytes(framelen, 8, 'big'))
client.send(head + frame)
It works fine.
I want the server to use all the processor cores for improved performance. And this code is not effective in high quantities connections. How to implement a multi-threaded solution for this case?
sorry for my bad english.
In CPython, the global interpreter lock, or GIL, is a mutex that
prevents multiple native threads from executing Python bytecodes at
once.
So your code won't work. You can use processeses instead of threads (not on Windows*), twisted or asyncore if you want to support more than one client at the same time.
If your choice is multiprocessing, try this:
client.py:
import socket
def main():
s = socket.socket()
s.connect(("localhost", 5555))
while True:
data = raw_input("> ")
s.send(data)
if data == "quit":
break
s.close()
if __name__ == "__main__":
main()
server.py:
from multiprocessing import Process
from os import getpid
import socket
def receive(conn):
print "(%d) connected." % getpid()
while True:
data = conn.recv(1024)
if data:
if data == "quit":
break
else:
print "(%s) data" % getpid()
def main():
s = socket.socket()
s.bind(("localhost", 5555))
s.listen(1)
while True:
conn, address = s.accept()
print "%s:%d connected." % address
Process(target=receive, args=(conn,)).start()
s.close()
if __name__ == "__main__":
main()
*On Windows this code will throw an error when pickling the socket:
File "C:\Python27\lib\pickle.py", line 880, in load_eof
raise EOFError