I have a pyzmq Publisher which sends around 1000 messages per second. I am trying to start around 10 Subscribers in an asyncio event_loop.
It works but around 2.5 times slower than speed of the only one Subscriber.
What could possibly be wrong with the code?
import asyncio
import zmq
import json
from zmq.backend.cython.constants import NOBLOCK
from zmq.asyncio import Context, Poller
from loop_ import Loop
class Client:
REQUEST_TIMEOUT = 35000
SERVER_ENDPOINT = "tcp://localhost:6666"
def __init__(self, id_):
self.id = id_
def get_task(self):
return asyncio.create_task(self.client_coroutine())
async def client_coroutine(self):
context = Context.instance()
socket = context.socket(zmq.SUB)
socket.connect(self.SERVER_ENDPOINT)
socket.setsockopt(zmq.SUBSCRIBE, b'4')
poller = Poller()
poller.register(socket, zmq.POLLIN)
while True:
event = dict(await poller.poll(self.REQUEST_TIMEOUT))
if event.get(socket) == zmq.POLLIN:
reply = await socket.recv_multipart(flags=NOBLOCK)
if not reply:
break
else:
print(eval(json.loads(reply[1].decode('utf-8'))))
else:
print("No response from server, retrying...")
socket.setsockopt(zmq.LINGER, 0)
socket.close()
poller.unregister(socket)
async def tasks():
_tasks = [Client(id_).get_task() for id_ in range(10)]
done, pending = await asyncio.wait(_tasks, return_when=asyncio.FIRST_EXCEPTION)
loop = asyncio.get_event_loop()
loop.run_until_complete(tasks())
Q : What could possibly be wrong with the code?
Given the code is using the same localhost ( as seen from using the address ), the suspect number one is, that having 10x more work to process, the such workload will always stress the localhost's O/S and the CPU, won't it?
Next comes the choice of the transport-class. Given all the SUB-s are co-located on the same localhost as the PUB, there is all the L3-stack-based TCP/IP protocol work going wasted. To compare the relative costs ( the add-on effect of using the tcp:// transport-class for this hardware-singular messaging ), test the very same with using inproc:// transport-class, where none of the protocol-related TCP/IP-stack add-on processing will take place.
Last, but not least, my code will never mix different event-loops ( using ZeroMQ since v2.11, so someone may consider my a bit old-fashioned in avoiding relying on async-decorated capabilities available in recent py3.6+ )
My code will use an explicit, non-blocking, zero-waiting test for a presence of a message per-aSocketINSTANCE, as in aSocketINSTANCE.poll( zmq.POLLIN, 0 ) rather than using any "externally" added decoration, which may report the same, but via some additional (expensive and outside of my code domain of control) event-handling. All real-time, low-latency use-cases strive to bear as minimum latency/overheads as possible, so using explicit control will always win in my Projects, to any "modern" syntax-sugar sweetened tricks.
Anyway, enjoy the Zen-of-Zero
Related
Lets say I have a Python gRPC server and a corresponding client.
According to this Question the same gRPC channel can be utilized to be passed to client stubs, each running in different threads.
Lets say RPC function foo() is called from thread T1 and the response takes about one second. Can I call foo() from thread T2 in the meantime, while T1 is still waiting for the response in thread or is the channel somehow locked until the first call returns? In other words: Is performance gained by using more threads, because the corresponding server is based on a thread pool and able to handle more requests "in parallel" and, if yes, should I use the same channel or different channels per thread?
EDIT: According to a quick test it seems that parallel requests using the same channel from different threads are possible and it gives sense, to do it in such way. However, because before closing the question, I would like to see a confirmation from an expert whether this code is correct:
import time
import threading
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
def run_in_thread1(channel):
n = 10000
for i in range(n):
stub = helloworld_pb2_grpc.GreeterStub(channel)
response = stub.SayHello(helloworld_pb2.HelloRequest(name='1'))
print("client 1: " + response.message)
def run_in_thread2(channel):
n = 10000
for i in range(n):
stub = helloworld_pb2_grpc.GreeterStub(channel)
response = stub.SayHello2(helloworld_pb2.HelloRequest(name='2'))
print("client 2: " + response.message)
if __name__ == '__main__':
print("I'm client")
channel = grpc.insecure_channel('localhost:50051')
x1 = threading.Thread(target=run_in_thread1, args=(channel,))
x1.start()
x2 = threading.Thread(target=run_in_thread2, args=(channel,))
x2.start()
x1.join()
x2.join()
gRPC uses HTTP/2 and can multiplex many requests on one connection and gRPC client connections should be re-used for the lifetime of the client app.
If you are inspired by what is done when working with databases, I would say you don't need to worry about it as the opening connection overhead doesn't exist when working with gRPC.
I am trying to call a subprocess inside a subprocess in order to send information using ZMQ to an Unity Application. When I call socket.recv() or time.sleep, it stucks the parent process(which is a child process of a main process)
import json
import zmq
from multiprocessing import Process
import multiprocessing as mp
from absl import app, flags, logging
from absl.flags import FLAGS
def send_unity_data(arg):
context = zmq.Context()
socket = context.socket(zmq.ROUTER)
socket.bind("tcp://*:8080")
while True:
if(arg.poll()):
message=arg.recv()
x = { "x":str(message[0]), "y":str(message[1])}
app_json = json.dumps(x)
socket.send_string(app_json)
message = socket.recv()
print("Received request: %s" % message)
def streaming(detection,args):
try:
vid = cv2.VideoCapture(int(FLAGS.video))
except:
vid = cv2.VideoCapture(FLAGS.video)
receiver1 , sender1 = mp.Pipe()
b_proc3 = Process(target=send_unity_data, args=[receiver1])
b_proc3.start()
while(True):
...
def Main(_argv):
receiver , sender = mp.Pipe()
b_proc = Process(target=streaming, args=[receiver,FLAGS])
b_proc.start()
while(True):
...
I want to send positional coordinates to an Unity application, which is calculated by the streaming process, if someone has a better way to do it, I can change my code as well.
Avoid any undeterministically long blocking state in Video streaming
Without deeper analysis, your code uses blocking-mode operations, which will block, whenever there are no messages yet in the Context()-s instance receiving queue and the code submits a call to a socket.recv()-method, like in the message = socket.recv() SLOC above.
Designing multi-layer / multi-process coordination is to avoid each and every potential blocking - ZeroMQ has .poll()-methods for non-blocking or deterministic ( max-latency-budget consolidated MUX-ed ) priority polling (mainloop-alike)-"Controller"-policies.
Feel free to read more details about how to best use ZeroMQ Hierarchy for your projects.
Where the code blocks? Let's review the as-is state:
multiprocessing module has other defaults and exhibits other behaviour than the tools based on the ZeroMQ messaging/signaling infrastructure. Best use the ZeroMQ on both sides - no need to rely on 2nd layer of multiprocessing.Pipe tools for delivering content into ZeroMQ operating realms. ZeroMQ stack-less transport classes, as inproc:// and ipc:// and even cluster-wide tipc:// deliver way better performance as they may ejnoy a Zero-Copy for ultimate shaving off the processing latency.
Anyway, avoid all blocking forms of methods being called and design a code so that it does not depend on (not yet, the more on not ever) delivered messages:
def send_unity_data( arg ): ### an ad-hoc called
context = zmq.Context() ### 1st: spends time to instantiate a .Context()
socket = context.socket( zmq.ROUTER ) ### 2nd: spends time to instantiate a .socket()
socket.bind("tcp://*:8080") ### 3rd: spends time to ask/acquire O/S port(s)
while True: ### INFINITE-LOOP----------------------------------------
if( arg.poll() ### ?-?-?-?-?-?-? MAY BLOCK depending on arg's .poll()-method
): ### IF .poll()-ed:
message = arg.recv() ### ?-?-?-?-?-?-? MAY BLOCK till a call to arg.recv()-finished
x = { "x": str( message[0] ), ### try to assemble a dict{}
"y": str( message[1] ) ### based on both message structure
} ### and content
app_json = json.dumps( x ) ### try to JSON-ify the dict{}
socket.send_string( app_json ) ### x-x-x-x-x-x-x WILL BLOCK till a socket.send_string() finished
message = socket.recv() ### x-x-x-x-x-x-x WILL BLOCK till a socket.recv()is ever finished
print( "Received request: %s" ### try to print
% message ### a representation of a <message>
) ### on CLI
########################################### This is
# SPIN/LOOP OTHEWRISE ### hasty to O/S resources, wasting GIL-lock latency masking
########################################################################################################
I need a webserver which routes the incoming requests to back-end workers by batching them every 0.5 second or when it has 50 http requests whichever happens earlier. What will be a good way to implement it in python/tornado or any other language?
What I am thinking is to publish the incoming requests to a rabbitMQ queue and then somehow batch them together before sending to the back-end servers. What I can't figure out is how to pick multiple requests from the rabbitMq queue. Could someone point me to right direction or suggest some alternate apporach?
I would suggest using a simple python micro web framework such as bottle. Then you would send the requests to a background process via a queue (thus allowing the connection to end).
The background process would then have a continuous loop that would check your conditions (time and number), and do the job once the condition is met.
Edit:
Here is an example webserver that batches the items before sending them to any queuing system you want to use (RabbitMQ always seemed overcomplicated to me with Python. I have used Celery and other simpler queuing systems before). That way the backend simply grabs a single 'item' from the queue, that will contain all required 50 requests.
import bottle
import threading
import Queue
app = bottle.Bottle()
app.queue = Queue.Queue()
def send_to_rabbitMQ(items):
"""Custom code to send to rabbitMQ system"""
print("50 items gathered, sending to rabbitMQ")
def batcher(queue):
"""Background thread that gathers incoming requests"""
while True:
batcher_loop(queue)
def batcher_loop(queue):
"""Loop that will run until it gathers 50 items,
then will call then function 'send_to_rabbitMQ'"""
count = 0
items = []
while count < 50:
try:
next_item = queue.get(timeout=.5)
except Queue.Empty:
pass
else:
items.append(next_item)
count += 1
send_to_rabbitMQ(items)
#app.route("/add_request", method=["PUT", "POST"])
def add_request():
"""Simple bottle request that grabs JSON and puts it in the queue"""
request = bottle.request.json['request']
app.queue.put(request)
if __name__ == '__main__':
t = threading.Thread(target=batcher, args=(app.queue, ))
t.daemon = True # Make sure the background thread quits when the program ends
t.start()
bottle.run(app)
Code used to test it:
import requests
import json
for i in range(101):
req = requests.post("http://localhost:8080/add_request",
data=json.dumps({"request": 1}),
headers={"Content-type": "application/json"})
Hello I am working on develop a rpc server based on twisted to serve several microcontrollers which make rpc call to twisted jsonrpc server. But the application also required that server send information to each micro at any time, so the question is how could be a good practice to prevent that the response from a remote jsonrpc call from a micro be confused with a server jsonrpc request which is made for a user.
The consequence that I am having now is that micros are receiving bad information, because they dont know if netstring/json string that is comming from socket is their response from a previous requirement or is a new request from server.
Here is my code:
from twisted.internet import reactor
from txjsonrpc.netstring import jsonrpc
import weakref
creds = {'user1':'pass1','user2':'pass2','user3':'pass3'}
class arduinoRPC(jsonrpc.JSONRPC):
def connectionMade(self):
pass
def jsonrpc_identify(self,username,password,mac):
""" Each client must be authenticated just after to be connected calling this rpc """
if creds.has_key(username):
if creds[username] == password:
authenticated = True
else:
authenticated = False
else:
authenticated = False
if authenticated:
self.factory.clients.append(self)
self.factory.references[mac] = weakref.ref(self)
return {'results':'Authenticated as %s'%username,'error':None}
else:
self.transport.loseConnection()
def jsonrpc_sync_acq(self,data,f):
"""Save into django table data acquired from sensors and send ack to gateway"""
if not (self in self.factory.clients):
self.transport.loseConnection()
print f
return {'results':'synced %s records'%len(data),'error':'null'}
def connectionLost(self, reason):
""" mac address is searched and all reference to self.factory.clientes are erased """
for mac in self.factory.references.keys():
if self.factory.references[mac]() == self:
print 'Connection closed - Mac address: %s'%mac
del self.factory.references[mac]
self.factory.clients.remove(self)
class rpcfactory(jsonrpc.RPCFactory):
protocol = arduinoRPC
def __init__(self, maxLength=1024):
self.maxLength = maxLength
self.subHandlers = {}
self.clients = []
self.references = {}
""" Asynchronous remote calling to micros, simulating random calling from server """
import threading,time,random,netstring,json
class asyncGatewayCalls(threading.Thread):
def __init__(self,rpcfactory):
threading.Thread.__init__(self)
self.rpcfactory = rpcfactory
"""identifiers of each micro/client connected"""
self.remoteMacList = ['12:23:23:23:23:23:23','167:67:67:67:67:67:67','90:90:90:90:90:90:90']
def run(self):
while True:
time.sleep(10)
while True:
""" call to any of three potential micros connected """
mac = self.remoteMacList[random.randrange(0,len(self.remoteMacList))]
if self.rpcfactory.references.has_key(mac):
print 'Calling %s'%mac
proto = self.rpcfactory.references[mac]()
""" requesting echo from selected micro"""
dataToSend = netstring.encode(json.dumps({'method':'echo_from_micro','params':['plop']}))
proto.transport.write(dataToSend)
break
factory = rpcfactory(arduinoRPC)
"""start thread caller"""
r=asyncGatewayCalls(factory)
r.start()
reactor.listenTCP(7080, factory)
print "Micros remote RPC server started"
reactor.run()
You need to add a enough information to each message so that the recipient can determine how to interpret it. Your requirements sounds very similar to those of AMP, so you could either use AMP instead or use the same structure as AMP to identify your messages. Specifically:
In requests, put a particular key - for example, AMP uses "_ask" to identify requests. It also gives these a unique value, which further identifies that request for the lifetime of the connection.
In responses, put a different key - for example, AMP uses "_answer" for this. The value matches up with the value from the "_ask" key in the request the response is for.
Using an approach like this, you just have to look to see whether there is an "_ask" key or an "_answer" key to determine if you've received a new request or a response to a previous request.
On a separate topic, your asyncGatewayCalls class shouldn't be thread-based. There's no apparent reason for it to use threads, and by doing so it is also misusing Twisted APIs in a way which will lead to undefined behavior. Most Twisted APIs can only be used in the thread in which you called reactor.run. The only exception is reactor.callFromThread, which you can use to send a message to the reactor thread from any other thread. asyncGatewayCalls tries to write to a transport, though, which will lead to buffer corruption or arbitrary delays in the data being sent, or perhaps worse things. Instead, you can write asyncGatewayCalls like this:
from twisted.internet.task import LoopingCall
class asyncGatewayCalls(object):
def __init__(self, rpcfactory):
self.rpcfactory = rpcfactory
self.remoteMacList = [...]
def run():
self._call = LoopingCall(self._pokeMicro)
return self._call.start(10)
def _pokeMicro(self):
while True:
mac = self.remoteMacList[...]
if mac in self.rpcfactory.references:
proto = ...
dataToSend = ...
proto.transport.write(dataToSend)
break
factory = ...
r = asyncGatewayCalls(factory)
r.run()
reactor.listenTCP(7080, factory)
reactor.run()
This gives you a single-threaded solution which should have the same behavior as you intended for the original asyncGatewayCalls class. Instead of sleeping in a loop in a thread in order to schedule the calls, though, it uses the reactor's scheduling APIs (via the higher-level LoopingCall class, which schedules things to be called repeatedly) to make sure _pokeMicro gets called every ten seconds.
I'm trying to write a dead-simple interface for an IRC library, like so:
import simpleirc
connection = simpleirc.Connect('irc.freenode.net', 6667)
channel = connection.join('foo')
find_command = re.compile(r'google ([a-z]+)').findall
for msg in channel:
for t in find_command(msg):
channel.say("http://google.com/search?q=%s" % t)
Working from their example, I'm running into trouble (code is a bit lengthy, so I pasted it here). Since the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option. Using exceptions or threads seems like the wrong thing here, is there a simpler (blocking?) way of using twisted that would make this possible?
In general, if you're trying to use Twisted in a "blocking" way, you're going to run into a lot of difficulties, because that's neither the way it's intended to be used, nor the way in which most people use it.
Going with the flow is generally a lot easier, and in this case, that means embracing callbacks. The callback-style solution to your question would look something like this:
import re
from twisted.internet import reactor, protocol
from twisted.words.protocols import irc
find_command = re.compile(r'google ([a-z]+)').findall
class Googler(irc.IRCClient):
def privmsg(self, user, channel, message):
for text in find_command(message):
self.say(channel, "http://google.com/search?q=%s" % (text,))
def connect():
cc = protocol.ClientCreator(reactor, Googler)
return cc.connectTCP(host, port)
def run(proto):
proto.join(channel)
def main():
d = connect()
d.addCallback(run)
reactor.run()
This isn't absolutely required (but I strongly suggest you consider trying it). One alternative is inlineCallbacks:
import re
from twisted.internet import reactor, protocol, defer
from twisted.words.protocols import irc
find_command = re.compile(r'google ([a-z]+)').findall
class Googler(irc.IRCClient):
def privmsg(self, user, channel, message):
for text in find_command(message):
self.say(channel, "http://google.com/search?q=%s" % (text,))
#defer.inlineCallbacks
def run():
cc = protocol.ClientCreator(reactor, Googler)
proto = yield cc.connectTCP(host, port)
proto.join(channel)
def main():
run()
reactor.run()
Notice no more addCallbacks. It's been replaced by yield in a decorated generator function. This could get even closer to what you asked for if you had a version of Googler with a different API (the one above should work with IRCClient from Twisted as it is written - though I didn't test it). It would be entirely possible for Googler.join to return a Channel object of some sort, and for that Channel object to be iterable like this:
#defer.inlineCallbacks
def run():
cc = protocol.ClientCreator(reactor, Googler)
proto = yield cc.connectTCP(host, port)
channel = proto.join(channel)
for msg in channel:
msg = yield msg
for text in find_command(msg):
channel.say("http://google.com/search?q=%s" % (text,))
It's only a matter of implementing this API on top of the ones already present. Of course, the yield expressions are still there, and I don't know how much this will upset you. ;)
It's possible to go still further away from callbacks and make the context switches necessary for asynchronous operation to work completely invisible. This is bad for the same reason it would be bad for sidewalks outside your house to be littered with invisible bear traps. However, it's possible. Using something like corotwine, itself based on a third-party coroutine library for CPython, you can have the implementation of Channel do the context switching itself, rather than requiring the calling application code to do it. The result might look something like:
from corotwine import protocol
def run():
proto = Googler()
transport = protocol.gConnectTCP(host, port)
proto.makeConnection(transport)
channel = proto.join(channel)
for msg in channel:
for text in find_command(msg):
channel.say("http://google.com/search?q=%s" % (text,))
with an implementation of Channel that might look something like:
from corotwine import defer
class Channel(object):
def __init__(self, ircClient, name):
self.ircClient = ircClient
self.name = name
def __iter__(self):
while True:
d = self.ircClient.getNextMessage(self.name)
message = defer.blockOn(d)
yield message
This in turn depends on a new Googler method, getNextMessage, which is a straightforward feature addition based on existing IRCClient callbacks:
from twisted.internet import defer
class Googler(irc.IRCClient):
def connectionMade(self):
irc.IRCClient.connectionMade(self)
self._nextMessages = {}
def getNextMessage(self, channel):
if channel not in self._nextMessages:
self._nextMessages[channel] = defer.DeferredQueue()
return self._nextMessages[channel].get()
def privmsg(self, user, channel, message):
if channel not in self._nextMessages:
self._nextMessages[channel] = defer.DeferredQueue()
self._nextMessages[channel].put(message)
To run this, you create a new greenlet for the run function and switch to it, and then start the reactor.
from greenlet import greenlet
def main():
greenlet(run).switch()
reactor.run()
When run gets to its first asynchronous operation, it switches back to the reactor greenlet (which is the "main" greenlet in this case, but it doesn't really matter) to let the asynchronous operation complete. When it completes, corotwine turns the callback into a greenlet switch back into run. So run is granted the illusion of running straight through, like a "normal" synchronous program. Keep in mind that it is just an illusion, though.
So, it's possible to get as far away from the callback-oriented style that is most commonly used with Twisted as you want. It's not necessarily a good idea, though.