I would like to load-test a SignalR service using Locust. I found that the following library can send and receive SignalR requests: https://pypi.org/project/signalrcore/
Now, according to the Locust docs, the next step would be to write a custom client for Locust that can send SignalR requests. But there is the following warning:
Any protocol libraries that you use must be gevent-friendly (use the
Python socket module or some other standard library function like
subprocess), or your calls are likely to block the whole Locust/Python
process.
Some C libraries cannot be monkey patched by gevent, but allow for
other workarounds. For example, if you want to use psycopg2 to
performance test PostgreSQL, you can use psycogreen
I am a beginner in Python so I don't understand exactly what it means. The library "signalrcore" I am using is 100% synchronous. Does it means I can't use it with Locust?
I found an a fork of signalrcore that uses asyncio. Should I use that fork instead and just make sure all my signalr calls are non blocking?
Thanks!
SignalRCore seems to use requests and websocket-client under the hood, both of which are gevent-friendly. I cant say for sure, but I’d give it 90% probability that it will work ”out of the box” :)
If you do use the asyncio one you’d need to do some magic yourself. At least I have never combined that with gevent.
Related
I have a running CLI application in Python that uses threads to execute some workers. Now I am writing a GUI using electron for this application. For simple requests/responses I am using gRPC to communicate between the Python application and the GUI.
I am, however, struggling to find a proper publishing mechanism to push data to the GUI: gRPCs integrated streaming won't work since it uses generators; as already mentioned my longer, blocking tasks are executed using threads (subclasses of threading.Thread). Also I'd like to emit certain events (e.g., the progress) from within those threads.
Then I've found the Flasks SocketIO implementation, which is, however, a blocking execution and thus not really suited for what I have in mind - I'd have to again execute two processes (Flask and my CLI application)...
Another package I've found is websockets but I can't get my head around how I could implement this producer() function that they mention in the patterns.
My last idea would be to deploy a broker-based message system like Redis or simply fall back to the brokerless zmq, which is a bit of a hassle to setup for the GUI application.
So the simple question:
Is there any easy framework that allows to create a server-"task" in a Python that I can pass messages to publish to?
For anyone struggling with concurrency in python:
No, there isn't any simple framework. IMHO pythons' concurrency handling is a bit of a mess (compared to other languages like golang, where concurrency is built in). There's multiple major packages implementing this, one of them asyncio, but most of them are incompatible. I've ended up using a similar solution like proposed in this question.
I want to write a daemon in python which gets started via systemd.
I wan to use Type=notify, this way I don't have to do the double fork magic.
According to the docs:
The reference implementation for this notification is provided by libsystemd-daemon.so
... how to do this with Python?
Probably, you could use sdnotify python module which is a pure-python implementation of sd_notify protocol. Actually, the protocol is rather simple, so the module implementation is quite short.
To use watchdog machinery you should add WatchdocSec=<smth> to the unit file, and then send WATCHDOG=1 messages on a regular basis from your service. Check Restart= option as well.
use the package
https://pypi.org/project/systemd-python/
it is the offical systemd devs and maintained.
I am writing an application using pycurl, and need to make it work in Twisted. I've been searching for either making pycurl somehow compatible with Twisted framework, or using an existing Twisted library. I am suggested Twisted web, but there is no direct map of functions from pycurl to Twisted web. Can anyone point me to the right direction?
Edit: One solution is run pycurl in another thread, but preferably, I wish to use Twisted framework or pycurl that is nonblocking, so I don't have to create another thread.
For any blocking function, if there's no other asynchronous alternative, Twisted lets you run it on another thread but treat it as a Deferred.
from twisted.internet import threads
d = threads.deferToThread(pycurl.some_function)
d.addCallback(callback)
See Integrating blocking code with Twisted in Generatin Deferreds.
Have you considered using Twisted web.client.Agent? It is very basic agent, but integrates very well with the Twisted event loop.
I am using buildbot version 0.8.5 and need to send an HTTP post request from it as a step. After searching for it on internet, I found that the latest version 0.8.8 has a step called HTTPStep for doing so. Is there any similar step in the older version?
I know it can be done using batch file or python program using urllib2. but is there any other way to do it?
You should be able to use the HTTPStep from 0.8.8 (provided you install the necessary dependency (https://pypi.python.org/pypi/txrequests and http://python-requests.org/)). Just copy the http.py file from 0.8.8 next to your master.cfg, and have your master.cfg import the HTTPStep derived class POST from module http instead of buildbot.steps.http.
Some small adjustments might be needed to make it work with the API of 0.8.5 though.
Just my thoughts..As far as I know it is better to use a python script from a build step. Simple and easy to control. The logic being:
the entire buildbot is inside one http connection/session and sending another http request somewhere might have issues with the connection/session.
from the buildbot httpstep description, you need to install additional python packages which might be not be so convenient to do on multiple slaves/masters.
Is there any way to make DNS resolving using Python with asyncore?
I can not install adns and I do not like to use gevent library.
(for URL downloading gevent give me slow performance than syncore.loop)
I asked google your question, and it told me about http://subvert-rpki.hactrn.net/trunk/rpkid/rpki/adns.py which seems to be a DNS client implementation based on asyncore. So, there you have it, there is a way.