What limits the number of SSE(server sent event) connections?
I have been working on a project using django/gunicorn/django-sse.
My project works great when i limit the number of sse connections to the page (5 works 6 hangs), this isnt a huge problem cause i use pagination so can limit the number per page. but i would prefer to be able to have as many as i like.
My question is: is it the number of connections that is slowing it down, or is it the amount of data being transfered?
the first problem i think i could fix by making them share a connection but the second would probably limit me a bit more.
Any ideas which it may be?
EDIT:
client side JS SSE code:
function event(url, resource_name, yes, no, audio_in, audio_out, current_draw){
/**
* Listens for events posted by the server
*
* Useful site for understanding Server Sent Events:
* http://www.w3.org/TR/eventsource/
*/
var source = new EventSource(url);
source.addEventListener("message", function(e) {
resetTime(resource_name);
data = updateStatus(e.data, yes, no, audio_in, audio_out, current_draw);
document.getElementById(resource_name+"-in").src = data.audio_in_src
document.getElementById(resource_name+"-in").alt = data.audio_in_alt
document.getElementById(resource_name+"-out").src = data.audio_out_src
document.getElementById(resource_name+"-out").alt = data.audio_out_alt
document.getElementById(resource_name+"-current").innerHTML = data.current_draw + " A"
});
}
in views.py
class ServerSentEvent(RedisQueueView):
def get_redis_channel(self):
"""
Overrides the RedisQueueView method to select the channel to listen to
"""
return self.kwargs["resource_name"]
in urls.py
urlpatterns = patterns('',
url(r'^$',
views.Resources_page.as_view(),
name='resources_page'),
url(r'^(?P<resource_name>\w+)/$',
views.StatusPage.as_view(),
name='status_page'),
url(r'^(?P<resource_name>\w+)/sse/$',
views.ServerSentEvent.as_view(),
name='sse'),)
If you're using the sync worker for gunicorn (the default), then you can only have as many concurrent connections to your server as you have worker processes.
The sync worker is designed for CPU-bound tasks, hence the recommendation to use 2N + 1 workers (where N is the number of cores available). If your SSE endpoint is the logical equivalent of this...
while True:
msg = "foo"
yield msg
sleep(1)
...then you have an I/O-bound view. No matter how much CPU time you throw at that block of code, it's designed to never end. If you use the django_sse project, then this is almost exactly what your SSE view is doing.
The solution is to use an asynchronous worker class for gunicorn. Install gevent and pass --worker-class=gevent option to gunicorn and you're on your way to an asynchronous utopia.
I had the exact same problem. I was sure I was using gevent as the worker, but I only got around 6 connections.
The solution was stupid. This was a browser limitation. I am writing it down here for the next person to stumble on this to see..
In firefox, there is a parameter in about:config that is named network.http.max-persistent-connections-per-server that controls this. So the solution in my case was increasing that number from 6 (default), or using several browsers..
The browser usually limit the connections to same server, you can check that from the configuration of your browser, e.g. in Firefox, you can check on page "about:config", where you can see such key:value { network.http.speculative-parallel-limit 6 }, of course, you can change the number to more and test.
So this is not the problem on server side.
Related
I have a standard function-based view in Django which receives some parameters via POST after the user has clicked a button, computes something and then returns a template with context.
#csrf_exempt
def myview(request, param1, param2):
if request.method == 'POST':
return HttpResponseRedirect(reverse("app1:view_name", args=[param1, param2]))
'''Calculate and database r/w'''
template = loader.get_template('showData.html')
return HttpResponse(template.render(context, request))
It works with no problem as long as one request is processed at the time (tested both with runserver and in an Apache server).
However, when I use two devices and click on the button simultaneously in each, both requests are mixed up, run simultaneously, and the website ends up trowing a 500 error, or 404 or sometimes success but cannot GET static files.. (again, tested both with runserver and Apache).
How can I force Django to finish the execution of the current request before starting the next?
Or is there a better way to tackle this?
Any light on this will be appreciated. Thanks!
To coordinate threads within a single server process, use
from threading import RLock
lock = RLock()
and then within myview:
lock.acquire()
... # get template, render it
lock.release()
You might start your server with $ uwsgi --processes 1 --threads 2 ...
Django web server on local machine is not for production environment. So it processes one request at a time. In production, you need to use WSGI server, like uwsgi. With that your app can be set up to serve more than one request at a time. Check https://docs.djangoproject.com/en/2.1/howto/deployment/wsgi/uwsgi/
I post my solution in case its of any help to other.
Finally I configured Apache with a pre-forking to isolate requests from each other. According to the documentation the pre-forking is advised for sites using non-thread-safe libraries (my case, apparently).
With this fix Apache can handle well simultaneous requests. However I will still be glad to hear if someone else has other suggestions!
There should be ways to rewrite the code such, that things do not get mixed up. (At least in many cases this is possible)
One of the pre-requirements (if your server uses threading) is to write thread safe code
This means not using global variables (which is bad practice anyway) (or protecting them with Locks)
and using no calls to functions that aren't thread safe. (or protect them with Locks)
As you don't provide any details we cannot help with this. (this = finding a way to not make the whole request blocking, but keep data integrity)
Otherwise you could use a mutex / Lock, that works across multiple processes.
you could for example try to access a locked file
https://pypi.org/project/filelock/ and block until the file is unlocked by the other view.
example code (after pip installing filelock)
from filelock import FileLock
lock = FileLock("my.lock")
with lock:
if request.method == 'POST':
return HttpResponseRedirect(reverse("app1:view_name", args=[param1, param2]))
'''Calculate and database r/w'''
template = loader.get_template('showData.html')
return HttpResponse(template.render(context, request))
If you use uwsgi, then you could look at the uwsgi implementation of locks:
https://uwsgi-docs.readthedocs.io/en/latest/Locks.html
Here the example code from the uwsgi documentation:
def use_lock_zero_for_important_things():
uwsgi.lock() # Implicit parameter 0
# Critical section
uwsgi.unlock() # Implicit parameter 0
def use_another_lock():
uwsgi.lock(1)
time.sleep(1) # Take that, performance! Ha!
uwsgi.unlock(1)
I've built a Flask application, that computes some paths in a graph. Usually, it's a very greedy task and it takes a lot of time to finish calculations. While I was busy with configuring algorithm, I didn't really pay attention to the server side implementations. We've set up an Nginx server, that servers that whole thing. Here's main Flask route:
#app.route('/paths', methods=['POST'])
def paths():
form = SampleForm(request.form)
if form.validate_on_submit():
point_a = form.point_a.data
point_b = form.point_b.data
start = form.start.data.strftime('%Y%m%d')
end = form.end.data.strftime('%Y%m%d')
hops = form.hops.data
rendering_time, collections = make_collection(point_a, point_b, start, end, hops)
return render_template(
'result.html',
searching_time=rendering_time,
collections=collections)
else:
logger.warning('Bad form: {}'.format(form.errors))
return render_template('index.html', form=form)
The whole calculation thing lies under make_collection method. So, whenever user sends request to the server.com/path, he will have to wait, until the method completes calculations and returns something. This is not a pleasing solution, sometimes Nginx just goes timeout.
The next version of this was with a simple idea of delegating labor work to some thread and just returning an empty page to the user. Later on we can just update page contents with the latest searhing results.
#app.route('/paths', methods=['POST'])
def paths():
form = SampleForm(request.form)
if form.validate_on_submit():
point_a = form.point_a.data
point_b= form.point_b.data
start = form.start.data.strftime('%Y%m%d')
end = form.end.data.strftime('%Y%m%d')
hops = form.hops.data
finder = threading.Thread(
target=make_collection,
kwargs={
'point_a': point_a,
'point_b': point_b,
'start': start,
'end': end,
'hops': hops})
finder.start()
rendering_time, collections = 0, []
return render_template(
'result.html',
searching_time=rendering_time
collections=collections)
else:
logger.warning('Bad form: {}'.format(form.errors))
return render_template('index.html', form=form)
The code above works fine and with accepatable searching time(didn't changed from the first version, like expected). The problem is, it works like that only on my local machine. When I deploy this to the Nginx, the total performance is not even nearly close to what I'm expecting. For comparison, results that I find on my local machine under 30 seconds, Nginx cannot fully find even under 300 seconds. What to do?
P.S. Originially, setting up Nginx server wasn't my part of the job and I'm not very familiar how Nginx works, but if you need any info, please, ask.
First code snippet looks like an easy way to let client fetch calculations results.
However, make_collection is a blocking one and Nginx will keep one of its workers busy with it. Since usual way of Nginx configuration is to have one worker per CPU core, that leaves you with one worker less each time you make a HTTP request to /paths. If there are multiple requests to /paths then is not surprise that you get a poor performance. Not to mention WSGI server that you probably have e.g. uwsgi, gunicorn, etc. and their workers and threads per worker process.
Solution with threads might look like a good solution, but you can end up with a lot of threads. Pay attention to threads in Python and try to avoid CPU bound work from being delegated to threads in Python unless you really know what you are doing.
In general you should try to avoid these blocking calls, like the one you make and offload them to a separate worker queue while keeping a reference for getting results later on.
I have a REP socket that's connected to many REQ sockets, each running on a separate Google Compute Engine instance. I'm trying to accomplish the synchronization detailed in the ZMQ Guide's syncpub/syncsub example, and my code looks pretty similar to that example:
context = zmq.Context()
sync_reply = context.socket(zmq.REP)
sync_reply.bind('tcp://*:5555')
# start a bunch of other sockets ...
ready = 0
while ready < len(all_instances):
sync_reply.recv()
sync.reply.send(b'')
ready += 1
And each instance is running the following code:
context = zmq.Context()
sync_request = context.socket(zmq.REQ)
sync_request.connect('tcp://IP_ADDRESS:5555')
sync_request.send(b'')
sync_request.recv()
# start other sockets and do other work ...
This system works fine up until a certain number of instances (around 140). Any more, though, and the REP socket will not receive all of the requests. It also seems like the requests it drops are from different instances each time, which leads me to believe that all the requests are indeed being sent, but the socket is just not receiving any more than (about) 140 of them.
I've tried setting the high water mark for the sockets, spacing out the requests over the span of a few seconds, switching to ROUTER/DEALER sockets - all with no improvement. The part that confuses me the most is that the syncsub/syncpub example code (linked above) works fine for me with up to 200 Google Compute Engine instances, which is as many as I can start. I'm not sure what about my code specifically is causing this problem - any help or tips would be appreciated.
Answering my own question - it seems like it was an issue with the large number of sockets I was using, and also possibly the memory limitations of the GCE instances used. See comment thread above for more details.
I have a python web-app that uses Pyramid/CherryPy for the webserver.
It has a few periodic housekeeping tasks that need to be run - Clearing out stale sessions, freeing their resources, etc...
What is the proper way to manage this? I can fairly easily just run a additional "housekeeping" thread (and use a separate scheduler, like APscheduler), but having a separate thread reach into the running server thread(s) just seems like a really clumsy solution. CherryPy is already running the server in a (multi-threaded) eventloop, it seems like it should be possible to somehow schedule periodic events through that.
I was lead to this answer by #fumanchu's answer, but I wound up using an instance of the cherrypy.process.plugins.BackgroundTask plugin:
def doHousekeeping():
print("Housekeeper!")
-
def runServer():
cherrypy.tree.graft(wsgi_server.app, "/")
# Unsubscribe the default server
cherrypy.server.unsubscribe()
# Instantiate a new server object
server = cherrypy._cpserver.Server()
# Configure the server object
server.socket_host = "0.0.0.0"
server.socket_port = 8080
server.thread_pool = 30
# Subscribe this server
server.subscribe()
cherrypy.engine.housekeeper = cherrypy.process.plugins.BackgroundTask(2, doHousekeeping)
cherrypy.engine.housekeeper.start()
# Start the server engine (Option 1 *and* 2)
cherrypy.engine.start()
cherrypy.engine.block()
Results in doHousekeeping() being called at 2 second intervals within the CherryPy event loop.
It also doesn't involve doing something as silly as dragging in the entire OS just to call a task periodically.
Have a look at the "main" channel at http://cherrypy.readthedocs.org/en/latest/progguide/extending/customplugins.html
Do yourself a favour and just use cron. No need to roll your own scheduling software.
I have a bunch of servers with multiple instances accessing a resource that has a hard limit on requests per second.
I need a mechanism to lock the access on this resource for all servers and instances that are running.
There is a restful distributed lock manager I found on github: https://github.com/thefab/restful-distributed-lock-manager
Unfortunately there seems to be a min. lock time of 1 second and it's relatively unreliable. In several tests it took between 1 and 3 seconds to unlock a 1 second lock.
Is there something well tested with a python interface I can use for this purpose?
Edit: I need something that auto unlocks in under 1 second. The lock will never be released in my code.
My first idea was using Redis. But there are more great tools and some are even lighter, so my solution builds on zmq. For this reason you do not have to run Redis, it is enough to run small Python script.
Requirements Review
Let me review your requirements before describing solution.
limit number of request to some resource to a number of requests within fixed period of time.
auto unlocking
resource (auto) unlocking shall happen in time shorter than 1 second.
it shall be distributed. I will assume, you mean that multiple distributed servers consuming some resource shall be able and it is fine to have just one locker service (more on it at Conclusions)
Concept
Limit number of requests within timeslot
Timeslot can be a second, more seconds, or shorter time. The only limitation is precision of time measurement in Python.
If your resource has hard limit defined per second, you shall use timeslot 1.0
Monitoring number of requests per timeslot until next one starts
With first request for accessing your resource, set up start time for next timeslot and initialize request counter.
With each request, increase request counter (for current time slot) and allow the request unless you have reached max number of allowed requests in current time slot.
Serve using zmq with REQ/REP
Your consuming servers could be spread across more computers. To provide access to LockerServer, you will use zmq.
Sample code
zmqlocker.py:
import time
import zmq
class Locker():
def __init__(self, max_requests=1, in_seconds=1.0):
self.max_requests = max_requests
self.in_seconds = in_seconds
self.requests = 0
now = time.time()
self.next_slot = now + in_seconds
def __iter__(self):
return self
def next(self):
now = time.time()
if now > self.next_slot:
self.requests = 0
self.next_slot = now + self.in_seconds
if self.requests < self.max_requests:
self.requests += 1
return "go"
else:
return "sorry"
class LockerServer():
def __init__(self, max_requests=1, in_seconds=1.0, url="tcp://*:7777"):
locker=Locker(max_requests, in_seconds)
cnt = zmq.Context()
sck = cnt.socket(zmq.REP)
sck.bind(url)
while True:
msg = sck.recv()
sck.send(locker.next())
class LockerClient():
def __init__(self, url="tcp://localhost:7777"):
cnt = zmq.Context()
self.sck = cnt.socket(zmq.REQ)
self.sck.connect(url)
def next(self):
self.sck.send("let me go")
return self.sck.recv()
Run your server:
run_server.py:
from zmqlocker import LockerServer
svr = LockerServer(max_requests=5, in_seconds=0.8)
From command line:
$ python run_server.py
This will start serving locker service on default port 7777 on localhost.
Run your clients
run_client.py:
from zmqlocker import LockerClient
import time
locker_cli = LockerClient()
for i in xrange(100):
print time.time(), locker_cli.next()
time.sleep(0.1)
From command line:
$ python run_client.py
You shall see "go", "go", "sorry"... responses printed.
Try running more clients.
A bit of stress testing
You may start clients first and server later on. Clients will block until the server is up, and then will happily run.
Conclusions
described requirements are fulfilled
number of requests is limited
no need to unlock, it allows more requests as soon as there is next time slot available
LockerService is available over network or local sockets.
it shall be reliable, zmq is mature solution, python code is rather simple
it does not require time synchronization across all participants
performance will be very good
On the other hand, you may find, that limits of your resource are not so predictable as you assume, so be prepared to play with parameters to find proper balance and be always prepared for exceptions from this side.
There is also some space for optimization of providing "locks" - e.g. if locker runs out of allowed requests, but current timeslot is already almost completed, you might consider waiting a bit with your "sorry" and after a fraction of second provide "go".
Extending it to real distributed lock manager
By "distributed" we might also understand multiple locker servers running together. This is more difficult to do, but is also possible. zmq allows very easy connection to multiple urls, so clients could really easily connect to multiple locker servers. There is a question, how to coordinate locker servers not to allow too many request to your resource. zmq allows inter-server communication. One model could be, that each locker server would publish each provided "go" on PUB/SUB. All other locker servers would be subscribed, and used each "go" to increase their local request counter (with a bit modified logic).
The lowest effort way to implement this is to use lockable.
It offers low-level lock semantics and it comes with a Python client. Iportantly, you don't need to set up any database or server, it works by storing the lock on the lockable servers.
Locks have variable TTLs, but you can also release them early:
$ pip install lockable-dev
from lockable import Lock
my_lock = Lock('my-lock-name')
# acquire the lock
my_lock.acquire()
# release the lock
my_lock.release()
For my cluster I'm using ZooKeeper with python-kazoo library for queues and locks.
Modified example from kazoo api documentation for your purpose:
http://kazoo.readthedocs.org/en/latest/api/recipe/lock.html
zk = KazooClient()
lock = zk.Lock("/lockpath", "my-identifier")
if lock.acquire(timeout=1):
code here
lock.release()
But you need at least three nodes for ZooKeeper as I remember.
Your requirements seem very specific. I'd consider writing a simple lock server then implementing the locks client side with a class that acquires a lock when it is created then deletes the lock when it goes out of scope.
class Lock(object):
def __init__(self,resource):
print "Lock acquired for",resource
# Connect to lock server and acquire resource
def __del__(self):
print "Lock released"
# Connect to lock server and unlock resource if locked
def callWithLock(resource,call,*args,**kwargs):
lock = Lock(resource)
return call( *args, **kwargs )
def test( asdf, something="Else" ):
return asdf + " " + something
if __name__ == "__main__":
import sys
print "Calling test:",callWithLock( "resource.test", test, sys.argv[0] )
Sample output
$ python locktest.py
Calling test: Lock acquired for resource.test
Lock released
locktest.py Else
The distributed lock manager Taooka http://taooka.com has a TTL accuracy to nanoseconds. But it only has Golang client library.