I am using a single threaded cherrypy server(threadpool = 1). I am trying to prioritise a specific request. That's i want a particular api request to be served even if main thread is busy. What are the possibilities to achieve this ?
Any help is appreciated.
Related
I am having trouble troubleshooting a performance bottleneck in one of my APIs and I have a theory that I need somebody with deeper knowledge of Python to validate for me.
I have a fastapi web service and a nodejs web service deployed on AWS. The node.js api is performing perfeclty under heavier loads, multiple concurrent requests taking same amount of time to be served.
My fastapi service however, is performing absurdly. If I make two requests concurrently, only one is served while the other has to wait for the first to be finished, hence the response time for the second request is twice as long as the first one.
My theory is that I am using Singleton pattern to instantiate the controller after a request comes to a route and the object already being in use and locked is causing the second request to wait until the first is resolved. Could this be it or am I missing something very obvious here? 2 concurrent requests should absolutely not be a problem for any type of web server.
I'm using flask and socket.io to build an app that queries an external API every 10 seconds, and emits the response to the users.
I've set up rq-scheduler to run a function every 10 seconds to query an API. I'd like to emit a socket.io message from this job, but I can't figure out how to do that, or what best practices are.
I see in the flask-socket.io docs, emitting from an external process, but this example is very limited. Is there a clear example of how to actually emit an event from an external process? Does the code written in the docs go in the rq task, or in the flask server code?
Do I need to instantiate a socket.io server every time the task is run? How does the task know about socket.io otherwise?
I'd really appreciate if someone could point me in the right direction here. If there is a more robust example that would be awesome.
I am in the process of making a web application that essentially takes in some web-stream from the client via their browser, and in real-time, sends it to a python server (Flask probably) that processes the frames in real-time and sends a response to the user. Now the backend has to be capable of handling web-streams from multiple clients simultaneously.
I am trying to grasp the framework for this entire application. What I have in mind is the following:
The user accesses the web-cam via their browser (e.g using webcamJS), the frames are sent from the frontend to the back-end through a web-socket. The task here is to establish a seemless handshake between the multiple clients and their processing requests.
There is a need for concurrency if the processing is to be done in real-time, multiple threads of the same image-processing-algorithm need to be executed. My take is that I make use of the multiple threads for this purpose or is there a better way of doing this? Is this even a feasible approach as the image-processing-algorithm (trained model) takes some time to load up , so it has to be always initialized at the backend and not start from scratch at every request.
The response from image-processing-algorithm need to get back to the frontend and the process goes on.
What I really need help is in drawing out the complete framework of this implementation. Any suggestions on the modules/frameworks to use with some implementations would be greatly appreciated.
Thank you.
You can use Flask for your web server, Keras to process the videos.
The standard library multiprocessing module will also be helpful to treat multiple feeds at once.
I'm using a frontend built in angularjs and a backend built in python and webapp2 in app engine.
The backend makes calls to a third party API, fetches data and returns to the frontend.
The API request from the backend may take upto 30s or more. The problem is the frontend can't really progress any further until it gets the data.
I tried running 3 simultaneous requests to the backend using different tabs and 2 of them failed. I'm afraid that this seems to suggest that the app only allows one user at a time.
What's the best way to handle this? One thought I have is:
Use task queues to run the API call to 3rd party in the background
Create a new handler which reads from the queue for the last task sent and let the frontend poll this one at regular intervals
Update the frontend once data is available
Is that the right way? I'm sure this is a problem solved in a frontend+backend kind of world, but I just don't know what to search for.
Thanks!
Requests from the frontend are capped at 30 seconds; after that they time out in the server side. That is part of GAE's design. Requests originating from the task queue get 10 minutes, so your idea is viable. However, you'll want some identifier to use for polling, rather than just using "the last sent," to distinguish between concurrent tasks.
I am trying to move away from CherryPy for a web service that I am working on and one alternative that I am considering is Tornado. Now, most of my requests look on the backend something like:
get POST data
see if I have it in cache (database access)
if not make multiple HTTP requests to some other web service which can take even a good few seconds depending on the number of requests
I keep hearing that one should not block the tornado main loop; I am wondering if all of the above code is executed in the post() method of a RequestHandler, does this mean that I am blocking the code ? And if so, what's the appropriate approach to use tornado with the above requirements.
Tornado comes shipped with an asynchronous (actually two iirc) http client (AsyncHTTPClient). Use that one if you need to do additional http requests.
The database lookup should also be done using an asynchronous client in order to not block the tornado ioloop/mainloop. I know there are a couple of tornado tailor made database clients (e.g redis, mongodb) out there. The mysql lib is included in the tornado distribution.