I have a time-consuming task in route /fibo and a route /home that return hello.
from flask import Flask, request
from fibo import fib_t
from datetime import datetime
app = Flask(__name__)
#app.route('/home')
def hello():
return 'hello '
#app.route('/fibo')
def cal_fibo():
request_rec_t = datetime.now().strftime("%M:%S")
v, duretion = fib_t(37)
return f'''
<div><h1>request recieved by server:</h1>
<p>{request_rec_t}</p>
<p>
<ul>
<li><b>calculated value</b>:{v}</li>
<li><b>that takes</b>: {duretion} seconds</li>
<ul>
</p>
</div>
<div>
<h1>resonse send by server:</h1>
<p>{datetime.now().strftime("%M:%S")}</p>
</div>'''
if __name__ == '__main__':
app.run(debug=True)
the fibo module
import time
def fib(a):
if a == 1 or a==2:
return 1
else:
return fib(a-1) + fib(a-2)
def fib_t(a):
t = time.time()
v = fib(a)
return v, f'{time.time()-t:.2f}'
if I made two requests to /fibo the second one will start running after the first one finishes his work.
but if I send a request to the time-consuming task and another one to /home I will receive the /home route response immediately while still first request isn't resolved.
if this is matters, I make the requests from different tabs of the browser.
for example, if I request /fibo in two different tabs. the page starts to load. the captured time for the first request is 10:10 and 10:20(the execution of Fibonacci number is tasked 10 seconds) during this request to be finished, the second tab still is loading. after that the second request is finished i see that the process started at the 10:20(the time that first request finishes process) and finished in 10:29(the execution of Fibonacci number is tasked 9 seconds).
but if I request /home while the /fibo in the first tab isn't returned, I will get the response.
why this is happening?
my general question is how and by whom the multiple requests are being handled?
how can I log the time requests reached the server?
EDIT:
if I add another route fibo2 with the same logic as cal_fibo view function, now I can run them concurrently(like /fibo and /home that I showed that are running concurrently, /fibo and /fibo2 are also running concurrently.)
EDIT 2:
as shown in the image, the second request was sent after the first request's response was received. why and how did this has happened?
(the related TCP handshake of each request has happened close to each other but how and who managed that the related HTTP get request is sent after the first request's response received?)
actually you have only one process running with only one thread all the time to enable threading and running more than one process this depends on the context (development or production) .
for development purposes
to enable threading you need to run
app.run(host="your.host", port=4321, threaded=True)
also to enable running more than one process e.g. 3 according to documentation
you need to run the following line
app.run(host="your.host", port=4321, processes=3)
for production purposes
when you run the app in production environment it is the responsability of the WSGI gateway (eg. gunicorn) that you configure on the cloud provider like heroku or aws to support the handling of multiple requests.
we use this method in production as it is more robust to crashes and also more efficent.
this was just a summary of the answers in this question
Related
in short:
I expect that while a single-threaded server is handling a request other requests must be dropped, (so the browser must continuously send that HTTP request to finally get the response) but in my example, this is not happening. why?
in my simple flask app I have two routes that, corresponding view function returns as follow:
/fibo that returns a JSON that contains the start and end of execution of view function that calculates Fibonacci 37. I use the recursive version of Fibonacci as a CPU-bound process.
start_process and end_process value is in the form of %M:%S (the current time's minute and second will be displayed).
/fibo2 that contains the same logic as /fibo. (i will use it to better illuster my question)
an example of returned JSON response is like this:
{"start_process":"02:02",
"fibo execution time":"7.22","fibo(37)":24157817,
"end_process":"02:09"}
I run the app by flask run command. I send two get requests to /fibo route, in separate tabs with the browser (while the first tab is loading I sent the second request)
the responses for each one are:
{"start_process":"02:02",
"fibo execution time":"7.22",
"fibo(37)":24157817,
"end_process":"02:09"}
and
{"start_process":"02:09",
"fibo execution time":"6.93",
"fibo(37)":24157817,
"end_process":"02:16"}
as the JSON responses illuster the second process started after that the first request is finished its job.
the Wireshark log of this process is :
in this section, my question is why after that for each of the separate requests TCP handshake happened the second request's get was sent after receiving the response of the first request? I expect that I must see lost HTTP packets, in Wireshark
I repeat this step and send requests to each of routes /fibo and /fibo2 the results are:
{"start_process":"29:41",
"fibo execution time":"16.30",
"fibo(37)":24157817,
"end_process":"29:58"
}
and
{"start_process":"29:43",
"fibo execution time":"16.21",
"fibo(37)":24157817,
"end_process":"29:59"
}
this time the requests are started approximately at the same time. ( maybe the 2-second delay is because of the time spent entering the second request in the browser.) but this time the time of executing the Fibonacci is doubled.
this time as the Wireshark shows the HTTP requests are sent separately from each other.
codes for flask app:
import re
from flask import Flask, request
from fibo import fib_t
from datetime import datetime
app = Flask(__name__)
#app.route('/home')
def hello():
return 'hello '
#app.route('/fibo')
def cal_fibo():
start_p = datetime.now().strftime("%M:%S")
v, duretion = fib_t(37)
end_p = datetime.now().strftime("%M:%S")
d = {'start_process':start_p,'fibo(37)':v,'fibo execution time':duretion, 'end_process':end_p}
return d
#app.route('/fibo2')
def cal_fibo2():
start_p = datetime.now().strftime("%M:%S")
v, duretion = fib_t(37)
end_p = datetime.now().strftime("%M:%S")
d = {'start_process':start_p,'fibo(37)':v,'fibo execution time':duretion, 'end_process':end_p}
return d
if __name__ == '__main__':
app.run(debug=True)
fibo module:
def fib(a):
if a == 1 or a==2:
return 1
else:
return fib(a-1) + fib(a-2)
def fib_t(a):
t = time.time()
v = fib(a)
return v, f'{time.time()-t:.2f}'
if __name__ == '__main__':
print(fib_t(38))
I'm trying to implement Fire and Forget mechanism using FastAPI. I'm facing few difficulties when implementing the mechanism.
I have two applications. One is developed with FastAPI and other is Flask. FastAPI will run in AWS Lambda and it will send requests to the Flask app running on AWS ECS.
Currently, I was able to send a request to the Flask API and receive an immediate response from the FastAPI app. But I see FastAPI still running bg_tasks.add_task(make_request, request) in the background which times out after lambda execution threshold time (15 mins).
Fast API application:
def make_request(data):
"""
Function to make a post request to flask application
:param data: Data from the user to write into the file
:return: None
"""
print("***** Inside post *****")
requests.post(url=root_url, data=data)
print("***** Post completed *****")
#router.post("/write-to-file")
async def write_to_file(request: Dict, bg_tasks: BackgroundTasks):
"""
Function to queue the requests and return to the post function
:param request: Request from the user
:param bg_tasks: Background task instance
:return: Some message
"""
print(f"****** Request call started ******")
bg_tasks.add_task(make_request, request)
print(f"****** Request completed ******")
return {"Message": "Data will be written into the file"}
Flask Application:
#app.route('/', methods=['POST'])
def write():
"""
Function to write the request data into the file
:return:
"""
request_data = request.form
try:
print(f"Sleep time {int(request_data['sleep_time'])}")
time.sleep(int(request_data["sleep_time"]))
request_data = dict(request_data)
request_data['current_time'] = str(datetime.now())
with open("data.txt", "a") as f:
f.write("\n")
f.write(json.dumps(request_data, indent=4))
return {"Message": "Success"}
except Exception as e:
return {"Message": e}
Fast API (http://localhost:8000/write-to-file/) calls the write_to_file method, which adds all the tasks (requests) into the background queue and runs them in background.
This function does not wait for the process to be completed. However, it returns the response to the client side. make_request method will then trigger the Flask endpoint (http://localhost:5000/), which in turn will process the request and write to a file. Consider make_request as one AWS lambda, if flask application takes more hours to process, the lambda will wait for longer time.
Is it possible to kill lambda once the request is published, or do something else to solve the timeout issue?
With the current setup, your lambda would run for as long, as the Flask endpoint would require to process your request. Effectively, both APIs run exactly the same time.
This is because the requests.post in the lambda function must wait for the response to finish. Given that you don't care about the results of that response, I can think of several other ways to solve this.
If I were you, I would move the queue processing to the ECS side. Then the only thing that lambda would only be responsible for putting a job into the queue that the ECS worker would process when it has capacity.
This option would let you get rid of one of the APIs: you would be able to query the Flask API directly and kill the lambda function, or instead kill the Flask API and run a worker process on ECS.
Alternatively, you could respond early on the Flask API side, which would finish your HTTP request, and thus the lambda execution, sooner. This can be confusing to set up and defeats the purpose of exposing an HTTP API in the first place. Also, under some circumstances, the Flask request execution could be terminated by the webserver after a default timeout (~30 seconds).
And finally, in case you really-really want to leave your code as it is now, you could set a request to timeout after a short period of time. In case you go this route, make sure to choose a long enough timeout for Flask to start processing the request:
try:
requests.post(url=root_url, data=data, timeout=5) # throw after 5 seconds of waiting
except requests.exceptions.Timeout:
pass
I created a python app in flask. Here the Skelton of the code
app = Flask(__name__)
#app.route('/', methods=['GET'])
def authentication():
'''athentication process'''
return 'authenticated'
so when user call the app it will authenticate. but if two user call that at same time or while processing one authentication i want to hold the new request until the old one finished then I want to start the new request. I've tried with semaphore but not working. Here is what I've tried
#app.route('/', methods=['GET'])
def authentication():
sem.acquire()
'''athentication process'''
sem.release()
return 'authenticated'
and I have deployed this in Heroku. Any idea how I can achieve this?
PS: If this can't be done at least i want to response the new request that another request is in process and try again after some time
Short answer: Dont worry about it.
This is the job of a web server. When you host the application in any Server like Apache , Nginx etc the server creates multiple processes of your flask app.When requst comes the, server program forwards it to any of the free processes, if no process is free server will queue the request until one process becomes free.
This is high level overwiew of how HTTP servers work.
I am trying to build REST API with only one call.
Sometimes it takes up to 30 seconds for a program to return a response. But if user thinks that service is lagging - he makes a new call and my app returns response with error code 500 (Internal Server Error).
For now it is enough for me to block any new requests if last one is not ready. Is there any simple way to do it?
I know that there is a lot of queueing managers like Celery, but I prefer not to overload my app with any large dependencies/etc.
You could use Flask-Limiter to ignore new requests from that remote address.
pip install Flask-Limiter
Check this quickstart:
from flask import Flask
from flask_limiter import Limiter
from flask_limiter.util import get_remote_address
app = Flask(__name__)
limiter = Limiter(
app,
key_func=get_remote_address,
default_limits=["200 per day", "50 per hour"]
)
#app.route("/slow")
#limiter.limit("1 per day")
def slow():
return "24"
#app.route("/fast")
def fast():
return "42"
#app.route("/ping")
#limiter.exempt
def ping():
return "PONG"
As you can see, you could ignore the remote IP address for a certain amount of time meanwhile you finish the process you´re running
DOCS
Check these two links:
Flasf-Limiter Documentation
Flasf-Limiter Quick start
I am working on a small python/flask project, which interfaces a heavy computation routine with a browser interface. For practical reasons, I have to keep the computation in a background process and reload/redirect the page (with output results) when the computation is done. The following is a minimal code of what I have so far (in reverse order):
interface.py
from flask import Flask
from threading import Thread
import time
app = Flask(__name__)
# step 4: rerender browser with output data
#app.route('/done')
def done(data_to_pass):
# rerender browser's html here?
print data_to_pass
return data_to_pass
# step 3: heavy computation routine
def background():
print "start runing backgroun process"
time.sleep(3) # simulate heavy computation routine
data = 'done from background'
done(data)
# step 2: initiate background process
def init():
t = Thread(target=background)
t.daemon = True
t.start()
# step 1: home interface
#app.route('/')
def front_end():
init()
return 'initiate bachground process'
if __name__ == '__main__':
app.run()
When the interface.py is running, accessing 127.0.0.1:5000 get a string initiate bachground process in the browser. However, the final data (string done from background in this case) only been processed in the server's terminal, not to the browser.
I believe this procedure is commonly done for most of the server, but I can't find any flask solution... Or do I go in the wrong direction?
If you want to know when the process is finished I would suggest to use one of:
Long polling
WebSocket
However you can reload whole page:
window.location.reload()
it is a good practice to return from the server only the result of the background process and update only related fragment of the page.