I want to do an asynchronous HTTP call to an external service at server startup and get an URL from there which I can then use in my own Sanic routing. E.g., fetch the string which needs to be my actual route via an httpx call (for the sake of simplicity, let's say the string returned by the external service is api/users/) and then use it as a route in my Sanic microservice.
Unfortunately it seems a before_server_start listener does not do the trick, as that is run after routes are loaded and I get a FinalizationError("Cannot finalize router more than once.") if I try to update the string value of a route.
Any ideas of how else I could hook up my call before defining / adding routes? I would like to keep it as coupled as possible to the Sanic app, i.e., not use a utility script that would run before it, but instead have the call to the external service triggered every time the app starts.
You can do it inside the before_server_start event with a little roundabout.
#app.before_server_start
async def setup_dynamic_routes(app, _):
app.router.reset()
# add your routes here
app.router.finalize()
Related
Here is how I want my program to work. Step 2 is what I am unsure of how to implement.
Client makes API call to /email endpoint
/email endpoint has a script run that gather emails from GMAIL API
Put contents into response object
Returns response object back to client
I understand how to make a static api response. But I can't get a python script to run when the api endpoint is hit.
I saw the flask tag in your post.
I only played around with flask for certain interviews, but know enough to say calling a python script outside your running server is somewhat of an antipattern.
I assume your backend is a flask app, so ideally, you'd want to wrap whatever script you have in your python script file in a function and simply call it from your flask method when the endpoint is hit.
Something like:
from flask import Flask
from custom_emails_module import gather_email
#api.route('/email', methods=["GET"])
def method_associated_with_your_endpoint():
# additional
gather_email()
where custom_emails_module should be the module you create for your gather_emails script.
Now, at the end of your gather_emails function, simple remember to return the correct type, usually done with:
return json.dumps("success": True, "data": python_object_with_several_emails)
Use something like PostMan for local debugging and remember to use application/json in header for Content-Type.
Good luck!
I have a function which calls several API's and updates the database upon being called. I want to schedule the function to run daily at specific time.
Already tried flask_apscheduler and APScheduler which gives this error:
This typically means that you attempted to use functionality that needed an active HTTP request. Consult the documentation on testing for information about how to avoid this problem.
Any leads on this will be helpful.
You should:
Post the code where you define your flask application.
Specify how you try to access the app.
How you're calling the APIs.
Whether those APIs are 3rd party or part of your blueprint.
However, this is probably a context issue. I have come across a similar one with SQLAlchemy before.
You will need to somehow get access to your app, either by using app_context or by importing current_app from Flask and accessing the config.
Assuming you imported the app where your function is used, try this:
with app.app_context():
# call your function here
Refer to this document for more information: Flask Documentation
Another approach you can try, is passing your app configurations through a config class object.
You can define the jobs you want to schedule and pass a reference to your function inside.
Check this example from flask-apscheduler repository on GitHub.
I have a flask restful api with an endpoint
api.add_resource(TestGet, '/api/1/test')
and I want to use the data from that endpoint to populate my jinja template. But everytime I try to call it in a sample route like this
#app.route('/mytest')
def mytest():
t = get('http://localhost:5000/api/1/test')
It never returns anything and stays in a loop meaning it is doing something with the request and never returns. Is there a reason I am not able to call it within the same flask app? I am able to reach the endpoint on the browser and from another python REPL. Thoroughly confused why this would happen and why it never returns anything. At least expecting an error.
Here is the entire sample of what I am trying to run
from flask import Flask
from requests import get
app = Flask('test')
from flask_restful import Api, Resource
api = Api(app)
class TestGet(Resource):
def get(self):
return {'test': 'message'}
api.add_resource(TestGet, '/test')
#app.route('/something')
def something():
resp = get('http://localhost:5000//test').json
print(resp)
from gevent.wsgi import WSGIServer
WSGIServer(('', 5000), app).serve_forever()
Use app.run(threaded=True) if you just want to debug your program. This will start a new thread for every request.
Please see this SO thread with nice explanation of Flask limitations: https://stackoverflow.com/a/20862119/5167302
Specifically, in your case you are hitting this one:
The main issue you would probably run into is that the server is single-threaded. This means that it will handle each request one at a time, serially. This means that if you are trying to serve more than one request (including favicons, static items like images, CSS and Javascript files, etc.) the requests will take longer. If any given requests happens to take a long time (say, 20 seconds) then your entire application is unresponsive for that time (20 seconds).
Hence by making request from within request you are putting your application into deadlock.
I was reading the source code of the Werkzeug library in github and in one of the examples (Simplewiki to name it), in the application.py file there is function which binds the application to the current active context. I would like to know why this is necessary, or where can I find something that explains this?
The function is this:
def bind_to_context(self):
"""
Useful for the shell. Binds the application to the current active
context. It's automatically called by the shell command.
"""
local.application = self
And this is the part where the dispatcher binds the request.
def dispatch_request(self, environ, start_response):
"""Dispatch an incoming request."""
# set up all the stuff we want to have for this request. That is
# creating a request object, propagating the application to the
# current context and instanciating the database session.
self.bind_to_context()
request = Request(environ)
request.bind_to_context()
As far as I know, contexts in the Werkzeug is about separating environment between different threads. For example, contexts are very common thing in the Flask framework which is built on top of the Werkzeug. You can run Flask application in multi-threaded mode. In such case you'll have only one application object which is accessed by multiple threads simultaneously. Each thread requires a piece of data within the app for private usage. Storing such data is organized via thread's local storage. And this is called the context.
I have a process a user can launch which inserts a bunch of items into a queue. This queue sends a request to the worker's url /pipeline/foo and the handler takes it from there.
The handler that consumes items coming into /pipeline/foo is on a separate module module-pipeline.
The issue is that in production dispatch.yaml doesn't seem to dispatch the internal requests made by the queue to the right module. However, if I manually send a request to the URL (as a user inserting the URL in the browser), it seems to dispatch me to the right module...
dispatch.yaml
application: my-app
dispatch:
- url: "*/pipeline/*"
module: module-pipeline
load-queue.py
# does some things before...
for url in urls_to_load:
task = taskqueue.Task(url='/pipeline/foo', params={'id': url.key.id()})
queue.add(task)
This works fine on the dev_appserver however in production when the queue sends the request to pipeline/fooits processed by the default module (returning a 404 as it's not implemented there), whereas if I send the same request manually through my browser (GET request), it's processed by module-pipeline
Any ideas what's wrong?
Try to use parameter target when defining the queue in queue.yaml instead of routing defined in dispatch.yaml.
Target parameter
A string naming a module/version, a frontend version, or a backend, on which to execute all of the tasks enqueued onto this queue.
The module (or frontend or backend) and version in which the handler runs is determined by:
The target or header keyword arguments in your call to the Task() constructor.
The target directive in the queue.yaml file.