How can I start a SocketIO server and have it be listening in the background while I continue to execute code on the main thread?
Right now I am using the package python-socketio. I have also testedflask-socketio which using python-socketio anyway.
https://python-socketio.readthedocs.io/en/latest/server.html
What I've mostly tried is starting the server with sio.start_background_task.
For example:
class SocketIOServer(socketio.AsyncNamespace):
def __init__(self):
super().__init__()
self.sio = socketio.AsyncServer()
self.sio.register_namespace(self)
self.app = web.Application()
self.sio.attach(self.app)
self.task = None
def run(self):
web.run_app(self.app, port=8080)
def start(self):
self.task = self.sio.start_background_task(self.run)
I tried the above and multiple variations like using Flask, Tornado, etc.
To be more clear this is basically what I want to do:
if __name__ == '__main__':
# ...
# start app, e.g. -> web.run_app(self.app, port=8080)
# I want to continue to execute code here
I don't fully understand how everything is working, am I asking a stupid question?
The problem is as you described it. You do not fully understand how everything works.
The most important thing that you are missing is that the python-socketio package is not a web server. This package just implements the Socket.IO logic, but you have to add a web server through which your Socket.IO code is available.
From the code that you included in your question, it appears that you have selected to attach the Socket.IO code to a web application built with the aiohttp framework, and use its own web server. Correct?
Then the real question that you have is how to run the aiohttp web server in a non-blocking way.
And it runs out that aiohttp has information on this in their documentation: https://docs.aiohttp.org/en/stable/web_advanced.html#application-runners. The example they have does this:
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, 'localhost', 8080)
await site.start() # <-- starts the server without blocking
# you can do whatever you want here!
I am learning flask and PyMongo right now and came across ChangeStreams. I do understand how ChangeStreams work but I have only worked with them in Node and Express. I have implemented ChangeStreams in my Flask app as following:
with ms.db.collection.watch() as stream:
for change in stream:
print(change)
On the official docs pages, it says that it's a blocking method. But how would I go about in making it non-blocking? Because currently my ChangeStream logic is in a different file and I import it into the server.py file. So when it never goes past that import and the Flask App doesn't start at all. Below is my server.py
from flask import Flask, render_template, request
import mongo_starter as ms
import changestream as cs
app = Flask(__name__)
#app.route('/')
def home():
return render_template('index.html')
if __name__ == "__main__":
app.run(host="0.0.0.0", port="5000")
Below is my ChangeStream.py
import mongo_starter as ms
with ms.db.collection.watch() as stream:
for change in stream:
print(change)
Below is my MongoStarter.py that actually initiates the connection to Mongo
import pymongo
import mongo_config as mc
print(mc.data_header)
try:
print('Connecting to Database...')
mongo_client = pymongo.MongoClient(mc.mongo_url)
db = mongo_client['PyMongo']
collection = db['Test Data']
print("Connection to Database Successful!")
except pymongo.errors.InvalidURI:
print('Error Connecting to Database')
When I run the app using nodemon it prints the following to the output.
[nodemon] restarting due to changes...
[nodemon] starting `python server.py`
----------------- MONGO CONNECTION LOG --------------------
Connecting to Database...
Connection to Database Successful!
So it never actually goes past the change stream method. How can I make it so it worked in an async way? I have looked at asyncio, but wanted to see if there was any way to implement it without using asyncio.
I have been trying to follow the tutorials to get flask apps to run on Heroku, like this one: https://dev.to/emcain/how-to-set-up-a-twitter-bot-with-python-and-heroku-1n39.
They all tell you to put this in your code in a file server.py:
from flask import Flask
app = Flask(__name__)
app.run(host='0.0.0.0')
And then run the app via the following command:
python3 server.py
But the tutorials don't explain how to connect the actual function you want to run using the app. In my case, I have a File testbot.py that has the function test(arg1) that contains the code I want to execute:
def test(arg1):
while(1):
#do stuff with arg1 on twitter
I want to do something like this:
from flask import Flask
from testbot import test
from threading import Thread
app = Flask(__name__)
app.addfunction(test(arg1='hardcodedparameter'))
app.run(host='0.0.0.0')
So that when the app runs my test() function executes with the argument. Right now my server is starting, but nothing is happening.
Am I thinking about this correctly?
*Edit: I got it working with the solution, so my server.py now looks like this:
from flask import Flask
from testbot import test
def main_process():
test("hardcodeparam")
app = Flask(__name__)
Thread(target=main_process).start()
app.run(debug=True,host='0.0.0.0')
And now test runs as expected.
Before app.run, register the function with a path, e.g.
#app.route('/')
def test(): # no argument
... do one iteration
return 'ok'
Then visiting the URL will trigger the function. Sites such as https://cron-job.org/ can automate that visiting on a regular basis for free, as suggested here.
If the regular intervals aren't good enough, then you could try:
#app.route('/')
def index(): # no argument
return 'ok'
def test():
while True:
# do stuff
from threading import Thread
Thread(target=test).start()
app.run(...)
You will probably still need to have a job regularly visiting the URL so that Heroku sees that the server is alive and in use.
I have a website build using python3.4 and flask...I have generated my own self-signed certificate and I am currently testing my website through localhost.
I am using the python ssl module along with this flask extension: https://github.com/kennethreitz/flask-sslify
context = ('my-cert.pem', 'my-key.pem')
app = Flask(__name__)
sslify = SSLify(app)
...
if __name__ == '__main__':
app.debug = False
app.run(
host="127.0.0.1",
port=int("5000"),
ssl_context=context
)
This does not seem to be working however. I took a look in the sslify source code and this line does not seem to be working
def init_app(self, app):
"""Configures the configured Flask app to enforce SSL."""
app.before_request(self.redirect_to_ssl)
app.after_request(self.set_hsts_header)
Specifically the function call to redirect_to_ssl (I added my own print statement under the redirect_to_ssl function and my statement was never printed)
def redirect_to_ssl(self):
print("THIS IS WORKING")
"""Redirect incoming requests to HTTPS."""
Should we redirect?
criteria = [
request.is_secure,
current_app.debug,
request.headers.get('X-Forwarded-Proto', 'http') == 'https'
]
if not any(criteria) and not self.skip:
if request.url.startswith('http://'):
url = request.url.replace('http://', 'https://', 1)
code = 302
if self.permanent:
code = 301
r = redirect(url, code=code)
return r
I am pretty new to python. Any ideas?
To me, it appears you're making it more complicated than it needs to be. Here is the code I use in my views.py script to force user to HTTPS connections:
#app.before_request
def before_request():
if not request.is_secure:
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
According with the docs, after pip install Flask-SSLify you only need to insert the following code:
from flask import Flask
from flask_sslify import SSLify
app = Flask(__name__)
sslify = SSLify(app)
I have done it and it works very well. Am I missing something in the discussion ?
The Flask Security Guide recommends using Flask-Talisman.
$ pip install flask-talisman
Usage example:
from flask import Flask
from flask_talisman import Talisman
app = Flask(__name__)
Talisman(app)
It forces HTTPS by default (from the README):
force_https, default True, forces all non-debug connects to https.
Personally, I got some errors relating to CSP (Content Security Policy) which I disabled with:
Talisman(app, content_security_policy=None)
But use this at your own risk :)
Thanks to answer from Kelly Keller-Heikkila and comment by jaysqrd I ended up doing this in my Flask app:
from flask import request, redirect
...
#app.before_request
def before_request():
if app.env == "development":
return
if request.is_secure:
return
url = request.url.replace("http://", "https://", 1)
code = 301
return redirect(url, code=code)
I tried the flask_sslify solution suggested by Rodolfo Alvarez but ran into this issue and went with the above solution instead.
If the app is running in development mode or the request is already on https there's no need to redirect.
Here is a flask solution if you are on aws and behind a load balancer. Place it in your views.py
#app.before_request
def before_request():
scheme = request.headers.get('X-Forwarded-Proto')
if scheme and scheme == 'http' and request.url.startswith('http://'):
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
The standard solution is to wrap the request with an enforce_ssl decorator that after checking some flags in the app configuration (flags you can set depending on your debugging needs) modifies the request's url with request.url.
As it is written here.
You can modify the code to make it working with before_request as suggested by #kelly-keller-heikkila
I use a simple extra app that runs on port 80 and redirect people to https:
from flask import Flask,redirect
app = Flask(__name__)
#app.route('/')
def hello():
return redirect("https://example.com", code=302)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=80)
An alternative to the other answers that I've been able to use with great success:
from http import HTTPStatus
from typing import Optional
from flask import Response, redirect, request, url_for
def https_redirect() -> Optional[Response]:
if request.scheme == 'http':
return redirect(url_for(request.endpoint,
_scheme='https',
_external=True),
HTTPStatus.PERMANENT_REDIRECT)
# ..
if app.env == 'production':
app.before_request(https_redirect)
# ..
On app engine flex, add:
from werkzeug.middleware.proxy_fix import ProxyFix
def create_app(config=None):
app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app)
In addition to the solution of:
#app.before_request
def before_request():
if not request.is_secure:
url = request.url.replace('http://', 'https://', 1)
code = 301
return redirect(url, code=code)
Otherwise it'll cause infinite redirects since SSL is unwrapped behind the proxy but is noted in the headers.
I ran into the same solution running a Flask application in AWS Elastic Beanstalk behind a load balancer. The following AWS docs provided two steps to configure the environment for http redirects: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-httpredirect.html Following both steps fixed my issue.
One thing to note is that you'll have to create the .ebextenions folder at the root level of your application source bundle and add the config file to that .ebextensions folder. The readme here: https://github.com/awsdocs/elastic-beanstalk-samples explains this in a bit more detail.
For some reason it seems, requests from a Private AWS API Gateway with a VPC endpoint don't include the "X-Forwarded-Proto" header. This can break some of the other solutions (either it doesn't work or it continuously redirects to the same url). The following middleware forces https on most flask generated internal redirects:
class ForceHttpsRedirects:
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
environ["wsgi.url_scheme"] = "https"
return self.app(environ, start_response)
# Usage
app = flask.Flask(__name__)
app.wsgi_app = ForceHttpsRedirects(app.wsgi_app) # Add middleware to force all redirects to https
Use:
app.run(port="443")
All modern browsers automatically use HTTPS when the port is 443 or 8443.
I'm using cloud foundry python app which is behind a load balancer (like https://stackoverflow.com/users/5270172/kelly-keller-heikkila said) .
This resolution helped me by adding (_external and _Scheme to the url_for function). https://github.com/pallets/flask/issues/773
I had the same issue and mine is a brute-force solution, but it works.
Heroku in the past suggested flask_sslify, which is not maintained anymore. Nowadays the proper way in Flask should be flask-talisman, but I tried it and it has bad interactions with boostrap templates.
I tried the anulaibar solution but it did not always worked for me.
The following is what I came up with:
#app.before_request
def before_request():
# If the request is sicure it should already be https, so no need to redirect
if not request.is_secure:
currentUrl = request.url
if currentUrl.startswith('http://'):
# http://example.com -> https://example.com
# http://www.example.com -> https://www.example.com
redirectUrl = currentUrl.replace('http://', 'https://', 1)
elif currentUrl.startswith('www'):
# Here we redirect the case in which the user access the site without typing any http or https
# www.example.com -> https://www.example.com
redirectUrl = currentUrl.replace('www', 'https://www', 1)
else:
# I do not now when this may happen, just for safety
redirectUrl = 'https://www.example.com'
code = 301
return redirect(redirectUrl, code=code)
I have the domain registered in godaddy which is also redirecting to https://www.example.com.
In my case Flask app is sitting behind AWS API Gateway and solutions with #app.before_request were giving me permanent redirects.
The following simple solution finally worked:
#app.after_request
def adjust_response(response):
....
if response.location:
if app.env != "development":
response.location = response.location.replace("http://", "https://", 1)
return response
I want to create a webapp that dynamically redirects to a URL, based on address that user typed. When a user visit my site by a address like this:
http://mydomain1.com/a1b2c3d4
I want redirect this user to URL:
http://mydomain2.com/register.php?id=a1b2c3d4&from=mydomain1.com
Yay, I love a good fight!
from pyramid.config import Configurator
from pyramid.httpexceptions import HTTPFound
from paste.httpserver import serve
config = Configurator()
config.add_route('redirect', '/{arg}')
def redirect_view(request):
dst = 'http://mydomain2.com/register.php?id={id}&from={host}'
args = {
'id': request.matchdict['arg'],
'host': request.host,
}
return HTTPFound(dst.format(**args))
config.add_view(redirect_view, route_name='redirect')
serve(config.make_wsgi_app(), host='0.0.0.0', port=80)
Here goes my attempt, I'm almost newbie in flask, so it should have room to improve
from flask import Flask, redirect, request
app = Flask(__name__)
host = 'domain2.org'
#app.route('/<path>')
def redirection(path):
return redirect('http://'+host+'/register.php?id='+path+'&from='+request.host)
if __name__ == '__main__':
app.run()
Edited to add the host to the from parameter
My solution was to use a Werkzeug rule using the path type :
host = 'domain2.org'
#app.route('/<path:path>')
def redirection(path):
return redirect('http://%s/%s' % (host, path), code=301)
This can be useful if you move a site and want another site instead with redirection on others pages.
There's a pyramid_rewrite extension (https://pypi.python.org/pypi/pyramid_rewrite/) that looks unmaintained, but seems to work. I had a use case it didn't handle, though: using Configure.include() with the route_prefix parameter.
It occurred to me that the usual approach is to do URL rewrites in the server, and I was using a WSGI server from the Python standard library. How hard could it be?
Make a custom request handler class:
from wsgiref.simple_server import make_server, WSGIRequestHandler
class MyReqHandler(WSGIRequestHandler):
def get_environ(self):
env = WSGIRequestHandler.get_environ(self)
if env['PATH_INFO'].startswith('/foo'):
env['PATH_INFO'] = env['PATH_INFO'].replace('foo', 'bar', 1)
return env
Pass it to make_server() when creating your server:
srvr = make_server('0.0.0.0', 6543, app, handler_class=MyReqHandler)
It works!
Straight-up substitution is all I needed for the problem at hand. Extending it to use regular expressions and exposing it via a nice API would be pretty straightforward.
I have another solution, that is straight-up pyramid, so it will work with some other wsgi server:
from pyramid.events import NewRequest, subscriber
#subscriber(NewRequest)
def mysubscriber(event):
req = event.request
if req.path_info.startswith('/~cfuller'):
req.path_info = req.path_info.replace('foo', 'bar', 1)
That's the declarative way, and it requires a config.scan(). Imperitively, you'd do something like
config.add_subscriber(mysubscriber, NewRequest)
See http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/events.html for the skinny on events.