Why server is not detecting changes source code in my python project? - python

I change the original file to add a new routing, but the changes don't work even if I restart the gunicorn server. What is the reason for this? Is it Git, Visual Code, my remote Linux, VirtualanEnv ... or what? I'm deeply confused

It is hard to what the reason is, as there are many "moving" parts (the IDE, Gunicorn, etc); but I cannot reproduce your issue.
Maybe the server was not reloaded or restarted, against your expectation, and you are simply still dealing with Gunicorn running old code?
If you are sure you have restarted your debug server, then it should not matter; otherwise make sure to pass the --reload option, see also: gunicorn autoreload on source change.
As to the application itself, the following MRE works for me:
import falcon
class Contest:
def on_get(self, req, resp, contest_id):
resp.media = {
'contest_id': contest_id,
'uri_template': req.uri_template,
}
def on_get_ping(self, req, resp):
resp.content_type = falcon.MEDIA_TEXT
resp.text = 'PONG\n'
application = falcon.App()
contest = Contest()
application.add_route('/api/v1/ping', contest, suffix='ping')
application.add_route('/api/v1/member/contest/{contest_id:int}', contest)
application.add_route('/api/v1/member/contest/new/{contest_id:int}', contest)
When running with gunicorn --reload --access-logfile - test:application, I can even comment out routes or bring them back, save, and the changes are reflected in the application's behaviour.
Checking the end points in question:
$ curl http://localhost:8000/api/v1/ping
PONG
$ curl http://localhost:8000/api/v1/member/contest/1
{"contest_id": 1, "uri_template": "/api/v1/member/contest/{contest_id:int}"}
$ curl http://localhost:8000/api/v1/member/contest/new/2
{"contest_id": 2, "uri_template": "/api/v1/member/contest/new/{contest_id:int}"}

Related

Python not getting called(?) in React/Flask Project

Summary: I am trying to teach myself Flask and React with the tools I have at hand during the pandemic. I am hampered by the lack of a good React editor for remote files, I suspect I may not be seeing error messages such an editor would be showing me, and I suspect that the python script I am trying to invoke is never getting touched.
Any help, including how to better see error messages, or what might be mis-configured, are desperately appreciated.
What I have:
A command line only interface to an Ubuntu 18.04 container. This means: no remote desktop, no GUI, no specialized react editor. I cannot do anything about this.
React server up and running on container port 3000, accessible to the outside world with a path prefix per this question and answer. Meaning, I can run react from the text terminal command line, edit react project files in another window with vi, point a browser to it from well outside the system and it works.
Flask server up and running on container port 3001 (port 5000 already doing something else) and React server configured with port 3001 as a proxy. God help me, I am using the DANGEROUSLY_DISABLE_HOST_CHECK option to get that to work. No special consideration for port prefixing or port forwarding-- this seems all internal to the container, so I don't think any special handling is necessary.
A tutorial project as per this site. I will include the two relevant code files below.
The results of this are:
No errors in either of the terminals where I invoked the react and flask servers.
The web page displays, but improperly: Rather than displaying the correct time, the page displays only whatever I pre-loaded into currentTime.
I am highly skeptical that the python script is ever running at all: If I mangle the api.py file with egregious syntactical errors... no change in behavior. If I mangle the App.js with errors, the system dies. If I mangle the App.js with the specific error of fetch('/garbage').then... rather than fetch('/time').then... there are still no errors, which I find HIGHLY suspicious.
This makes me think I may not be seeing some critical error or warning messages.
My suspicion was that this might be some type of CORS issue, but two attempts at adapting solutions to this question failed.
Code from App.js:
import React, { useState, useEffect } from 'react';
import logo from './logo.svg';
import './App.css';
function App() {
const [currentTime, setCurrentTime] = useState(2);
useEffect(() => {
fetch('/time').then(res => res.json()).then(data => {
setCurrentTime(data.time);
});
}, []);
return (
<div className="App">
<header className="App-header">
<img src={logo} className="App-logo" alt="logo" />
<p>
Edit <code>src/App.js</code> and save to reload.
</p>
<a
className="App-link"
href="https://reactjs.org"
target="_blank"
rel="noopener noreferrer"
>
Learn React
</a>
<p>The current time is {currentTime}.</p>
</header>
</div>
);
}
export default App;
Code from api.py (not including CORS attempts):
import time
from flask import Flask
app = Flask(__name__)
#app.route('/time')
def get_current_time():
return {'time': time.time()}
UPDATE:
On the advice of this answer, I added some console logging statements and looked at the console logging in Chrome.
App.js:11 GET http://<CORRECT_ADDRESS>/time 404 (Not Found)
(anonymous) # App.js:11
commitHookEffectListMount # react-dom.development.js:19731
commitPassiveHookEffects # react-dom.development.js:19769
callCallback # react-dom.development.js:188
invokeGuardedCallbackDev # react-dom.development.js:237
invokeGuardedCallback # react-dom.development.js:292
flushPassiveEffectsImpl # react-dom.development.js:22853
unstable_runWithPriority # scheduler.development.js:653
runWithPriority$1 # react-dom.development.js:11039
flushPassiveEffects # react-dom.development.js:22820
(anonymous) # react-dom.development.js:22699
workLoop # scheduler.development.js:597
flushWork # scheduler.development.js:552
performWorkUntilDeadline # scheduler.development.js:164
Note 1: Port 3001 is not included, which seems wrong.
Note 2: The path prefix is not included, which seems correct.
I would have expected this to be:
App.js:11 GET http://<CORRECT_ADDRESS>:3001/time
If the port, and not the path prefix, needs to be included in that call, how do I arrange that?
Flask is running on port 3001, and the last line of package.json is:
"proxy": "http://localhost:3001"
Further Update:
using fetch('http://<CORRECT_ADDRESS>:3001/time') now yields the following console errors:
WebSocket connection to 'ws://<CORRECT_ADDRESS>/sockjs-node' failed: Error during WebSocket handshake: Unexpected response code: 404
The development server has disconnected.
Refresh the page if necessary.
GET http://<CORRECT_ADDRESS>:3001/time net::ERR_CONNECTION_TIMED_OUT
Uncaught (in promise) TypeError: Failed to fetch
So the GET seems to be well-formed, now, but times out due to the failed WebSockets call, which may or may not be correctly formed.
You may be seeing an infinite loop, where state is not having a chance to update.
Try this:
#app.route('/time')
def get_current_time():
print( time.time() )
return {'time': time.time()}
Is /time getting hits? Because your useEffect() code will run every time the state changes, which is every time it runs: infinite loop.
If /time is not getting hits, then your problem is in that relative path. Try this:
useEffect(() => {
console.log('fetching');
fetch('/time').then(res => res.json()).then(data => {
console.log('fetched:' + data);
setCurrentTime(data.time);
});
}, []);
Next, check your ports. It sounds like you may have React/Webpack running on 3001, and the server running on 5000.
Try these:
http://localhost:5000/time
http://localhost:3000/time
http://localhost:3001/time

gunicorn and flask-jwt-extended not getting current user

I've been building an app that uses flask, flask_jwt_extended and the decorator #jwt_required around protected functions that need an access token to access. From these endpoints, I can use flask_jwt_extended's get_current_user function to fetch the currrent user.
This has been working fine while in development, but now I'm planning to serve a production application using gunicorn. However, when I run the gunicorn server, it seems like the get_current_user function is always returning none. For example, the following code works with python3 -m flask run, but not with gunicorn run:app -b localhost:5000. What could be the problem here?
#jwt_required
def get_user_by_id(user_id: str) -> Dict[str, Any]:
# returns user when using flask run, but not with gunicorn
curr_user = get_current_user()
Class-worker
Do you use multiple workers and sync class ?
Maybe try to look at gunicorn worker-class
If you try to use the sync worker type and set the threads setting to more than 1, the gthread worker type will be used instead.
In your python configuration file or you can change worker-class directly in the command line.
worker_class = 'gthread'
Check your JWT_SECRET_KEY
JWT_SECRET_KEY need to have the same value all over each workers

ajax request to a linux server sometimes returns me the answer, sometimes returns me error 500

I have a python application with gunicorn and flask on a centos machine. I access the application from another machine in the following way:
http://host:port/nameOfFunction?path=https://site/directory1/directory2/directory3/file.pdf
The path is correct, because it works. When i enter the same path multiple times, sometimes it works, others, don't.
I am using gunicorn (with the follow command):
gunicorn -b 0.0.0.0:8080 start:app
to start my application. When the answer returns, the screen showing gunicorn prints some piece of code that i put to see some informations.
When it returns me error 500, the code from start isn't printed at all.
I dont have a clue what is the cause of this.
the beginning of the code is:
#app.route("/call", methods=['POST', 'GET'])
def call():
print '\n============================BEGINS======================\n'
path =request.args.get('path')
newPath = path.rsplit('/',1)[1]
directoriesRoot(root)
directoriesRoot(pdfs)
parameters = ["curl", path.replace(' ','%20'), '--output',pdfs+newPath]
p = subprocess.Popen(parameters, stdout=subprocess.PIPE)
output, err = p.communicate()
if err:
return 'ERROR'
person= functionSecundaryWithNoImportance([newPath])
return jsonify(person)
Someone has some clue or already pass for a similar problem?
With the help of user3788685, i found out that my python application was on port 8080 (and probably one of apache's default ports were 8080) so it conflicts. Sometimes it directs to python, others to apache. So i changed the port for my python application to 1234:
gunicorn -b 0.0.0.0:1234 start:app
and now works as a charm!

Streaming server issue with gunicorn and flask and Nginx

I am using gunicorn and flask for a web service. I am trying to get my head around running a streaming route (not sure if that is the correct terminology).
my route looks like this:
#app.route('/delay')
def delay():
from time import sleep
def delay_inner():
for i in range(10):
sleep(5)
yield json.dumps({'delay': i})
return Response(delay_inner(), mimetype="text/event-stream")
I expect that the server would yield the output each time that delay_inner does a yield. But, what I am getting is all the json responses at once, and only when the delay_inner finishes execution.
What am I missing here?
--EDIT--
I have fixed the issue for Flask and Gunicorn, I am able to run it as expected by using the flask server, and by going to the Gunicorn port. It streams the data as expected. However, and I should have mentioned this in the original post, I am also running behind nginx. And that is not set up correctly to stream. Can anyone help with that?
You need to turn off the nginx proxy buffering.
location /delay {
proxy_pass http://127.0.0.1:8080;
proxy_buffering off;
}
and reload the config
nginx -s reload

Debugging a Flask app running in Gunicorn

I've been working on a new dev platform using nginx/gunicorn and Flask for my application.
Ops-wise, everything works fine - the issue I'm having is with debugging the Flask layer. When there's an error in my code, I just get a straight 500 error returned to the browser and nothing shows up on the console or in my logs.
I've tried many different configs/options.. I guess I must be missing something obvious.
My gunicorn.conf:
import os
bind = '127.0.0.1:8002'
workers = 3
backlog = 2048
worker_class = "sync"
debug = True
proc_name = 'gunicorn.proc'
pidfile = '/tmp/gunicorn.pid'
logfile = '/var/log/gunicorn/debug.log'
loglevel = 'debug'
An example of some Flask code that borks- testserver.py:
from flask import Flask
from flask import render_template_string
from werkzeug.contrib.fixers import ProxyFix
app = Flask(__name__)
#app.route('/')
def index():
n = 1/0
return "DIV/0 worked!"
And finally, the command to run the flask app in gunicorn:
gunicorn -c gunicorn.conf.py testserver:app
Thanks y'all
The accepted solution doesn't work for me.
Gunicorn is a pre-forking environment and apparently the Flask debugger doesn't work in a forking environment.
Attention
Even though the interactive debugger does not work in
forking environments (which makes it nearly impossible to use on
production servers) [...]
Even if you set app.debug = True, you will still only get an empty page with the message Internal Server Error if you run with gunicorn testserver:app. The best you can do with gunicorn is to run it with gunicorn --debug testserver:app. That gives you the trace in addition to the Internal Server Error message. However, this is just the same text trace that you see in the terminal and not the Flask debugger.
Adding the if __name__ ... section to the testserver.py and running python testserver.py to start the server in development gets you the Flask debugger. In other words, don't use gunicorn in development if you want the Flask debugger.
app = Flask(__name__)
app.config['DEBUG'] = True
if __name__ == '__main__':
app.run()
## Tip for Heroku users:
Personally I still like to use `foreman start`, instead of `python testserver.py` since [it sets up all the env variables for me](https://devcenter.heroku.com/articles/config-vars#using-foreman). To get this to work:
Contents of Procfile
web: bin/web
Contents of bin/web, file is relative to project root
#!/bin/sh
if [ "$FLASK_ENV" == "development" ]; then
python app.py
else
gunicorn app:app -w 3
fi
In development, create a .env file relative to project root with the following contents (docs here)
FLASK_ENV=development
DEBUG=True
Also, don't forget to change the app.config['DEBUG']... line in testserver.py to something that won't run Flask in debug mode in production.
app.config['DEBUG'] = os.environ.get('DEBUG', False)
The Flask config is entirely separate from gunicorn's. Following the Flask documentation on config files, a good solution would be change my source to this:
app = Flask(__name__)
app.config.from_pyfile('config.py')
And in config.py:
DEBUG = True
For Heroku users, there is a simpler solution than creating a bin/web script like suggested by Nick.
Instead of foreman start, just use foreman run python app.py if you want to debug your application in development.
I had similiar problem when running flask under gunicorn I didn't see stacktraces in browser (had to look at logs every time). Setting DEBUG, FLASK_DEBUG, or anything mentioned on this page didn't work. Finally I did this:
app = Flask(__name__)
app.config.from_object(settings_map[environment])
if environment == 'development':
from werkzeug.debug import DebuggedApplication
app_runtime = DebuggedApplication(app, evalex=False)
else:
app_runtime = app
Note evalex is disabled because interactive debbugging won't work with forking (gunicorn).
I used this:
gunicorn "swagger_server.__main__:app" -w 4 -b 0.0.0.0:8080
You cannot really run it with gunicorn and for example use the flask reload option upon code changes.
I've used following snippets in my api launchpoint:
app = Flask(__name__)
try:
if os.environ["yourapp_environment"] == "local":
run_as_local = True
# some other local configs e.g. paths
app.logger.info('Running server in local development mode!')
except KeyError as err:
if "yourapp_environment" in err.args:
run_as_local = False
# some other production configs e.g. paths
app.logger.info('No "yourapp_environment env" given so app running server in production mode!')
else:
raise
...
...
...
if __name__ == '__main__':
if run_as_local:
app.run(host='127.0.0.1', port='8058', debug=True)
else:
app.run(host='0.0.0.0')
For above solution you need to give export yourapp_environment = "local" in the console.
now I can run my local as python api.py and prod gunicorn --bind 0.0.0.0:8058 api:app
The else statement app.run() is not actually needed, but I keep it for reminding me about host, port etc.
Try setting the debug flag on the run command like so
gunicorn -c gunicorn.conf.py --debug testserver:app
and keep the DEBUG = True in your Flask application. There must be a reason why your debug option is not being applied from the config file but for now the above note should get you going.

Categories