uWSGI has an option for disabling logging from the application it is running. This way, I can control my application logging from Python. However, disabling these still leaves the uWSGI startup messages in the usual format, e.g.
[uWSGI] getting INI configuration from /app/uwsgi.ini
*** Starting uWSGI 2.0.18 (64bit) on [Tue Nov 12 08:36:47 2019] ***
compiled with version: 8.3.0 on 12 November 2019 08:35:14
...
The problem with these is that they are not formatted as json, which means that all log messages are shown as errors in my logging viewer, Google Stackdriver.
uWSGI has an option for encoding/formatting logs, see here, specifically the log-encoder = json { ... option.
The problem is that if I encode the logs as json, using the option above, specifically the following,
log-encoder = json {"unix":${unix}, "msg":"${msg}"}
the startup logs simply disappear.
How can I format startup logs as json?
uWSGI has three sources of log messages:
Application logs (python logs)
uWSGI server logs (information about server startup, errors, warnings)
Request logs
If I'm getting it right, you have problems with logs under point 2 (uWSGI server logs).
A configuration like this one works for me:
logger = default stdio
log-route = default ^((?!\{).)*$
log-encoder = json:default {"time":"${micros}", "source":"uwsgi", "message":"${msg}"}
log-encoder = nl
You can separately format the request logs using the logger-req and log-req-encoder options.
More on JSON logging in uWSGI and full configuration of all log types can be found here
Related
I'm attempting to debug a Flask/Python app running on Google Appengine flexible environment.
However, I see a warning message within the Stackdriver Debug interface in Google Console, and am unable to set any breakpoints.
The warning reads:
Stackdriver Debugger is not set up for the python runtime on GAE Flex
Screenshot of warning
Any thoughts on what I'm doing wrong?
I've:
Enabled the Stackdriver Debugger API (as mentioned here)
Imported and initialised the Debugger (following the instructions here)
Included google-python-cloud-debugger in requirements.txt
main.py (the app entry point defined in app.yaml)
from werkzeug.serving import run_simple
from werkzeug.wsgi import DispatcherMiddleware
from wsgi import api, frontend, manage
try:
import googleclouddebugger
googleclouddebugger.AttachDebugger()
except ImportError:
pass
app = DispatcherMiddleware(frontend.create_app(), {
'/api': api.create_app(),
'/manage': manage.create_app()
})
if __name__ == '__main__':
run_simple('0.0.0.0', 5000, app, use_reloader=True, use_debugger=True)
app.yaml
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 2
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
env_variables:
SQLALCHEMY_DATABASE_URI: "postgresql+psycopg2://myusername:mypassword!#/instancename?host=/cloudsql/instancename"
beta_settings:
cloud_sql_instances: "instancename"
Update 1
After a comment and noticing a urllib import error, I wondered if the wsgi nature of my application was causing issues. I went back to the documentation, saw a note about Django framework doing something similar and changed the following:
googleclouddebugger.AttachDebugger()
to
googleclouddebugger.enable()
This got rid of the urllib import error, but hasn't resolved the overall issue.
I don't see anything wrong with this configuration, but here are some suggestions:
Once you open the GCP Console Debug page, please make sure you selected the correct version of the app from the top dropdown menu before starting the debugging session. It may be pointing to an old app version that is not debuggable, or the page and the dropdown might need a refresh to display the latest version.
Similarly, please check if your Stackdriver stderr logs contains a line that like this:
2017-12-24 15:14:14.000 PST I1224 23:14:14.600462 12 gcp_hub_client.py:335] Debuggee registered successfully, ID: gcp:1025611681465:7144dac417e43025
This means the debugger is actually setup/working properly, and most likely indicates that the UI is requires a refresh or selecting the right app version from the dropdown.
Make sure you have the google-python-cloud-debugger in the requirements.txt file.
Consider adding a simple print statement in the try/except block for the debugger, as folllows:
...
except ImportError as e:
print 'Failed to import Google Cloud Debugger: %s' % str(e)
pass
Then, check the stderr logs on Stackdriver Logging to see if this printed any exceptions.
I don't know about werkzeug, but you might want to try disabling the default debugger there by passing use_debugger=False, just to avoid possible conflicts.
Try deploying a simpler flask app (e.g., start with the hello world for Flex from here but modify it to use python_version: 2) and see if the debugger works for you there.
Following a bunch of tutorials, I wrote my first django app and then decided to deploy it on a linode server. Following their tutorial, I only got so far. The best I could tell, that was based on an earlier version of Ubunto and I tried some other things, including the uwsgi quickstart tutorial.
I got far enough that the command 'python manage.py runserver' would serve up my site and wanted to move on to using uwsgi to do the same.
It seems as though there is some environment variable missing. When I try to start uwsgi from the command line with:
uwsgi --http :8000 --module dynamicefl.wsgi
I get the following:
*** Starting uWSGI 2.0.15 (64bit) on [Fri Aug 11 19:37:04 2017] ***
compiled with version: 6.3.0 20170406 on 10 August 2017 23:41:13
os: Linux-4.9.36-x86_64-linode85 #1 SMP Thu Jul 6 15:31:23 UTC 2017
nodename: roosevelt
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/django/worksheets/dynamic-efl
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 3941
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :8000 fd 4
spawned uWSGI http 1 (pid: 5998)
uwsgi socket 0 bound to TCP address 127.0.0.1:43637 (port auto-assigned) fd 3
Python version: 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170118]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x562bdad6fda0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72760 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
File "./dynamicefl/wsgi.py", line 12, in <module>
from django.core.wsgi import get_wsgi_application
ImportError: No module named django.core.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 5997, cores: 1)
A lot of the tutorials emphasize putting the path to the virtual environment in the .ini file, and I did (double checking with 'env' that I had the virtual environment right.) Here is the .ini file I'm working with
[uwsgi]
home = /home/django/Env/worksheets
chdir = /home/django/worksheets/dynamic-efl/dynamicefl
module = dynamicefl.wsgi::application
uid = www-data
gid = www-data
socket = localhost:9191
chmod-socket = 644
chown-socket = www-data
processes = 4
threads = 2
logto = /var/log/uwsgi.log
I have experimented with changing the chdir command some, as I'm not sure exactly where it should be pointing, but regarless of where it points that results in a similar line in uwsgi.log:
-- no python application found, check your startup logs for errors ---
(Fun fact: I haven't been able to find reference to uwsgi startup logs)
If I go to the site in my web browser, I get an 'Internal Server Error' message.
I don't think that anything is getting as far an nginx, but I can include the .conf file and error logs, too, if that's helpful.
I know that there are a log of similar questions, but I've looked at them and they're not helping me. (Which is not to say that they don't contain the answer.)
The django docs helped me out here. It turns out that my reference to the module was apparently wrong. I had a colon too many in the reference to the module. The corrected .ini reads:
module = dynamicefl.wsgi:application
What helped was resolving to battle it out until I could serve the site purely from uwsgi, so that I could stop restarting services.
I am trying to setup graphite with nginx. Because of this I need to run it using a configuration or ini file in /etc/uwsgi, but I am unable to get the application to start correctly.
Using, the command,
uwsgi --http :9090 --wsgi-file /opt/graphite/conf/graphite.py
Graphite starts and runs fine, I can navigate it and look at stats.
I proceeded to create an ini file, with the contents:
[uwsgi]
processes = 2
socket = localhost:8081
gid = nginx
uid = nginx
chdir = /opt/graphite/conf
uswsgi-file = graphite.py
running the ini file I see:
mapped 145536 bytes (142 KB) for 2 cores
*** Operational MODE: preforking ***
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
I can only guess it something is misconfigured here in the ini file but I am not seeing what it is.
Any help is appreciated!
There are some differences between your command line and ini file:
you're using socket instead of http inside ini. That means, uWSGI server will talk using uwsgi protocol instead of http. If you're using uwsgi_pass in nginx and trying to access your website from browser through that nginx, it's fine. But if you're trying to access uwsgi directly from browser, bypassing nginx, you won't succeed because browsers doesn't talk uwsgi.
You've put uswgi-file instead of wsgi-file into your config. That won't work at all and uwsgi won't be able to find your wsgi file.
And if you're chdiring into directory with your wsgi file, it is better to use:
module = wsgi
instead of wsgi-file.
I'm trying to run a Flask application using a ubuntu 12.04 server with nginx and uWSGI however after starting the server, specifying the wsgi.ini file and then starting nginx, I try to access the page but I get a 502 bad gateway error with nginx. I know that it is an issue with the flask application and uWSGI. I checked the uWSGI logs and found this:
*** Starting uWSGI 2.0.11.1 (64bit) on [Thu Sep 17 19:38:03 2015] ***
compiled with version: 4.8.4 on 17 September 2015 15:58:04
os: Linux-3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015
nodename: ip-172-31-30-153
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/ubuntu/myapp
writing pidfile to /home/ubuntu/myapp/myapp.pid
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 7862
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:5000 fd 3
Python version: 2.7.6 (default, Jun 22 2015, 18:01:27) [GCC 4.8.2]
Set PythonHome to /home/ubuntu/myapp
ImportError: No module named site
I found this question's top answer but saw received the same error. If it is something to do with PYTHONHOME, I'm following this guide and it states:
The $PYTHONPATH variable seems to get unset randomly, and causes
errors for later steps. To avoid this, add the following line to the
top of your ~/.bashrc file:
export
PYTHONPATH='/usr/lib/python2.7:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat'
So what could be the issue? Is it PYTHONHOME or another factor affecting it? Thanks.
Edit:
My uwsgi.ini:
[uwsgi]
home = /home/ubuntu/myapp
wsgi-file = %(home)/run.py
socket = 127.0.0.1:5000
callable = app
module = app
pythonpath = %(home)
daemonize = %(home)/myapp.log
pidfile = %(home)/myapp.pid
To run it, I use uwsgi uwsgi.ini.
I am taking over a django project which another developer maintained. The service is run on an Ubuntu machine. ZEO is used for content caching. Ajax/Dajax is used for asynchronious content. Celery is used for task management and Django is used for the project itself.
The service is usually reached via a specific IP address which limits access to specific URLs. http://my_server_ip. Without knowingly changing anything, this started to not work. Instead of taking me to the splash page, entering the IP would hang, unsuccesfully connecting. I don't get a 404, 500 or some other error, it just sits and continually tries to load as if waiting to connect or to receive content.
I attempted to restart the service in the hopes that this would solve the problem, it did not. I performed a system reboot and followed the following commands, as per the prior developer's documentation, to reboot the server.
From within the django project:
runzeo -a localhost:8090 -f /path/to/operations_cache.fs
su project_owner
python manage.py celery worker -n multiprocessing_worker --loglevel=debug -Q multiprocessing_queue
python manage.py celery worker --concurrency=500 --pool=eventlet --loglevel=debug -Q celery -n eventlet_worker
The two celery commands had to be run as the owner of the project directory.
Finally I ran sudo service apache2 restart. Upon completion, I tried to navigate to the webpage but received the same response: hung on connecting. The trac pages do work at http://my_server_ip/trac.
The following is all I have found in the apache log files.
error.log
[Fri Feb 06 16:01:11 2015] [error] /usr/lib/python2.7/dist-packages/configobj.py:145: DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.
[Fri Feb 06 16:01:11 2015] [error] import compiler
[Fri Feb 06 16:01:11 2015] [error]
access.log
my.ip - user [06/Feb/2015:15:55:40 -0500] "GET / HTTP/1.1" 500 632 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0 Gecko/20100101 Firefox/35.0"
I have tried looking into the django logs, but nothing appears there. Perhaps I am not finding the correct logs.
As a good start, where can I find out what the system is hanging on. How do I determine if it is an issue with django, apache, the interaction between the two, etc. This would help me zero in on specifically what is happening.
Edit
I was able to resolve my problem, though I cannot say for sure what resolved the issue. I suspect that the issue had to do with permissions on the static files folder. I serve my content through the www-user. After going through the steps described above I then ran yes yes | python manage.py collectstatic as the user www-data. The server was able restart and I was able to access the trac content as well as the django content.
I am hesitant to post this as an answer because I do not know with certainty whether my solution described here is the step which solved the problem.