We Keep Coding

uWSGI stats using uwsgitop and socket showing JSONDecodeError

I am not getting uwsgi stats using uwsgitop and socket. I have put uwsgi configuration for the stats with socket and when I tried to get the stat using the command:
uwsgitop /var/www/uwsgi/proj.socket
It's throwing the error
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I am using uwsgi version 2.0.17.1.
Here is my uwsgi ini file
[uwsgi]
# Multi Thread Support
enable-threads = true
# Django-related settings
# the base directory (full path)
chdir = /home/user/base-dir/proj-path/
# Django's wsgi file
module = proj.wsgi
# the virtualenv (full path)
home = /home/user/base-path/
# process-related settings
# master
master = true
# maximum number of worker processes
processes = 10
socket = /var/www/uwsgi/proj.socket
# ... with appropriate permissions - may be needed
chmod-socket = 666
# clear environment on exit
vacuum = true
daemonize = /var/www/uwsgi/uwsgi.log
pidfile = /var/www/uwsgi/uwsgi_hub.pid
logto = /var/log/proj_uwsgi%n.log
uid = user
gid = user
http-auto-gzip = true
memory-report = True
py-tracebacker=/var/www/uwsgi/proj.socket
--stats /var/www/uwsgi/proj.socket

I think that you should have something like this in your config file:
socket = /var/www/uwsgi/proj.socket
stats = /var/www/uwsgi/stats.socket
And run uwsgitop on the stats socket, like so:
uwsgitop /var/www/uwsgi/stats.socket

Django passenger_wsgi.py issue

I run Django on an A2 Hosting Linux shared hosting account. Previously, I was using an old version of Django and Python. I was also using FCGI. I recently changed to a new server and am now using Django 2.1.2, Python 3.6.0. and Phusion Passenger.
I can't get the passenger_wsgi.py to load my website and I can't find the error log to try and resolve the problem.
The .htaccess file is in the public_html directory.
My app files are stored in:
/home/username/example.com/
The folder/file structure in the example.com directory is:
-media
-mysite
--- init.py
--- settings.py
--- urls.py
--- wsgi.py
-public
-tmp
-manage.py
-passenger_wsgi.py
#
The contents of the passenger_wsgi.py file:
import os
import sys
import mysite.wsgi
application = mysite.wsgi.application
cwd = os.getcwd()
sys.path.append(cwd)
#sys.path.append(os.getcwd())
sys.path.append(cwd + '/mysite')
SCRIPT_NAME = os.getcwd()
class PassengerPathInfoFix(object):
"""
Sets PATH_INFO from REQUEST_URI since Passenger doesn't provide it.
"""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = PassengerPathInfoFix(application)
The site produces an error (image below). I've been trying to fix it all day by tinkering with the passenger_wsgi.py file.
I can't find the Passenger error log file. I don't have any experience with Passenger. I've run out of ideas of ways to try and resolve the issue.
When I run 'python passenger_wsgi.py' in SSH, no comments are produced.
If anyone could help me to either:
(1) Identify what is wrong with the passenger_wsgi.py content or
(2) How I can find the Passenger error log on a shared Linux hosting account.

Not sure it'll be possible to diagnose anything without logs, but I've had the similar issues on shared A2 Hosting finding logs. I was able to find some Passenger logs in the "Errors" section of cPanel.
On the cPanel home screen, scroll down to the "Metric" subheader. There will be a link for "Errors" in this section. You should be able to find critical Passenger logs there.

This structure of passenger_wsgi.py works for me on A2 Hosting:
import os
import sys
import MyApp.wsgi
SCRIPT_NAME = '/home/user/domain'
class PassengerPathInfoFix(object):
""" Sets PATH_INFO from REQUEST_URI because Passenger doesn't provide it. """
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = MyApp.wsgi.application
application = PassengerPathInfoFix(application)

Why are my Django / uWSGI vassal logs empty?

I am running my Django site as a vassal of UWSGI emperor. I have created /etc/uwsgi-emperor/vassals/mysite.ini as follows:
[uwsgi]
socket = /var/opt/mysite/uwsgi.sock
chmod-socket = 775
chdir = /opt/mysite
master = true
virtualenv = /opt/mysite_virtualenv
env = DJANGO_SETTINGS_MODULE=mysite.settings
module = mysite.wsgi:application
uid = www-data
gid = www-data
processes = 1
threads = 1
plugins = python3,logfile
logger = file:/var/log/uwsgi/app/mysite.log
vacuum = true
But /var/log/uwsgi/app/mysite.log does not get created. If I touch it, it remains empty. This occurs even after I trigger 500-style errors in the application.
Why aren't my logs being written?

The vassal does not have permission to write to the file (or create the file in the first place). You should
cd /var/log/uwsgi/app
touch mysite.log # create the file
chown www-data:www-data mysite.log # give the vassal permission
(where www-data:www-data matches the uid and gid values in your ini file).
Logs will start appearing shortly.

Pyramid Project: "ValueError: Variable assignment..." when trying to pserve

I get the following error when trying to run a Pyramid project. As far as I'm aware, this appeared overnight, and I've no idea how to begin to debug this:
C:\mas\mas>..\Scripts\pserve.exe serve development.ini --reload
Starting subprocess with file monitor
Traceback (most recent call last):
File "C:\mas\Scripts\pserve-script.py", line 8, in <module>
load_entry_point('pyramid==1.3.2', 'console_scripts', 'pserve')()
File "C:\mas\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\scripts\pserve.
py", line 47, in main
return command.run()
File "C:\mas\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\scripts\pserve.
py", line 221, in run
vars = self.parse_vars(restvars)
File "C:\mas\lib\site-packages\pyramid-1.3.2-py2.7.egg\pyramid\scripts\pserve.
py", line 330, in parse_vars
% arg)
ValueError: Variable assignment 'development.ini' invalid (no "=")
What is the problem, or how a should I go about determining it? Sorry my question is rather vague, but if I had any more an idea of what I was asking I may have found the answer on Google :).
Line 328 - 330:
raise ValueError(
'Variable assignment %r invalid (no "=")'
% arg)
development.ini
[app:main]
use = egg:mas
pyramid.reload_templates = true
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
pyramid.includes = pyramid_debugtoolbar
pyramid_tm
pyramid_beaker
sqlalchemy.url = sqlite:///%(here)s/mas.db
# Cache settings
cache.regions = default_term, second, short_term, long_term
cache.type = memory
cache.second.expire = 1
cache.short_term.expire = 60
cache.default_term.expire = 300
cache.long_term.expire = 3600
# Beaker sessions
#session.type = file
#session.data_dir = %(here)s/data/sessions/data
#session.lock_dir = %(here)s/data/sessions/lock
session.type = memory
session.key = akhet_demo
session.secret = 0cb243f53ad865a0f70099c0414ffe9cfcfe03ac
[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 6543
# Begin logging configuration
[loggers]
keys = root, mas, sqlalchemy
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = INFO
handlers = console
[logger_mas]
level = DEBUG
handlers =
qualname = mas
[logger_sqlalchemy]
level = INFO
handlers =
qualname = sqlalchemy.engine
# "level = INFO" logs SQL queries.
# "level = DEBUG" logs SQL queries and results.
# "level = WARN" logs neither. (Recommended for production systems.)
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s
# End logging configuration

Ich, I'e worked it out, I was looking for the problem in the wrong place. the problem is this command:
..\Scripts\pserve.exe serve development.ini --reload
Should be this command:
..\Scripts\pserve.exe development.ini --reload
I have no idea how or when my batch file got changed, but if someone has a similar problem hopefully this will help.

It appears that you have the order of the arguments backwards - it should be:
..\Scripts\pserve.exe development.ini --reload

CherryPy Logging: How do I configure and use the global and application level loggers?

I'm having trouble with logging. I'm running CherryPy 3.2 and I've been reading through the docs here, but haven't found any examples of how to configure a local log file for output and how to write to it.
Raspberry.py:
import socket
import sys
import cherrypy
app_roots = {
# Sean's laptop dev environment.
"mylaptop": "/home/src/local-mydomain.com/py",
# Hosted dev environment.
"mydomain.com" : "/home/dev/src/py"
}
hostname = socket.gethostname()
CherryPyLog = cherrypy.tree.mount().log
if hostname not in app_roots:
CherryPyLog("The following hostname does not have an app_root entry in raspberry.py. Exiting early.")
sys.exit()
sys.stdout = sys.stderr
sys.path.append(app_roots[hostname])
import os
os.chdir(app_root)
# Setup for raspberry application logging.
import datetime
today = datetime.datetime.today()
log.access_file = "{0}/{1}.raspberry.access.log".format(app_roots[hostname],today.strftime("%Y%m%d-%H%M"))
log.error_file = "{0}/{1}.raspberry.error.log".format(app_roots[hostname],today.strftime("%Y%m%d-%H%M"))
#Testing logger
log("{0} -- Logger configured".format(today.strftime("%Y%m%d-%H%M%S")))
import atexit
cherrypy.config.update({'environment': 'embedded'})
if cherrypy.__version__.startswith('3.0') and cherrypy.engine.state == 0:
cherrypy.engine.start(blocking = False)
atexit.register(cherrypy.engine.stop)
from web.controllers.root import RaspberryRequestHandler
application = cherrypy.Application(RaspberryRequestHandler(), script_name = None, config = None)
UPDATE: Here's the code block that I ended up going with.
app_roots = {
# Sean's laptop dev environment.
"mylaptop": "/home/src/local-plottools.com/py",
# Hosted dev environment.
"myDomain" : "/home/dev/src/py"
}
import socket
hostname = socket.gethostname()
import cherrypy
import sys
if hostname not in app_roots:
cherrypy.log("The hostname {0} does not have an app_root entry in {1}. Exiting early.".format(hostname,__file__))
sys.exit()
sys.stdout = sys.stderr
sys.path.append(app_roots[hostname])
import os
os.chdir(app_roots[hostname])
from web.controllers.root import RaspberryRequestHandler
cherrypy.config.update({
'log.access_file': "{0}/cherrypy-access.log".format(app_roots[hostname]),
'log.error_file': "{0}/cherrypy.log".format(app_roots[hostname]),
"server.thread_pool" : 10
})
# special case, handling debug sessions when quickstart is needed.
if __name__ == "__main__":
cherrypy.config.update({
'log.screen': True,
"server.socket_port": 8000
})
cherrypy.quickstart(RaspberryRequestHandler())
sys.exit()
# This configuration is needed for running under mod_wsgi. See here: http://tools.cherrypy.org/wiki/ModWSGI
cherrypy.config.update({'environment': 'embedded'})
applicationLogName = "{0}/raspberry.log".format(app_roots[hostname])
from logging import handlers
applicationLogFileHandler = handlers.RotatingFileHandler(applicationLogName, 'a', 10000000, 1000)
import logging
applicationLogFileHandler.setLevel(logging.DEBUG)
from cherrypy import _cplogging
applicationLogFileHandler.setFormatter(_cplogging.logfmt)
cherrypy.log.error_log.addHandler(applicationLogFileHandler)
application = cherrypy.Application(RaspberryRequestHandler(), None)

Simplifying a bit:
import os
import socket
import sys
import cherrypy
app_roots = {
# Sean's laptop dev environment.
"mylaptop": "/home/src/local-mydomain.com/py",
# Hosted dev environment.
"mydomain.com" : "/home/dev/src/py"
}
hostname = socket.gethostname()
if hostname not in app_roots:
cherrypy.log("The hostname %r does not have an app_root entry in "
"raspberry.py. Exiting early." % hostname)
sys.exit()
sys.path.append(app_roots[hostname])
os.chdir(app_root)
cherrypy.config.update({
'environment': 'embedded',
'log.access_file': "{0}/raspberry.access.log".format(app_roots[hostname]),
'log.error_file': "{0}/raspberry.error.log".format(app_roots[hostname]),
})
from web.controllers.root import RaspberryRequestHandler
application = cherrypy.tree.mount(RaspberryRequestHandler(), '/')
# Insert log changes here
cherrypy.engine.start()
If you want different logs per day, use a RotatingFileHandler as described at http://www.cherrypy.org/wiki/Logging#CustomHandlers The important point I think you're missing is that you should muck about with app.log only after you've instantiated your app (e.g. via tree.mount(), as above), but before engine.start. That is, for the error log:
application = cherrypy.tree.mount(RaspberryRequestHandler(), '/')
log = application.log
log.error_file = ""
# Make a new RotatingFileHandler for the error log.
fname = "{0}/raspberry.error.log".format(app_roots[hostname])
h = handlers.RotatingFileHandler(fname, 'a', 10000000, 1000)
h.setLevel(DEBUG)
h.setFormatter(_cplogging.logfmt)
log.error_log.addHandler(h)
cherrypy.engine.start()
Hope that helps...