We Keep Coding

How to start a PM2 process using a virtual python environment with variables and disable pm2 logging

I am trying to start a python script in pm2 with a a variable.
Without a variable, I would run:
pm2 start --name myapp /home/user/myapp/start.py --interpreter ~/myapp_venv/bin/python3
The python command I would like to run is:
python3 /home/user/myapp/start.py -cf configs/myapp2.ini
If I activate the virtual environment, I can start the app just fine.
I am looking for the PM2 start command to run this in PM2.
Also, I would like to stop the pm2 logs from generating and writing as I log in my own app, so they are useless writes for me.
Something like the below I thought would work adding into the PM2 start script.
-o "/dev/null" -e "/dev/null"
If anyone would be able to advise on the PM2 start command to run this app with the viable in PM2, I would be very grateful.

I have since found the answer to my own question.
Run the below command from the myapp directory:
pm2 start "~/myapp_venv/bin/python3 start.py -cf configs/myapp2.ini" --name myapp2

Start django app as service

I want to create service that will be start with ubuntu and will have ability to use django models etc..
This service will create thread util.WorkerThread and wait some data in main.py
if __name__ == '__main__':
bot.polling(none_stop=True)
How I can to do this. I just don't know what I need to looking for.
If you also can say how I can create ubuntu autostart service with script like that, please tell me )
P.S. all django project run via uwsgi in emperor mode.

The easiest way in my opinion is create a script and run on crontab.
First of all create a script to start your django app.
#!/bin/bash
cd /path/to your/virtual environment #path to your virtual environment
. bin/activate #Activate your virtual environment
cd /path/to your/project directory #After that go to your project directory
python manage.py runserver #run django server
Save the script and open crontab with the command:
crontab -e
Now edit the crontab file and write on the last line:
#reboot path/to/your/script.sh
This way is not the best but the easiest, if you are not comfortable with Linux startup service creation.
I hope this help you :)

Take a look at supervisord. It is much easier than daemonizing python script.
Config it something like this:
[program:watcher]
command = /usr/bin/python /path/to/main.py
stdout_logfile = /var/log/main-stdout.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 5
stderr_logfile = /var/log/main-stderr.log
stderr_logfile_maxbytes = 10MB
stderr_logfile_backups = 5

Ok, that is answer - https://www.raspberrypi-spy.co.uk/2015/10/how-to-autorun-a-python-script-on-boot-using-systemd/
In new versions ubuntu services .conf in /etc/init fail with error Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
But services works using systemd

Why does uWSGI fail to start in Docker?

I am relatively new to using uWSGI to serve Python applications and I am attempting to start a uWSGI process in emperor mode with a vassal, but every time I try to start uWSGI inside of Docker with the following command (as root):
# /usr/local/bin/uwsgi --ini /etc/uwsgi/emperor.ini
What I get as a response is:
[uWSGI] getting INI configuration from /etc/uwsgi/emperor.ini
2.0.13.1
The emperor.ini configuration file looks like:
# files/etc/uwsgi/emperor.ini
[uwsgi]
emperor = /etc/uwsgi/apps-enabled
die-on-term = true
log-date = true
While the only vassal's configuration looks like:
# files/etc/uwsgi/apps-enabled/application.ini
[uwsgi]
app_dir = /var/www/server
plugin = python
master = true
callable = app
chdir = %(app_dir)
mount = /=%(app_dir)/start.py
protocol = uwsgi
socket = :8079
uid = www-data
gid = www-data
buffer-size = 32768
enable-threads = true
single-interpreter = true
processes = 1
stats = 127.0.0.1:1717
(NB: The filenames above are given in terms of where they live relative to the Dockerfile which will then copy them to the correct locations, basically removing the prefix files)
Currently the uWSGI Docker image I'm using is built off of an ubuntu:trusty base image (though I've tried ubuntu:latest and alpine:latest and encountered the same problem), and although I am attempting to launch the uWSGI process with supervisor, as stated previously, it also fails when run directly from the command line. In the Docker image I'm installing uWSGI using pip but have also tried using apt-get with the same result.
I should also mention that I've tried different versions of uWSGI 2.0.13.1 and 1.9.something with the same result, if that helps.
# Dockerfile
FROM ubuntu:trusty
MAINTAINER Sean Quinn "me#mail.com"
RUN apt-get update \
&& apt-get install -y \
ack-grep git nano \
supervisor \
build-essential gcc python python-dev python-pip
RUN sed -i 's/^\(\[supervisord\]\)$/\1\nnodaemon=true/' /etc/supervisor/supervisord.conf \
&& sed -i 's/^\(\[supervisord\]\)$/\1\nloglevel=debug/' /etc/supervisor/supervisord.conf \
&& sed -i 's/^\(files = .*\)$/;\1/' /etc/supervisor/supervisord.conf \
&& sed -i 's/^\(\[include\]\)$/\1\nfiles = \/etc\/supervisor\/conf.d\/*.conf/' /etc/supervisor/supervisord.conf
ENV UWSGI_VERSION 2.0.13.1
RUN pip install uwsgi==${UWSGI_VERSION}
RUN mkdir -p /etc/uwsgi \
&& mkdir -p /etc/uwsgi/apps-available \
&& mkdir -p /etc/uwsgi/apps-enabled \
&& mkdir -p /var/log/uwsgi
COPY files/etc/supervisor/conf.d/uwsgi.conf /etc/supervisor/conf.d/uwsgi.conf
COPY files/etc/uwsgi/emperor.ini /etc/uwsgi/emperor.ini
VOLUME /etc/uwsgi/apps-enabled
VOLUME /var/www
ENTRYPOINT ["/usr/bin/supervisord"]
CMD ["-c", "/etc/supervisor/supervisord.conf"]
As mentioned, the supervisord process attempts to launch the uWSGI process using the following supervisor configuration.
# files/etc/supervisor/conf.d/uwsgi.conf
[program:uwsgi]
command=/usr/local/bin/uwsgi --ini /etc/uwsgi/emperor.ini
user=root
The application Python files are mounted in a subdirectory of /var/www and the application uWSGI configuration is mounted into /etc/uwsgi/apps-enabled.
The bizarre thing is, if I install supervisor and uWSGI on a new Ubuntu VM (outside of Docker) with all of the configuration and files in place I can see uWSGI properly process the emperor.ini and read the vassal .ini files. I haven't yet attempted to add nginx into the equation because I want to make sure uWSGI is starting and reading configuration files correctly first and foremost.
Is there any way to increase the logging or ascertain why I'm only seeing what appears to be the version number of the uWSGI binary? It's like the uWSGI process is completely ignoring command line options. I feel like I'm missing something that should be obvious.
Thanks in advance for any help anyone can give!

tl;dr don't use UWSGI_VERSION as an environment variable, apparently it forces uWSGI to only print the version number instead of start?
I believe I solved my own issue!
After experimenting with other uWSGI images on Docker's hub, I found that they were also running into the same issue so I began to look further into possible configuration issues. I tried changing permissions among other things.
I noticed however that when I used jpetazzo/nsenter to enter the running container that I saw uWSGI start (rather than simply output the uWSGI version information as highlighted above). When entering using docker exec, uWSGI would only print the version information. After playing around a bit more, I discovered that issuing the command su - from within the container launched using docker exec I again saw uWSGI start.
After some inspection, I discovered several differences in the environment variables between the root user in one shell vs. another. It led me to the UWSGI_VERSION environment variable, which appears to have been the culprit because removing UWSGI_VERSION allowed uWSGI to start.
I modified my Dockerfile to use UWSGI_PIP_VERSION instead as the environment variable to indicate the version of uWSGI to install, which seems to be a safe alternative to UWSGI_VERSION. YMMV.

How can I run uWsgi as a service in CentOs?

I am in a hurry, I can find out how to do this but I need some help to achieve this without loosing too much time.
Currently what I do to run a uWsgi instance along with my ini file is just:
uwsgi --ini /home/myonlinesite/uwsgi.ini --pidfile /var/run/uwsgi_serv.pid
and then to stop:
uwsgi --stop /var/run/uwsgi_serv.pid.
By the way, I have this code inside a uwsgi init file in my /etc/init.d/uwsgi.
so when I run /etc/init.d/uwsgi start it executes the ini config file and when I execute /etc/init.d/uwsgi stop it stops the uwsgi process id.
The problem is that when I start the uWsgi service it runs normally and logs every http request, any debug print and so on, but when I close putty which is where I run my Vps it kills all uWsgi process and quits the site from being displayed.
I do not know if I have to touch the pid file only, or what do I need to do leave the uWsgi process executing and I can close putty.
Thanks in advance.

If you are setting the parameters in the command line, add the flag -d file.log to your command (-d stands for daemonize):
uwsgi --ini /home/myonlinesite/uwsgi.ini --pidfile /var/run/uwsgi_serv.pid -d file.log
If you are setting the parameters in a config file, add the following line in your config:
daemonize = /absolute/path/to/file.log
In both cases, uWsgi will run in the background and log everything in file.log. Given these options, there is no need using nohup et al.

Using nohup to start the uWsgi process should solve your problem of the process stopping when you log out.
A tutorial

Be sure to add
daemon = logfile
to your config

Deploying CherryPy (daemon)

I've followed the basic CherryPy tutorial (http://www.cherrypy.org/wiki/CherryPyTutorial). One thing not discussed is deployment.
How can I launch a CherryPy app as a daemon and "forget about it"? What happens if the server reboots?
Is there a standard recipe? Maybe something that will create a service script (/etc/init.d/cherrypy...)
Thanks!

Daemonizer can be pretty simple to use:
# this works for cherrypy 3.1.2 on Ubuntu 10.04
from cherrypy.process.plugins import Daemonizer
# before mounting anything
Daemonizer(cherrypy.engine).subscribe()
cherrypy.tree.mount(MyDaemonApp, "/")
cherrypy.engine.start()
cherrypy.engine.block()
There is a decent HOWTO for SysV style here.
To summarize:
Create a file named for your application in /etc/init.d that calls /bin/sh
sudo vim /etc/init.d/MyDaemonApp
#!/bin/sh
echo "Invoking MyDaemonApp";
/path/to/MyDaemonApp
echo "Started MyDaemonApp. Tremble, Ye Mighty."
Make it executable
sudo chmod +x /etc/init.d/MyDaemonApp
Run update-rc.d to create our proper links in the proper runtime dir.
sudo update-rc.d MyDaemonApp defaults 80
sudo /etc/init.d/MyDaemonApp

There is a Daemonizer plugin for CherryPy included by default which is useful for getting it to start but by far the easiest way for simple cases is to use the cherryd script:
> cherryd -h
Usage: cherryd [options]
Options:
-h, --help show this help message and exit
-c CONFIG, --config=CONFIG
specify config file(s)
-d run the server as a daemon
-e ENVIRONMENT, --environment=ENVIRONMENT
apply the given config environment
-f start a fastcgi server instead of the default HTTP
server
-s start a scgi server instead of the default HTTP server
-i IMPORTS, --import=IMPORTS
specify modules to import
-p PIDFILE, --pidfile=PIDFILE
store the process id in the given file
As far as an init.d script goes I think there are examples that can be Googled.
And the cherryd is found in your:
virtualenv/lib/python2.7/site-packages/cherrypy/cherryd
or in: https://bitbucket.org/cherrypy/cherrypy/src/default/cherrypy/cherryd

I wrote a tutorial/project skeleton, cherrypy-webapp-skeleton, which goal was to fill the gaps for deploying a real-world CherryPy application on Debian* for a web-developer. It features extended cherryd for daemon privilege drop. There's also a number of important script and config files for init.d, nginx, monit, logrotate. The tutorial part describes how to put things together and eventually forget about it. The skeleton part proposes a way of possible arrangement of CherryPy webapp project assets.
* It was written for Squeeze but practically it should be same for Wheezy.

Info on Daemonizer options
When using Daemonizer, the docs don't state the options, e.g. how to redirect stdout or stderr. From the source of the Daemonizer class you can find the options. As a reference take this example from my project:
# run server as a daemon
d = Daemonizer(cherrypy.engine,
stdout='/home/pi/Gate/log/gate_access.log',
stderr='/home/pi/Gate/log/gate_error.log')
d.subscribe()