We Keep Coding

How to keep django_q cluster running in linux server?

I am using django_q for some scheduling and automations in my django project.
I successfully configured all the needed stuff but to get django_q running I have to type in the server command line 'python manage.py qcluster' and after i close the shell session id doesn't work anymore.
In the django_q official documentation it says that there is no need for a supervisor, but this is not running.
Any ideas?

There are a few approaches you can use.
You could install the screen program to create a terminal session which stays around after logout. See also: https://superuser.com/questions/451057/keep-processes-alive-after-ssh-logout
You could use systemd to automatically start your qcluster. This has the advantage that it will start qcluster again if your server is rebooted. You'll want to write a service unit file with Type=simple. Here's a list of resources.
Here's an example unit file. (You may need to adapt this somewhat.)
[Unit]
Description=qcluster daemon
[Service]
User=<django user>
Group=<django group>
WorkingDirectory=<your working dir>
Environment=PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/ bin/
ExecStart=python manage.py qcluster
Restart=always
[Install]
WantedBy=multi-user.target

Crontab not starting python program

I created a python program, "test.py" and have saved it under /home/pi/. When I go to run it in the terminal using "python3 /home/pi/test.py" it runs properly and speaks "hello world". The code is shown below.
import os
import alsaaudio
m = alsaaudio.Mixer()
current_volume = m.getvolume()
m.setvolume(35)
os.system("espeak 'Hello World!'")
I want this program to start whenever my raspberry pi starts up. I tried to add this line in crontab but my raspberry pi doesn't execute the command. Does anyone know why my program won't execute?
#reboot python3 /home/pi/test.py
Here is an image of the syslog

can you try adding the full path to python3:
#reboot /usr/bin/python3 /home/pi/test.py
Also, regarding wanting to run the code on when the device boots - you can run your code as a service.
To do so create a .service file under /etc/systemd/system (for example my-code.service)
Enter the following inside the file
[Unit]
Description=My python service
After=network.target
[Service]
ExecStart=/usr/bin/python3 -u test.py
WorkingDirectory=/home/pi
[Install]
WantedBy=multi-user.target
Finally enable the service (in order for it to run on boot)
sudo systemctl enable my-code
If you want to run it independently you can also run
sudo systemctl start my-code

launching python script in terminal at boot and keeping open if/when exception

writing a data logging program that is intended to run when raspberry boots. I'm using lxsessions autostart to launch a shell script that has the command to launch my python program (my python script requires sudo)
while I continue to debug I would like the terminal window to stay open if/when it encounters an error.
I had done this successfully once before but lost my work.
my autostart file is:
#!/bin/bash
#lxpanel --profile LXDE-pi
#pcmanfm --desktop --profile LXDE-pi
#lxterminal -e sudo sh /home/pi/launcher.sh
#xscreensaver -no-splash
my script file is:
#!/bin/sh
echo Script is running
sudo /usr/bin/python3 /home/pi/hms/hms5-1.py
I thought something like this (in the autstart file) would work, but no:
#lxterminal -e -hold sudo sh /home/pi/launcher.sh
a simple internet search spit out of examples on how to execute command at boot, even launching scripts but nothing has helped so far. Thank you in advance.....

So I rebuilt my Raspberry Pi and had to go through this again. So after I was able to get it to work once, I followed my instructions from before, edited them to be clearer the posted here. NOTE - I think the mistake I made before was using sudo (sudo nano) when I should have just used nano....
Also note the python program I am launching is /home/pi/hms/hms2-v2.py
*** setting this up is a 4 step process ***
YOU MUST HAVE XTERM
STEP 1 - INSTALL XTERM:
sudo apt-get install xterm
STEP 2
read https://www.raspberrypi.org/forums/viewtopic.php?t=227191
FIRST CREATE autostart here: /home/pi/.config/lxsession/LXDE-pi/autostart
NOTE THE FOLDERS BELOW /home/pi/.config/ MAY NOT EXIST, IF NOT CREATE THEM EXACTLY AS ABOVE. NOTE: the directory must be LXDE-pi NOT LXDE
Then edit the autostart file
by using nano ~/.config/lxsession/LXDE-pi/autostart
NOTE: DO NOT use sudo in the above command
put the following in the file:
#!/bin/bash
#lxpanel --profile LXDE-pi
#pcmanfm --desktop --profile LXDE-pi
sh /home/pi/launcher.sh
#xscreensaver -no-splash
Step 3
create the script (.sh) file: launcher.sh in the directory /home/pi
include the following in the file launcher.sh:
#!/bin/sh
echo starting script
xterm -T "HMS" -geometry 100x70+10+35 -hold -e sudo /usr/bin/python3 /home/pi/hms/hms2-v2.py
Step 4
make the .sh file executable with: sudo chmod +x launcher.sh

Kafka Producer and Consumer Scripts to Run automatically

I have a Django project and I am using pykafka. I have created two files named producer.py and consumer.py inside the project. I have to change directory into the folder where these are present and then separately run python producer.py and consumer.py from the terminal. Everything works great.
I deployed my project online and the web-app is running so I want to run the producer and consumer automatically in the background. How do i do that?
EDIT 1: On my production server I did nohup python name_of_python_script.py & to execute it in the background. This works for the time being but is it a good solution?

You can create a systemd service MyKafkaConsumer.service under /etc/systemd/system with the following content:
[Unit]
Description=A Kafka Consumer written in Python
After=network.target # include any other pre-requisites
[Service]
Type=simple
User=your_user
Group=your_user_group
WorkingDirectory=/path/to/your/consumer
ExecStart=python consumer.py
TimeoutStopSec=180
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
In order to start the service (and configure it in order to run on boot) you should run
systemctl enable MyKafkaConsumer.service
systemctl start MyKafkaConsumer.service
To check its status:
systemctl status MyKafkaConsumer
And to see the logs:
journactl -u MyKafkaConsumer -f
(or if you want to see the last 100 lines)
journalctl -u MyKafkaConsumer -n 100
You'd need to create a similar service for your producer too.
There are a lot of options for systemd services. You can refer to this article if you need any further clarifications. It shouldn't be hard to find guides and additional material online though.

Start django app as service

I want to create service that will be start with ubuntu and will have ability to use django models etc..
This service will create thread util.WorkerThread and wait some data in main.py
if __name__ == '__main__':
bot.polling(none_stop=True)
How I can to do this. I just don't know what I need to looking for.
If you also can say how I can create ubuntu autostart service with script like that, please tell me )
P.S. all django project run via uwsgi in emperor mode.

The easiest way in my opinion is create a script and run on crontab.
First of all create a script to start your django app.
#!/bin/bash
cd /path/to your/virtual environment #path to your virtual environment
. bin/activate #Activate your virtual environment
cd /path/to your/project directory #After that go to your project directory
python manage.py runserver #run django server
Save the script and open crontab with the command:
crontab -e
Now edit the crontab file and write on the last line:
#reboot path/to/your/script.sh
This way is not the best but the easiest, if you are not comfortable with Linux startup service creation.
I hope this help you :)

Take a look at supervisord. It is much easier than daemonizing python script.
Config it something like this:
[program:watcher]
command = /usr/bin/python /path/to/main.py
stdout_logfile = /var/log/main-stdout.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 5
stderr_logfile = /var/log/main-stderr.log
stderr_logfile_maxbytes = 10MB
stderr_logfile_backups = 5

Ok, that is answer - https://www.raspberrypi-spy.co.uk/2015/10/how-to-autorun-a-python-script-on-boot-using-systemd/
In new versions ubuntu services .conf in /etc/init fail with error Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
But services works using systemd