I would like to reference this question because I am certain that someone will flag this as a duplicate.
I am not looking for another reference to supervisord. I'm sure that it is great and all, but the node PM2 has the functionality that I require and is more straightforward to implement and test.
Manual Start
During prototyping, I created a virtual environment called 'p3env'. At the top of each script, I place a bash directive:
#!./py3env/bin/python
This allows me to execute each script in the directory using this particular python environment without having to activate it. It is very convenient and useful and the python script works well when I start it by hand.
I should be clear about what I mean when I say 'start it by hand'. My script is called 'strain_to_db.py'. When I start it by hand, I am on the shell via ssh:
./strain_to_db.py
This gets everything working that I need to have working.
PM2 Start
Moving from relative to absolute paths
To get pm2 working, I started with:
pm2 start ./strain_to_db.py
Specifying the Interpreter
Apparently pm2 ignores the directive at the top of the python script and attempts to execute using the global 'python'. No problem, I can specify the interpreter:
pm2 start ./strain_to_db.py --interpreter /home/ubuntu/db_if/p3env/bin/python
No dice. Again, maybe try more absolute paths:
pm2 start /home/ubuntu/db_if/strain_to_db.py --interpreter /home/ubuntu/db_if/p3env/bin/python
Running script as command-line option
Now I'm getting frustrated. I try another tactic. I attempt to run the python executable in the command line using:
/home/ubuntu/db_if/p3env/bin/python /home/ubuntu/db_if/strain_to_db.py
This works fine when pm2 isn't involved. When I try to pass this to pm2 using the 'command line argument' style:
pm2 start /home/ubuntu/db_if/p3env/bin/python -- /home/ubuntu/db_if/strain_to_db.py
Frustration
Same error. The error is always 'can't import pymysql', which is only installed on the virtual environment.
I am not sure where else to go with this. I have several scripts that I want to add to the pm2 execution monitor, but I can't seem to get one of them to start and run correctly.
After looking around a bit more, the question that I referenced at the top of the email had a clue in one of the answers, but not the answer.
When files end in '.py', pm2 calls 'python'... no matter what. I believe that there is a configuration file in pm2 that you could modify to change this behavior. I simply removed the '.py' from my script and specified the interpreter:
pm2 start ./strain_to_db --interpreter ./py3env/bin/python
Works perfectly. When I use pm2 to create a startup script, I will use absolute paths. Thanks for anyone who was looking, and I hope this helps someone in the future.
This Medium article solved this problem for me.
https://medium.com/#gokhang1327/deploying-flask-app-with-pm2-on-ubuntu-server-18-04-992dfd808079
Command for running a python script in virtual enviroment:
pm2 start app.py --name flask-app --interpreter=python3
--name it´s optional, that´s the name of process displayed in pm2 status
Result:
"new" is the name of my virtualenv environment.
A bit late to this question but for anyone coming here with a fresh pair of eyes, I've found that if you activate the virtual environment eg. source venv/Scripts/activate then start your script via pm2 eg. pm2 start main.py --name migration, it will automatically use the environment you've activated.
Related
I'm new to stackoverflow and really hope to get answers to my question about crontab.
I'm using MacbookAir in BigSur(don't know if it affects)And my crontab just couldn't run python scripts below.
f = open("test_crontab.txt",'w+')
f.write("Success")
f.close()
But I could run this script with Spyder.
I could edit crontab with crontab -e, and my code in vi crontab was
13 01 * * * /bin/zsh /Users/myusername/Documents/Lynn\'s\ Python\ work/shopee/test.py >>/Users/myusername/Desktop/out.log 2>>/Users/tzulingkuo/Desktop/err.log
But it failed to generate txt file.
The messages in err.log were like below:
/Users/*myusername*/Documents/Lynn's Python work/shopee/test.py:5: missing end of string
/Users/*myusername*/Documents/Lynn's Python work/shopee/test.py:6: unknown username ''
I've searched stackoverflow for two nights and couldn't find the solution. I major in finance and I have no friends studying computer science.
Could anyone help me 🥺
Any help is reaaaaally appreciated!
Since that's a python script, you need to run it in python, not /bin/zsh. zsh is trying to interpret it as a shell script, but the syntax is all wrong, so you get errors.
The simplest way to fix this to replace /bin/zsh in your crontab entry with /usr/bin/python (assuming you're using the built-in Python version 2; if you have installed Python version 3 and want to use that, run which python at the command line to find its path).
But it's generally better to add a shebang line to your python script, and let that control how it's run. Add this as the very first line of the script:
#!/usr/bin/python
(Again, if you want to use Python 3, replace the path part with the path for the Python 3 interpreter.) Then make the script executable with chmod +x /Users/myusername/Documents/Lynn\'s\ Python\ work/shopee/test.py, and just remove the /bin/zsh part from the crontab entry.
BTW, you're probably going to have trouble with paths. By default, cron jobs run with their working directory set to your home directory, so when the script opens test_crontab.txt, that's going to be interpreted as /Users/myusername/test_crontab.txt. If you want it down in the Documents/Lynn's Python work/shopee directory, you'll have to specify that explicitly.
You may also run into trouble with missing environment variables. If you're setting any environment variables in your shell initialization scripts that configure Python, add libraries, etc, those won't be set up in the cron job's environment.
I am trying to run python script with arguments (I have to pass a path to the config for the script to run well).
I am doing it way:
pm2 start script.py --name appname --interpreter python3 -- config/xyz.yml
I have been using pm2 for quite some time and I know that custom parameters are after --
But with running python scripts it obviously doesn't work this way.
I launch my scripts in the same manner without any problems. The only difference is that I pass the whole path to python interpreter (/usr/bin/python3) instead of just python3. Have you tried with the absolute path for config/xyz.yml? Relative path should also work though
pm2 start script.py --name appname --interpreter python3 -- config/xyz.yml
This is the right way to call the argument. It was my fault, I have made mistake in reading those argument. So it works.
I have a python script that starts a fullscreen animation when run - how do i set this to start at startup in ubuntu 16.04? Other answers to the "script on startup" problem don't work - because I'm assuming they're executed before the desktop environment has a chance to start up...
I have a script called "startup.sh" that reads as:
xrandr --output HDMI-0 -s 1280x1024
source activate demo
cd ~/demo/
python run.py
Where the first line sets the correct resolution - the second line activates an anaconda environment, the third line changes directory (parts of the script need it to be in that directory, and the 4th line finally runs the script that launches a fullscreen graphic (using pygame, if it matters.)
How can i set this to auto run after unity finishes setting up? Thanks so much... any help or comments at all are really appreciated...
EDIT: it looks like someone here had a similar problem: https://www.raspberrypi.org/forums/viewtopic.php?t=128602
They wanted to start a pygame script on startup - and were able to do so after their desktop environment loaded by putting a call to their program in ~/.config/lxsession/LXDE-pi/autostart - I'm not using LXDE - i'm on unity, so is there an equivalent?
Easiest way to run things at startup is to add them to the file /etc/rc.local.
But to do this properly, you should use the update-rc.d command. You may try it like this:
update-rc.d /path_to_script/script_name.sh start 2
Or use update-rc.d /path_to_script/script_name.sh defaults to create both startup and kill scripts all at once.
For more precise actions you can visit here for details information: http://manpages.ubuntu.com/manpages/xenial/man8/update-rc.d.8.html.
I have a rails 4 app on AWS EC2 Ubuntu. It will call a python program using backticks:
pyresult = `python /path/to/abc.py`
The python programe works perfect on my local dev. But it failed on the production server, with no error. (pyresult is empty)
I've debugged for long time, and found a possible cause: $PYTHONPATH
On local, i have the follow in my .bashrc, as the python program asked to add:
export PYTHONPATH="$HOME/.local/lib/python2.7/site-packages/:$PYTHONPATH"
On server, i also added them in .bashrc on server. But run echo $PYTHONPATH in my app gives me empty string.
Then I added explicitly in my rails app,
`export PYTHONPATH="$HOME/.local/lib/python2.7/site-packages/:$PYTHONPATH"`
pyresult = `python /path/to/abc.py`
but $PYTHONPATH is still empty. and calling python program still failed with no error.
Is it correct to guess the missing $PYTHONPATH caused the issue? if yes, how to solve the problem? Thanks for helping.(sorry i'm new to linux and not familiar with python)
Linux doesn't have system variables, only environment variables, which are only inherited by child processes. This line
`export PYTHONPATH="$HOME/.local/lib/python2.7/site-packages/:$PYTHONPATH"`
only sets PYTHONPATH for the subshell started by the backquotes (and any processes started from that subshell). Once the shell exists, the setting for PYTHONPATH is gone.
You need to use whatever facility Ruby has for modifying the environment of the current process, which I gather (as someone who doesn't use Ruby) is
ENV['PYTHONPATH'] = '...'
Then the shell started by
pyresult = `python /path/to/abc.py`
will inherit its PYTHONPATH value from the current Ruby process.
After using Open4 gem to run the command, I found the rootcause is paython programe can't find the lib, which is installed via setup.py. So it's another issue.
This question: Automatically import models on Django shell launch has answers explaining how to import models upon start by using shell_plus, but no answer about how to run code in general.
But is there an easy way to just run a python script?
python manage.py shell [or shell_plus] --run=script.py
Would just run the script as if you'd typed the whole thing in as the shell started.
I realize that you can import things in the shell, but then they're stuck within a namespace.
I would think ipython should have a way to run a script, and then import its locals() into the toplevel namespace. In that case you could just do %magic script.py and we'd be down to just one step, which would be good.
Changing the way you start the shell should be fine - the main goal is to just be able to create a file that's run on startup of the shell.
You can create your own custom command just like shell_plus has done: see the source of the shell_plus command to see how. In that code you can specify and run the file that needs to be executed before starting the shell. Also useful is Django's documentation on creating custom commands.
You can try to use environment variable PYTHONSTARTUP.
Also try django-extensions: django-extensions
See django-extensions/management/commands/shell_plus.py command.
From source code of this command I see that it respects PYTHONSTARTUP env variable.
shell_plus uses a limited form of IPython which doesn't process its startup & configuration, which defeats most normal attempts to run things at django+ipython shell startup. You can switch it to use the full version which will solve most problems.
Modify django_extensions/management/commands/shell_plus.py
remove:
embed(user_ns=imported_objects)
and replace it with:
from IPython import start_ipython
start_ipython(argv=[], user_ns=imported_objects)
Then your python code in the startup directories will be loaded.
Not sure if there's a flag you can use, but if you have ipython installed it should be as simple as:
ipython
Then when you're in the prompt:
run script.py
Then:
run manage.py shell
It seems that the easiest way is to run
cat myscript.py | awx-manage shell
For reference, see https://github.com/ansible/awx-operator/blob/7d2d1b3c5e3766966bfec0f9f58037f654b93b59/roles/installer/tasks/initialize_django.yml#L21-L24