I have a bash file, it works fine when executed from terminal.
#!/bin/bash
source activate tensorflow_p36
python /home/ec2-user/abc/wsgi.py
Note: tensorflow_p36 being an in-built conda environment, does not require to be called from specific /env/bin directory. It can be activated from any directory. I think it's a feature of Amazon Deep Learning AMIs.
If I run this bash script with sudo it doesnt activate the virtual environment and works in default python environment. The python file can run in that virtual environment only.
I have tried all 3 alternatives (rc.local, .conf file, init.d config)here, also tried to use crontab as suggested here. I have also tried using supervisord to add this bash script as a program.
When the program runs from these methods, I always get the same import errors because it is using default python 3 environment which doesn't have the required dependencies.
I am working on Amazon CentOS (Deep learning AMI). Can someone please suggest a method to run this script every time system restarts?
In the rc.local, instruct root to run it as you:
su --command /path/to/bash/file --login grimlock
You can run it from your personal Crontab.
( crontab -l; printf '#reboot /path/to/bash/file\n' ) | crontab -
If you don't have a crontab there will be an error message from crontab -l but it's harmless.
crontab: no crontab for ec2-user
You just need to do this once, and the job will execute as yourself once the system comes up.
try to change source by .
. activate tensorflow_p36
python /home/ec2-user/abc/wsgi.py
also check chmod +x your path file.
Related
I'm using command docker run -e GRB_WLSACCESSID=xxxxxxx to set environment variables for Gurobi authorization. The OS of container is Ubuntu 16.04. This is OK if I login the container via SSH interactively and read the environment variables by python code os.getenv().
But, when I add this container as remote SSH interpreter in Pycharm and execute the python code along Pycharm, I can't get the environment variables.
At last I found the problem is that the environment variables generated by docker run -e can only be read by interactive shell. This can be validated by executing ssh root#x.x.x.x env, and interactively execute env after logging into the container. The former outputs less.
One possible solution is write some configuration manually after the container is generated, e.g., set the variables in /etc/environment (provided by this).
The other possible solution is add the variables manually in Pycharm edit configuration.
Is there a more elegant solution? :(
I finally understood the specific meaning of the relevant answers
It means, in the remote VM or container, create a linux script file named mypython as the python wrapper with the content:
#!/bin/bash -l
/path/to/interpreter/bin/python "$#"
, where /path/to/interpreter/bin/python is the path to the python interpreter. For conda interpreter, it might look like /root/miniconda3/envs/py37/bin/python.
The script mypython should be placed in the same path as binary python, i.e., /root/miniconda3/envs/py37/bin/mypython
And then add the execute permission to mypython:
chmod +x /root/miniconda3/envs/py37/bin/mypython
The above two steps can also be executed by command as an alternative:
echo '#!/bin/bash -l
/root/miniconda3/envs/py37/bin/python "$#"' > /root/miniconda3/envs/py37/bin/mypython
chmod +x /root/miniconda3/envs/py37/bin/mypython
At last, add the SSH interpreter in Pycharm, make sure the interpreter path is /root/miniconda3/envs/py37/bin/mypython
And the problem is solved.
I've written a shell script that does several things, such as...
define environment variables
activates a python virtual environment
The issue I am having is any commands that I put in after the 2nd step does not get recognized.
I do this by the following line
bash --rcfile "<PATH_TO_VIR_ENV>/bin/activate" -i
The way I use my shell script is to first log in to the linux server then do
bash ./myScript.sh
This activates my virtual environment but doesn't do the things I would like it to do afterwards. For example source by .bashrc file so that I can use aliases stored in there.
Thank you for your time in advance!
I am new to crontab and I'm struggling with the basics. I a lot of different issues. This is all being done on my Raspberry Pi. I am trying to schedule a python script to run every 10 minutes on weekdays. What makes this trickier is that my python script needs to use a virtual environment.
After doing some research I saw I could just activate the virtual environment with a bash script and then run the python script through that. This is the bash script:
#!/bin/bash
cd /home/pi/Desktop/projects/my_project
source env/bin/activate
python my_script.py
I have no idea if this is the best way to run python in a virtual environment through crontab but its all I could find online.
This is the code for the crontab itself:
SHELL=/bin/bash
0-59/10 * * * 1-5 /home/pi/Desktop/projects/my_project/cron_script.sh
I tested the crontab code by running this in the command line: sudo crontab -l | grep -v '^#' | cut -f 6- -d ' ' | while read CMD; do eval $CMD; done. Again I have no idea if this is the best way to test it immediately. This throws an error message:
bash: /home/pi/Desktop/projects/my_project/cron_script.sh: Permission denied
I'm really just confused at a lot of different steps here. Summary:
I don't know if I'm using a python virtual environment in crontab correctly or whether there's a cleaner way to do it.
I don't know if there is a better way to test crontab immediately. Writing the print statements from my python script to a txt file would also be nice.
I'm getting a permission denied error message.
The error you're having is likely due to the fact that your bash script is not marked as executable. You can use chmod to change access permissions.
Try this :
chmod +x /home/pi/Desktop/projects/my_project/cron_script.sh
I created a simple python script to change my ubuntu wallpaper. I want to this script run after whenever I logged in.
I tried to add command in startup application as python /bin/wallpaper_changer.py but it doesn't work.
I also tried to add a desktop entry in ~/.config/autostart/Myscript.desktop but it also doesn't work.
I also added this file path in crontab using sudo crontab -e #reboot python /bin/wallpaper_changer.py it also doesn't work.
I added entry in rc.local as python /bin/wallpaper_changer.py it also doesn't work
Maybe this link will help you. I prefer to put a start-up command in
/etc/rc.local. You can call it by typing on terminal sudo nano /etc/rc.local. You can also use crontab.
I found solution for this.
first make your source file executable.
chmod +x /path/to/your/file
after add it to cron tab.
sudo crontab -e
If it asks for any editor choose editor options.
after it add your path when # symbols end.
#reboot /path/to/your/file
I've seen a lot of scripts for daemonizing a python script in linux, but not much information about how to use them. Could anyone guide me on this?
I currently have a lengthy python script that listens on a socket for an incoming message, if it's the correct format accepts it and then stores it into the database. The script itself just opens the socket and then listens on a while true (which does the job!) and does all the work in there.
To daemonize it, would I have to modify my current script or call it from a separate script? I've seen examples of both but got neither to work.
Also, I'm using virtualenv which might the root of my problems, any hints on using this with daemonized scripts?
Create a shell-script that activates the virtual environment, and runs your Python script in the background.
Also, there should by a python module in the virtual environment that you can import and activate the environment from too. I don't have virtualenv working at the moment, so I can not check where it is, but search for activate (or something similar) in the virtual environment and you should find it.
Edit: Added a minimal Debian init.d script
The absolute minimal script needed to start a daemon when the computer boots, is this:
#!/bin/sh
/path/to/program &
The & makes the program run in the background, so it wont stop the rest of the boot process.
For a more complete script, copy /etc/init.d/skeleton and edit the new file. The important part to edit is the block at the beginning (between ### BEGIN INIT INFO and ### END INIT INFO, which is used by the update-rc.d program), and the NAME, DAEMON and DAEMON_ARGS variables. Hopefully that should be all that's needed for making a startup-script.
Activate the script as this:
sudo update-rc.d <name of script> defaults
sudo update-rc.d <name of script> enable
And to start it:
sudo update-rc.d <name of script> start
The <name of script> is just the name, not the full path.
script
export PYTHONPATH=.:/home/ubuntu/.local/lib/python2.7/site-packages/:/home/ubuntu/python/lib/python2.7/site-packages/
exec start-stop-daemon --start --chuid ubuntu --exec /home/ubuntu/python_envs/MyProj/bin/python /home/ubuntu/www/MyProj/MyProj.py -- --config-file-dir=/home/ubuntu/www/MyProj/config/ >> /home/ubuntu/startup.log 2>&1 &
end script
When you need to run an application in a python virtualenv, you can either 'activate' the virtualenv, or use that environment's unique python command.
As per the website "If you directly run a script or the python interpreter from the virtualenv's bin/ directory (e.g. path/to/env/bin/pip or /path/to/env/bin/python script.py) there's no need for activation." - http://pypi.python.org/pypi/virtualenv
I also have some python modules that were compiled from source. Those need to be in the PYTHONPATH environment variable. That could be part of your virtualenv activation, done with virtualwrapper, or explicitly called (as I do below.)
Calling the program from an UPSTART job works as well. My example is above.
On an Ubuntu 10.10 instance on Amazon EC2, I had better luck with the start-stop-daemon command.
I also struggled with some of the other upstart 'stanzas.' I am calling a python application with a specific virtualenv and some parameters to my executed program.