I am using python to run a bash script which contains a bash source command. For some reason the source command is not working.
Bash script code:
#!/bin/bash
SETTINGSFILE=/apps/settings
if test -f "$SETTINGSFILE"; then
source $SETTINGSFILE #This is not working
echo "settings file exists" #reaches here
else
echo "settings file not found"
fi
...
Python code:
import subprocess
rc = subprocess.call(["./<name-of-bash-script>.sh"])
Basically, I want to run the command source /apps/settings using a script. The control reaches to the echo statement (marked with comment), but still source command is not executed by the bash script. How to get it working?
I don't know why running "source file.sh" in a terminal would not load your environment variables (I am not on a linux system right now and can't really test this). However, in general, if you want to manipulate your environment, you should do that directly in python and not call a child process. For example,
import os
os.environ['THING'] = 'WORD'
I believe if you wish to change your environment variables from a child process then you need to imbed that child process in your current parent process (so that you become the child with the new environment) and you need to keep that child process open (you don't want it to terminate). You probably need something like "/bin/bash -i" at the end of your source file (to keep the subprocess running). You then need to use something like the pexpect module to imbed subprocess in current process.
import pexpect
child = pexpect.spawn('file.sh')
child.interact()
#check your os.environ to see if it is updated from your source file.
Sorry I can't be of more help.
Related
In Linux When I invoke python from the shell it replicates its environment, and starts the python process. Therefore if I do something like the following:
import os
os.environ["FOO"] = "A_Value"
When the python process returns, FOO, assuming it was undefined originally, will still be undefined. Is there a way for the python process (or any child process) to modify the environment of its parent process?
I know you typically solve this problem using something like
source script_name.sh
But this conflicts with other requirements I have.
No process can change its parent process (or any other existing process' environment).
You can, however, create a new environment by creating a new interactive shell with the modified environment.
You have to spawn a new copy of the shell that uses the upgraded environment and has access to the existing stdin, stdout and stderr, and does its reinitialization dance.
You need to do something like use subprocess.Popen to run /bin/bash -i.
So the original shell runs Python, which runs a new shell. Yes, you have a lot of processes running. No it's not too bad because the original shell and Python aren't really doing anything except waiting for the subshell to finish so they can exit cleanly, also.
It's not possible, for any child process, to change the environment of the parent process. The best you can do is to output shell statements to stdout that you then source, or write it to a file that you source in the parent.
I would use the bash eval statement, and have the python script output the shell code
child.py:
#!/usr/bin/env python
print 'FOO="A_Value"'
parent.sh
#!/bin/bash
eval `./child.py`
I needed something similar, I ended up creating a script envtest.py with:
import sys, os
sys.stdout = open(os.devnull, 'w')
# Python code with any number of prints (to stdout).
print("This is some other logic, which shouldn't pollute stdout.")
sys.stdout = sys.__stdout__
print("SomeValue")
Then in bash:
export MYVAR=$(python3 envtest.py)
echo "MYVAR is $MYVAR"
Which echos the expected: MYVAR is SomeValue
recently, I want to use python script to set environment in linux.This is one line of my code:
p = subprocess.call(['/bin/csh', '-c', "source setup.csh"])
My setup.csh file is below:
add questa10.2b
add ds5-2013.06
setenv MODELSIM modelsim.ini
But when I run my python, it shows that the files have sourced on screen, but it turns out I have to type myself on command line.
How could I solve these problem? Can any one please help me with this?
You're creating a new csh shell as a subprocess and then running your commands inside that shell, which then terminates. The commands do not run in, or affect, the parent shell within which Python is running. When you just run the commands yourself, they affect the current shell.
If you need these settings to persist in your current shell after Python terminates, your best bet in general is to source setup.csh rather than putting it in a Python script. If other child processes of the Python script need your environment variables, you can alter os.environ.
I want to write a python program that runs in the background.
I mean, like we install Python package. And later, we can run any script using python in front of the script name. This means that some python process is running in background which can take inputs and perform actions.
And in case of linux, you can call grep from anywhere. That means grep is also running in the background somehow.
I want to write something like that in python. When I call certain function with name and arguments at any time, it should perform the intended action without caring for the original code. But I am not able to find how to achieve that.
Can anyone please help me here?
Thanks in advance.
Clarification: the fact that you can run python or grep in a console just by typing their name, does not mean that they run in background. It means that there exist an executable file in some location, and this location is listed in the environment variable PATH.
For example, on my system I can run Python by typing python. The python executable is installed at /usr/local/bin/python, and has the execute permission bit on.
$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
and yes, /usr/local/bin is contained in PATH.
You can do the same with python scripts:
ensure that the very first line of your script contains #!/usr/bin/python or #!/usr/bin/env python
give your script execute permissions: chmod a+x yourScript
either move your script to one of the directories contained in $PATH, or add the directory where your script is located to PATH: export PATH=$PATH:/home/you/scripts
Have a look at
http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/
you can roll out your own daemon by inheriting the Daemon class and overriding run method
from daemon import Daemon
class run_daemon(Daemon):
def run(self):
import sys
run_daemon.execute_shell_command(sys.argv[1])
#staticmethod
def execute_shell_command(ShellCommand):
import subprocess
process = subprocess.Popen(ShellCommand, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.communicate()
I am trying to write a python script that will change my cwd to the desired directory. I was not able to do this task directly from python so I wrote a simple batch script to do that.
Changedir.bat
#echo off
chdir /D F:\cygwin\home\
If I execute the above script directly in my cmd it works fine but if I try to execute it with a python script nothing happens. My cwd remains same.
PythonScript.py
import shlex,subprocess
change_dir = r'cmd.exe /c C:\\Users\\test.bat'
command_change = shlex.split(change_dir)
subprocess.call(command_change)
Of course this can't work, because subprocess.call is spawning whole new process for your script. This executes the script in a completely separate environment.
If you want to change directory in the command prompt you have to use either cd or a .bat script.
You can't get another process (i.e. Python) to do it because changes to the current directory, made in another process are not reflected back to the parent process. The reason the .bat script works is that it is processed by the command shell that invokes it rather than by a child process.
You could try this. It works in Linux to change the CWD of the current shell. It is horrible.
def quote_against_shell_expansion(s):
import pipes
return pipes.quote(s)
def put_text_back_into_terminal_input_buffer(text):
# use of this means that it only works in an interactive session
# (and if the user types while it runs they could insert
# characters between the characters in 'text')
import fcntl, termios
for c in text:
fcntl.ioctl(1, termios.TIOCSTI, c)
def change_shell_working_directory(dest):
put_text_back_into_terminal_input_buffer("cd "+quote_against_shell_expansion(dest)+"\n")
I am writing a python script (Linux) that is adding some shell aliases (writes them to HOME/.bash_aliases).
In order to make an alias available immediately after it was written I should issue the following bash built-in:
source HOME/.bashrc
source is a bash built-in so I cannot just:
os.system(source HOME/.bashrc)
If i try something like:
os.system('/bin/bash -c source HOME/.bashrc')
...will freeze the script (just like is waiting for something).
Any suggestions ?
What you want is not possible. A program (your script) cannot modify the environment of the caller (the shell you run it from).
Another approach which would allow you to do something close is to write it in terms of a bash function, which is run in the same process and can modify the caller. Note that sourcing during runtime can have possible negative side-effects depending on what the user has in their bashrc.
what you are trying to do is impossible. or better: how you are trying to do it is impossible.
your bash command is wrong. bash -s command does not execute command. it just stores the string "command" in the variable $1 and then drops you to the prompt. that is why the python script seems to freeze. what you meant to do is bash -c command.
why do you source .bashrc? would it not be enough to just source .bash_aliases?
even if you got your bash command right, the changes will only take effect in the bash session started from python. once that bash session is closed, and your python script is done, you are back at your original bash session. all changes in the bash session started from python is lost.
everytime you want to change something in the current bash session, you have to do it from inside the current bash session. most of the commands you run from bash (system commands, python scripts, even bash scripts) will spawn another process, and everything you do in that other process will not affect your first bash session.
source is a bash builtin which allows you to execute commands inside the currently running bash session, instead of spawning another process and running the commands there. defining a bash function is another way to execute commands inside the currently running bash session.
see this answer for more information about sourcing and executing.
what you can do to achieve what you want
modify your python script to just do the changes necessary to .bash_aliases.
prepare a bash script to run your python script and then source .bash_aliases.
#i am a bash script, but you have to source me, do not execute me.
modify_bash_aliases.py "$#"
source ~/.bash_aliases
add an alias to your .bashrc to source that script
alias add_alias='source modify_bash_aliases.sh'
now when you type add_alias some_alias in your bash prompt it will be replaced with source modify_bash_aliases.sh and then executed. since source is a bash builtin, the commands inside the script will be executed inside the currently running bash session. the python script will still run in another process, but the subsequent source command will run inside your currently running bash session.
another way
modify your python script to just do the changes necessary to .bash_aliases.
prepare a bash function to run your python script and then source .bash_aliases.
add_alias() {
modify_bash_aliases.py "$#"
source ~/.bash_aliases
}
now you can call the function like this: add_alias some_alias
I had an interesting issue where I needed to source an RC file to get the correct output in my python script.
I eventually used this inside my function to bring over the same variables from the bash file I needed to source. Be sure to have os imported.
with open('overcloudrc') as data:
lines = data.readlines()
for line in lines:
var = line.split(' ')[1].split('=')[0].strip()
val = line.split(' ')[1].split('=')[1].strip()
os.environ[var] = val
Working solution from Can I use an alias to execute a program from a python script :
import subprocess
sp = subprocess.Popen(["/bin/bash", "-i", "-c", "nuke -x scriptpath"])
sp.communicate()