While there are many questions regarding accessing/ setting env variables in python, I could not find the answer for my particular scenario.
I have a shell script that when called it does the export of a bunch of env variables that are used later on.
When you need those variables to be available from your current session, then you would do
. ./script_exp_var.sh
that does, say
export MYVAR = MYVAL
then if you run python, you could access it with os.environ.get('MYVAR').
My question is how to invoke the script from python and then to have to access the env variables that called script just exported. Is it possible at all and if yes how?
Note: I know I could set the env var from python using os.environ["MYVAR"] = MYVAL but I would like to use the existing logic in my ./script_exp_var.sh because it exports many variables.
Also making sure to execute the script first and then execute the python is also not an option for my scenario.
You can't. Environment variables are copied from parent to child, never back to the parent.
If you execute a shell script from python then the environment variables will be set in that shell process (the child) and python will be unaware of them.
You could write a parser to read the shell commands from python, but that's a lot of work.
Better to write a shell script with the settings in that and then call the python program as a child of the script.
Alternatively, write a shell script that echoes the values back to python which can be picked-up using a pipe.
Related
I am trying - within a python script - to list all of the aliases that were defined in the shell that launched this python script.
Essentially I am looking for an analogous method to os.environ but one that would let me list the aliases in shell that called this python script (whether it returns a dict or list or string is not important).
Note: I am not trying to execute these aliases. I merely want to
list them.
I have tried subprocess.call("alias", shell=True) but it merely returns "0". I'm fairly sure that the subprocess is not the way to do it anyway. As far as I know, subprocess fires up a whole new shell which, even if it were to somehow read my .bashrc file, would not necessarily have the same aliases as the shell that called my python script.
In this particular case, the calling shell is a bash shell but I'd like to be able to extend this ability to other shells as well.
Is something like this even possible?
Thanks!
So I know how subprocess works and use it a lot, but I've run into a strange issue. I need to execute an export of some environment variables. The reason is that some program (black-box) executes a program that seems like it runs in a subshell, so it doesn't have access to the environment variables but it has access to all my files.
I can't hard code the environment variables so I want to source or . the file that has the export commands in it. However, if I source or . that file in a subprocess, it won't make any difference to its parent process. In which case I either need some function besides subprocess that can execute shell commands without creating a subprocess, if that exists. Another issue is that a subprocess doesn't have the proper permissions to read the file.
And copying the environment variables via os isn't really possible either.
Does anything besides subprocess exist? Or is there some other kind of workaround?
IMHO the simplest solution consists in creating a new shell script (let's call it run_black_box.sh) which sources the setup script (let's assume it is named setup.sh) to initialize the environment and then calls the black_box program.
Here is a possible content of run_black_box.sh:
#/bin/bash
source setup.sh
black_box
The you can pass run_black_box.sh to subprocess for execution.
I need to run a lot of bash commands from Python. For the moment I'm doing this with
subprocess.Popen(cmd, shell=True)
Is there any solution to run all these commands in the same shell? subprocess.Popen opens a new shell at every execution and I need to set up all the necessary variables at every call, in order for cmd command to work properly.
subprocess.Popen lets you supply a dictionary of environment variables, which will become the environment for the process being run. If the only reason you need shell=True is to set an environment variable, then I suggest you use an explicit environment dictionary instead; it's safer and not particularly difficult. Also, it's usually much easier to construct command invocations when you don't have to worry about quoting and shell metacharacters.
It may not even be necessary to construct the environment dictionary, if you don't mind having the environment variables set in the running process. (Most of the time, this won't be a problem, but sometimes it is. I don't know enough about your application to tell.)
If you can modify your own environment with the settings, just do that:
os.environ['theEnvVar'] = '/the/value'
Then you can just use a simple Popen.call (or similar) to run the command:
output = subprocess.check_output(["ls", "-lR", "/tmp"])
If for whatever reason you cannot change your own environment, you need to make a copy of the current environment, modify it as desired, and pass it to each subprocess.call:
env = os.environ.copy()
env['theEnvVar'] = '/the/value'
output = subprocess.check_output(["ls", "-lR", "/tmp"], env=env)
If you don't want to have to specify env=env every time, just write a little wrapper class.
Why not just create a shell script with all the commands you need to run, then just use a single subprocess.Popen() call to run it? If the contents of the commands you need to run depend on results calculated in your Python script, you can just create the shell script dynamically, then run it.
Use multiprocessing instead, it's more lightweight and efficient.
Unlike subprocess.Popen it does not open a new shell at every execution.
You didn't say you need to run subprocess.Popen and you may well not need to; you just said that's what you're currently doing. More justification please.
See set env var in Python multiprocessing.Process for how to set your env vars once and for all in the parent process.
I'm trying to use python to set environment variables that will persist in Pythons parent environment, even after python exits back to the shell, but will not persist once the parent shell is closed. Opening a new shell should require that the python script be run again in order to set the environment correctly.
Based off the recommendations from this post, I'm trying to do this using the win32com python library. Unfortunately, I have very little experience with the win32 api.
Basically, I need a way to get a handle to the current environment, and set environment variables in such a way that they will persist in python's parent environment, but will not persist after the parent environment exits.
The linked post tells how to change the default environment which will affect new processes. It manipulates registry values.
"A process can never directly change the environment variables of another process that is not a child of that process", says MS documentation. So you'll never reach your stated goal from within a child process, Python or not.
It is impossible to change the environment of the parent, by design. The best you can do is have your program emit commands that alter the environment, and then the caller of your program needs to eval the output of your command.
I use TextMate to debug python script, as I like the feature of using 'Command-R' for running python from TextMate, and I learned that emacs provide similar feature.
I need to know if the python is run from command line or from TextMate/emacs. How can I do that?
ADDED
I use TextMate for python coding/debugging, and it's pretty useful. But, sometimes I need to run the test using command line. I normally turn on debugging/logging mode with TextMate, and off with command line mode. This is the reason I asked the question. Also, I plan to use emacs for python debugging, so I wanted to ask the case for emacs.
I got an answer in the case with emacs, and I happen to solve this issue with TextMate.
Set variables in Preferences -> Advanced -> Shell Variables, and I found that TM_ORGANIZATION_NAME is already there to be used. So, I'll just use this variable.
Use this variable, if os.environ['TM_ORGANIZATION_NAME']: return True
I guess the shell variable from TextMate disappear when I'm done using it.
For Emacs: If python is run as an inferior process, then the environment variable INSIDE_EMACS will be set.
From docs:
Emacs sets the environment variable
INSIDE_EMACS in the subshell to a
comma-separated list including the
Emacs version. Programs can check this
variable to determine whether they are
running inside an Emacs subshell.
sys.argv will tell you how Python was invoked. I don't know about TextMate, but when I tell Emacs to eval buffer, its value is ['-c']. That means it's executing a specified command, according to the man page. If Python's run directly from the command line with no parameters, sys.argv will be []. If you run a python script, it will have the script name and whatever arguments you pass it. You might want to set up your python-mode in Emacs and whatever the equivalent in TextMate is to put something special like -t in the command line.
That's pretty hackish though. Maybe there's a better way.
From the docs for sys.path:
As initialized upon program startup,
the first item of this list, path[0],
is the directory containing the script
that was used to invoke the Python
interpreter. If the script directory
is not available (e.g. if the
interpreter is invoked interactively
or if the script is read from standard
input), path[0] is the empty string,
which directs Python to search modules
in the current directory first. Notice
that the script directory is inserted
before the entries inserted as a
result of PYTHONPATH.
So
if sys.path[0]:
# python was run interactively
else:
# python is running a script.
Or, for example, from the IPython prompt (inside Emacs):
In [65]: sys.path
Out[65]:
['', <-------------------- first entry is empty string
'/usr/bin',
'/usr/local/lib/python2.6/dist-packages/scikits.statsmodels-0.2.0-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/pyinterval-1.0b21-py2.6-linux-i686.egg',
... ]
Use Command-R to run the script directly
Use Shift-Command-R to run the script from terminal.