I'd like to save my credentials in a file (config_vars.env) which is gitignored.
Then I'd like to start my project and have those credentials set to environment variables because my program uses os.environ.get('DB_NAME') and the like.
So I'd like those environment variables to be set while my script runs, then deleted when my program stops running.
I could literally set them using python or bash, then unset them upon exit. But that's not ideal because if my program crashes the environment variables are left there.
Ideally, I'd be able to automatically set them in a virtual environment, only available to my process, and when the process stops running the env vars are gone.
Is there any way to do this in native python? I've looked into things like click or dotenv for python, but is there no other way?
Here's what I've got so far:
import os
import subprocess
def bash_command():
#not good
#subprocess.Popen(cmd, shell=False, executable=".\git-bash.exe")
#os.popen('setenv a b')
subprocess.call("config_vars.sh", shell=False)
print(os.environ.get('DB_NAME')) # prints None because the env var dies with bash
import time
time.sleep(5) # delays for 5 seconds
bash_command()
and config_vars.sh is:
export ENV_FILE=env/config_vars.env
echo $DB_NAME
That echo command shows that it worked, but the bash process ends, removing that environment variable (or the whole virtual environment altogether) and the python process continues without having access to it.
So the question is: How can I set environment variables that die when my python process dies?
You have to capture the output of the script; the child cannot affect the parent's environment, so you need some form of interprocess communication.
value = subprocess.check_output("config_vars.sh", shell=False).lstrip('\n')
Related
I work in tc shell and have environment set script with commands like setenv foo bar. I use to run this script by command source set_env.sh and then, when environment is set, run compilation command make my_ target...
I would like to automate the whole process by executing it from Python by calling subprocess.Popen or even os.system. The troubles are:
Shell opened form Python do not recognize command source. My assumption that /bin/bash shell is opened. How do I force Python to open /bin/tcsh?
How do I preserve changed environment setting for second command (make my_ target...)?
If you use the subprocess package in Python then it won't run the shell by default, it will just launch a process (without using the shell). You can add shell=True, but this won't do you much good since environment variables are set for the current process only, and are not sent back to the parent process.
Python starts a tcsh child process, and child processes can't affect parent processes.
The solution would be to make your script print out the environment variables, and then parse the output of that in Python.
So if your script prints something like:
XX=one
YY=two
Then in Python do something along the likes of:
for line in output.split('\n'):
if line == '':
continue
k, v = line.split('=')
os.environ[k] = v
I am trying to overwrite to environment variables in Python. I can read the value and then write the value and print the updated value. But then if I check the value in command line its still the original value. Why is that?
First, I create the variable
export MYVAR=old_val
My test script myvar.py
#!/usr/bin/env python3
import os
print (os.environ['MYVAR'])
os.environ['MYVAR'] = "new_val"
print (os.environ['MYVAR'])
Outputs
$ ./myvar.py
old_val
new_val
$ echo $MYVAR
old_val
As you can see, the last line of the output still shows the old_val
Short version:
The python script changes its environment. However this does not affect the environment of the parent process (The shell)
Long version:
Well this is a well know, but quite confusing problem.
What you have to know is, that there is not the environment, each process has its own environment.
So in your example above the shell (where you type your code) has one environment.
When you call ./myvar.py, a copy of the current environment is created and passed to your python script.
Your code 'only' changes this copy of the environment.
As soon as the python script is finished this copy is destroyed and the shell will see its initial unmodified environment.
This is true for most operating systems (Windows, Linux, MS-DOS, ...)
In other words: No child process can change the environment of the process, that called it.
In bash there is a trick, where you source a script instead of calling it as a process.
However if your python script starts another process (for example /bin/bash), then the child process would see the modified environment.
You started a new process that changed its environment and exited. That's all really.
You shouldn't expect that to affect the process you started it from (your shell).
I'm trying to generate an encryption key for a file and then save it for use next time the script runs. I know that's not very secure, but it's just an interim solution for keeping a password out of a git repo.
subprocess.call('export KEY="password"', shell=True) returns 0 and does nothing.
Running export KEY="password" manually in my bash prompt works fine on Ubuntu.
subprocess.call('export KEY="password"', shell=True)
creates a shell, sets your KEY and exits: accomplishes nothing.
Environment variables do not propagate to parent process, only to child processes. When you set the variable in your bash prompt, it is effective for all the subprocesses (but not outside the bash prompt, for a quick parallel)
The only way to make it using python would be to set the password using a master python script (using os.putenv("KEY","password") or os.environ["KEY"]="password") which calls sub-modules or processes.
Using Python:
#SET:
os.environ["EnvVar"] = "1"
#GET:
print os.environ["EnvVar"]
This isn't a thing you can do. Your subprocess call creates a subshell and sets the env var there, but doesn't affect the current process, let alone the calling shell.
recently, I want to use python script to set environment in linux.This is one line of my code:
p = subprocess.call(['/bin/csh', '-c', "source setup.csh"])
My setup.csh file is below:
add questa10.2b
add ds5-2013.06
setenv MODELSIM modelsim.ini
But when I run my python, it shows that the files have sourced on screen, but it turns out I have to type myself on command line.
How could I solve these problem? Can any one please help me with this?
You're creating a new csh shell as a subprocess and then running your commands inside that shell, which then terminates. The commands do not run in, or affect, the parent shell within which Python is running. When you just run the commands yourself, they affect the current shell.
If you need these settings to persist in your current shell after Python terminates, your best bet in general is to source setup.csh rather than putting it in a Python script. If other child processes of the Python script need your environment variables, you can alter os.environ.
i have a Python script here and it is called from the Windows CMD.
It is executing some commands and also changing the Windows environment variables. Now after i changed them with the command "setx". I have to restart another Shell so the new variables are loaded into it.
Is it possible that the main shell from which i called my script can update the variables itself ?
Or is it possible to start another shell with the new variables and the script will continue in the new opened shell ?
Thanks
Each process has its own environment. When a process starts another process, the new process gets a (eventually modified) copy of its parent environment.
The rule is :
a process can modify its own environment - this modifications will be inherited by child processes started later
a process can modify (at start time) the environment of its child processes
a process can never modify its parent's environment (*)
So when you start a Python script from a cmd.exe :
the script can change its own environment and those changes will be visible by all subsequent commands of the script and all its children
the script cannot change the environment for its parent cmd.exe nor for subsequent commands of that cmd.exe
If you need to execute other batch commands after changing the environment, you will have to start a new cmd.exe for the python script and have this new shell execute the other commands, or directly execute a .bat file (both via subprocessmodule).
setx is a completely different thing : it updates the default environmnent that is given to processes started from windows explorer (including cmd.exe). That environment is stored permanently in Windows registry, and every change to it is broadcasted to all active processes ... that monitors it. Any windows GUI application can process it (and explorer does - that's how every explorer window know immediatly what is the current default environment), but console applications normaly do not.
(*) well it used to be possible for .com executable in old MS/DOS system and was even documented. It should be possible on Windows recent system through WriteProcessMemory API call but is absolutely non documented (thanks to eryksun for noticing)
You can't change the value of a environment variable.
Allow me to clarify: environment variables represent the variables set on the environment of a process when that process starts.
From the point-of-view of the new process, its environment is unchanging. Changing a variable on the environment (the process' parent) will not change the value of the environment variable seen by the process. Changing a variable on the process will not make it's environment see the change.
So, what can you change?
Variables set on your process. This is achieved in python by changing os.environ, or using set on the shell. Any changes will be seen by your process and any children you make (os.system, subprocess, most commands on the shell).
Variables set by the system (what SetX does). These changes will be seen by any new process launched directly by the system (Explorer, in Windows) after you change them.