i have a Python script here and it is called from the Windows CMD.
It is executing some commands and also changing the Windows environment variables. Now after i changed them with the command "setx". I have to restart another Shell so the new variables are loaded into it.
Is it possible that the main shell from which i called my script can update the variables itself ?
Or is it possible to start another shell with the new variables and the script will continue in the new opened shell ?
Thanks
Each process has its own environment. When a process starts another process, the new process gets a (eventually modified) copy of its parent environment.
The rule is :
a process can modify its own environment - this modifications will be inherited by child processes started later
a process can modify (at start time) the environment of its child processes
a process can never modify its parent's environment (*)
So when you start a Python script from a cmd.exe :
the script can change its own environment and those changes will be visible by all subsequent commands of the script and all its children
the script cannot change the environment for its parent cmd.exe nor for subsequent commands of that cmd.exe
If you need to execute other batch commands after changing the environment, you will have to start a new cmd.exe for the python script and have this new shell execute the other commands, or directly execute a .bat file (both via subprocessmodule).
setx is a completely different thing : it updates the default environmnent that is given to processes started from windows explorer (including cmd.exe). That environment is stored permanently in Windows registry, and every change to it is broadcasted to all active processes ... that monitors it. Any windows GUI application can process it (and explorer does - that's how every explorer window know immediatly what is the current default environment), but console applications normaly do not.
(*) well it used to be possible for .com executable in old MS/DOS system and was even documented. It should be possible on Windows recent system through WriteProcessMemory API call but is absolutely non documented (thanks to eryksun for noticing)
You can't change the value of a environment variable.
Allow me to clarify: environment variables represent the variables set on the environment of a process when that process starts.
From the point-of-view of the new process, its environment is unchanging. Changing a variable on the environment (the process' parent) will not change the value of the environment variable seen by the process. Changing a variable on the process will not make it's environment see the change.
So, what can you change?
Variables set on your process. This is achieved in python by changing os.environ, or using set on the shell. Any changes will be seen by your process and any children you make (os.system, subprocess, most commands on the shell).
Variables set by the system (what SetX does). These changes will be seen by any new process launched directly by the system (Explorer, in Windows) after you change them.
Related
I am trying to overwrite to environment variables in Python. I can read the value and then write the value and print the updated value. But then if I check the value in command line its still the original value. Why is that?
First, I create the variable
export MYVAR=old_val
My test script myvar.py
#!/usr/bin/env python3
import os
print (os.environ['MYVAR'])
os.environ['MYVAR'] = "new_val"
print (os.environ['MYVAR'])
Outputs
$ ./myvar.py
old_val
new_val
$ echo $MYVAR
old_val
As you can see, the last line of the output still shows the old_val
Short version:
The python script changes its environment. However this does not affect the environment of the parent process (The shell)
Long version:
Well this is a well know, but quite confusing problem.
What you have to know is, that there is not the environment, each process has its own environment.
So in your example above the shell (where you type your code) has one environment.
When you call ./myvar.py, a copy of the current environment is created and passed to your python script.
Your code 'only' changes this copy of the environment.
As soon as the python script is finished this copy is destroyed and the shell will see its initial unmodified environment.
This is true for most operating systems (Windows, Linux, MS-DOS, ...)
In other words: No child process can change the environment of the process, that called it.
In bash there is a trick, where you source a script instead of calling it as a process.
However if your python script starts another process (for example /bin/bash), then the child process would see the modified environment.
You started a new process that changed its environment and exited. That's all really.
You shouldn't expect that to affect the process you started it from (your shell).
I'd like to save my credentials in a file (config_vars.env) which is gitignored.
Then I'd like to start my project and have those credentials set to environment variables because my program uses os.environ.get('DB_NAME') and the like.
So I'd like those environment variables to be set while my script runs, then deleted when my program stops running.
I could literally set them using python or bash, then unset them upon exit. But that's not ideal because if my program crashes the environment variables are left there.
Ideally, I'd be able to automatically set them in a virtual environment, only available to my process, and when the process stops running the env vars are gone.
Is there any way to do this in native python? I've looked into things like click or dotenv for python, but is there no other way?
Here's what I've got so far:
import os
import subprocess
def bash_command():
#not good
#subprocess.Popen(cmd, shell=False, executable=".\git-bash.exe")
#os.popen('setenv a b')
subprocess.call("config_vars.sh", shell=False)
print(os.environ.get('DB_NAME')) # prints None because the env var dies with bash
import time
time.sleep(5) # delays for 5 seconds
bash_command()
and config_vars.sh is:
export ENV_FILE=env/config_vars.env
echo $DB_NAME
That echo command shows that it worked, but the bash process ends, removing that environment variable (or the whole virtual environment altogether) and the python process continues without having access to it.
So the question is: How can I set environment variables that die when my python process dies?
You have to capture the output of the script; the child cannot affect the parent's environment, so you need some form of interprocess communication.
value = subprocess.check_output("config_vars.sh", shell=False).lstrip('\n')
I have a python script file that works perfectly when I use it from the terminal.
Now I have created the following .desktop file in order to launch it easily:
[Desktop Entry]
Name=Test
GenericName=Test
Comment=My test script
Type=Application
Exec=/opt/test.py
Icon=/opt/test.png
Categories=Utils;
When I launch it the GTK window appear but clicking a button that call an init.d script make it working not properly.
Therefore adding Terminal=true make it working perfectly but I don't want to have that terminal open.
So I have then put the following code in order to log the environment variables:
import os
with open("/tmp/py_env.log", "w") as env_log:
env_log.write(str(os.environ))
and found differences.
So my question is how to write the .desktop file so that my application is running like if I start it from my terminal (without having an opened terminal :))
The problem is valid, but I think "replicating the terminal environment" is the wrong approach to solve it.
Indeed, what makes the application work is not the fact that it's launched from the terminal, it's that the terminal happens to have some environment variables which matter to your application.
Therefore, what you should aim for is to have those environment variables set properly at all times, rather than assuming the terminal environment will always happen to contain them all the time for all your users.
Thus, you should:
Check which environment variables are different between the two environments
Make a list of those which matter (i.e. those which would make the .desktop file work properly), and of what their value needs to be for the script to work
Either:
Create a wrapper script for your Python script, which initializes those environment variables properly, OR
Set those environment variables from inside the Python script itself.
this question is similar to .bashrc not read when shell script is invoked from desktop shortcut
either initialize your environment in ~/.bash_profile instead of
~/.bashrc
OR
make your *.desktop file call a wrapper that initializes your
environment - e.g. by sourcing ~/.bashrc (or whatever script is
responsible now).
the second solution is more specific (does not effect all other unrelated launches of your shell) in should thus be preferred.
Thanks anyone to have participate to this question.
I have solved this issue by implemented use of pkexec instead of gksudo.
pkexec seems to reuse the current user environment then I don't have this issue anymore.
Thanks.
I have a requirement where 1 process sets a value as environment variable and I read that value in python using
os.environ
As per python doc:
This mapping is captured the first
time the os module is imported,
typically during Python startup as
part of processing site.py. Changes to
the environment made after this time
are not reflected in os.environ,
except for changes made by modifying
os.environ directly.
My problem is the process set/change the variable every time it calls the python script.
Please tell me a way to read the changed value.
Thanks,
I guess you can use os.getenv() to get the value of an environment variable any time, and this will reflect the most up-to-date state.
Update: note that there is no such thing as one "global" environment, at least not on Linux. Quoting Wikipedia:
In all Unix and Unix-like systems, each process has its own private set of environment variables. By default, when a process is created it inherits a duplicate environment of its parent process, except for explicit changes made by the parent when it creates the child.
Therefore, if you launch (fork) two processes from the same parent process (such as bash), and change an environment variable in one of the processes, the other process won't see it because it uses another copy of the environment of the parent process. Similarly, if you change the environment in the parent process after having launched the child processes, the child processes won't see the change because they have already created their private copy of the environment.
If your process sets/updates an environment variable and then calls the Python script, you would see the updated value in your Python script. But if these are parallel processes and the environment variable gets modified when the Python script is running then the updates to the environemnt variable is not seen in the Python script.
I'm trying to use python to set environment variables that will persist in Pythons parent environment, even after python exits back to the shell, but will not persist once the parent shell is closed. Opening a new shell should require that the python script be run again in order to set the environment correctly.
Based off the recommendations from this post, I'm trying to do this using the win32com python library. Unfortunately, I have very little experience with the win32 api.
Basically, I need a way to get a handle to the current environment, and set environment variables in such a way that they will persist in python's parent environment, but will not persist after the parent environment exits.
The linked post tells how to change the default environment which will affect new processes. It manipulates registry values.
"A process can never directly change the environment variables of another process that is not a child of that process", says MS documentation. So you'll never reach your stated goal from within a child process, Python or not.
It is impossible to change the environment of the parent, by design. The best you can do is have your program emit commands that alter the environment, and then the caller of your program needs to eval the output of your command.