I am trying to overwrite to environment variables in Python. I can read the value and then write the value and print the updated value. But then if I check the value in command line its still the original value. Why is that?
First, I create the variable
export MYVAR=old_val
My test script myvar.py
#!/usr/bin/env python3
import os
print (os.environ['MYVAR'])
os.environ['MYVAR'] = "new_val"
print (os.environ['MYVAR'])
Outputs
$ ./myvar.py
old_val
new_val
$ echo $MYVAR
old_val
As you can see, the last line of the output still shows the old_val
Short version:
The python script changes its environment. However this does not affect the environment of the parent process (The shell)
Long version:
Well this is a well know, but quite confusing problem.
What you have to know is, that there is not the environment, each process has its own environment.
So in your example above the shell (where you type your code) has one environment.
When you call ./myvar.py, a copy of the current environment is created and passed to your python script.
Your code 'only' changes this copy of the environment.
As soon as the python script is finished this copy is destroyed and the shell will see its initial unmodified environment.
This is true for most operating systems (Windows, Linux, MS-DOS, ...)
In other words: No child process can change the environment of the process, that called it.
In bash there is a trick, where you source a script instead of calling it as a process.
However if your python script starts another process (for example /bin/bash), then the child process would see the modified environment.
You started a new process that changed its environment and exited. That's all really.
You shouldn't expect that to affect the process you started it from (your shell).
Related
I am aware a lot of similar questions exist but I am unable to understand what is happening here. I am trying to follow instructions for this Stanford CoreNLP python wrapper here, one of the steps is to set theCORENLP_HOME environment variable.
I ran the command:
export CORENLP_HOME=/path/to/stanford-corenlp-full-2018-10-05
Restarted the terminal, or actually added to my ~/.bash_profile. Now when I do a echo $CORENLP_HOME in the terminal I am able to see the path correctly. But on the other hand, if the corenlp wrapper code tries to find the same path through python code it returns None.
So I separately checked two python commands, the wrapper code uses os.getenv():
import os
print(os.getenv('CORENLP_HOME')) #prints None
print(os.environ['CORENLP_HOME']) #Throws a KeyError exception
MacOS version: 10.15.4;Python: 3.7.6
I don't have a very deep understanding of environment variables in general, I want to understand what is happening here, or if I am missing something simple. Happy to provide more information!
Environment variables are not global in the UNIX process model. Each process is provided a set of environment variables by the parent process that starts it. That is typically a copy of the parent's environment variables. If you are not starting pycharm from the shell that ran the export command pycharm won't see that shell's environment variables.
The behavior you describe means you are not starting pycharm from the shell that did the export CORENLP_HOME=/path/to/stanford-corenlp-full-2018-10-05.
P.S., The UNIX process model also means that a process cannot modify the environment variables of a different process.
I'm trying to generate an encryption key for a file and then save it for use next time the script runs. I know that's not very secure, but it's just an interim solution for keeping a password out of a git repo.
subprocess.call('export KEY="password"', shell=True) returns 0 and does nothing.
Running export KEY="password" manually in my bash prompt works fine on Ubuntu.
subprocess.call('export KEY="password"', shell=True)
creates a shell, sets your KEY and exits: accomplishes nothing.
Environment variables do not propagate to parent process, only to child processes. When you set the variable in your bash prompt, it is effective for all the subprocesses (but not outside the bash prompt, for a quick parallel)
The only way to make it using python would be to set the password using a master python script (using os.putenv("KEY","password") or os.environ["KEY"]="password") which calls sub-modules or processes.
Using Python:
#SET:
os.environ["EnvVar"] = "1"
#GET:
print os.environ["EnvVar"]
This isn't a thing you can do. Your subprocess call creates a subshell and sets the env var there, but doesn't affect the current process, let alone the calling shell.
i have a Python script here and it is called from the Windows CMD.
It is executing some commands and also changing the Windows environment variables. Now after i changed them with the command "setx". I have to restart another Shell so the new variables are loaded into it.
Is it possible that the main shell from which i called my script can update the variables itself ?
Or is it possible to start another shell with the new variables and the script will continue in the new opened shell ?
Thanks
Each process has its own environment. When a process starts another process, the new process gets a (eventually modified) copy of its parent environment.
The rule is :
a process can modify its own environment - this modifications will be inherited by child processes started later
a process can modify (at start time) the environment of its child processes
a process can never modify its parent's environment (*)
So when you start a Python script from a cmd.exe :
the script can change its own environment and those changes will be visible by all subsequent commands of the script and all its children
the script cannot change the environment for its parent cmd.exe nor for subsequent commands of that cmd.exe
If you need to execute other batch commands after changing the environment, you will have to start a new cmd.exe for the python script and have this new shell execute the other commands, or directly execute a .bat file (both via subprocessmodule).
setx is a completely different thing : it updates the default environmnent that is given to processes started from windows explorer (including cmd.exe). That environment is stored permanently in Windows registry, and every change to it is broadcasted to all active processes ... that monitors it. Any windows GUI application can process it (and explorer does - that's how every explorer window know immediatly what is the current default environment), but console applications normaly do not.
(*) well it used to be possible for .com executable in old MS/DOS system and was even documented. It should be possible on Windows recent system through WriteProcessMemory API call but is absolutely non documented (thanks to eryksun for noticing)
You can't change the value of a environment variable.
Allow me to clarify: environment variables represent the variables set on the environment of a process when that process starts.
From the point-of-view of the new process, its environment is unchanging. Changing a variable on the environment (the process' parent) will not change the value of the environment variable seen by the process. Changing a variable on the process will not make it's environment see the change.
So, what can you change?
Variables set on your process. This is achieved in python by changing os.environ, or using set on the shell. Any changes will be seen by your process and any children you make (os.system, subprocess, most commands on the shell).
Variables set by the system (what SetX does). These changes will be seen by any new process launched directly by the system (Explorer, in Windows) after you change them.
I have a python script file that works perfectly when I use it from the terminal.
Now I have created the following .desktop file in order to launch it easily:
[Desktop Entry]
Name=Test
GenericName=Test
Comment=My test script
Type=Application
Exec=/opt/test.py
Icon=/opt/test.png
Categories=Utils;
When I launch it the GTK window appear but clicking a button that call an init.d script make it working not properly.
Therefore adding Terminal=true make it working perfectly but I don't want to have that terminal open.
So I have then put the following code in order to log the environment variables:
import os
with open("/tmp/py_env.log", "w") as env_log:
env_log.write(str(os.environ))
and found differences.
So my question is how to write the .desktop file so that my application is running like if I start it from my terminal (without having an opened terminal :))
The problem is valid, but I think "replicating the terminal environment" is the wrong approach to solve it.
Indeed, what makes the application work is not the fact that it's launched from the terminal, it's that the terminal happens to have some environment variables which matter to your application.
Therefore, what you should aim for is to have those environment variables set properly at all times, rather than assuming the terminal environment will always happen to contain them all the time for all your users.
Thus, you should:
Check which environment variables are different between the two environments
Make a list of those which matter (i.e. those which would make the .desktop file work properly), and of what their value needs to be for the script to work
Either:
Create a wrapper script for your Python script, which initializes those environment variables properly, OR
Set those environment variables from inside the Python script itself.
this question is similar to .bashrc not read when shell script is invoked from desktop shortcut
either initialize your environment in ~/.bash_profile instead of
~/.bashrc
OR
make your *.desktop file call a wrapper that initializes your
environment - e.g. by sourcing ~/.bashrc (or whatever script is
responsible now).
the second solution is more specific (does not effect all other unrelated launches of your shell) in should thus be preferred.
Thanks anyone to have participate to this question.
I have solved this issue by implemented use of pkexec instead of gksudo.
pkexec seems to reuse the current user environment then I don't have this issue anymore.
Thanks.
I have a requirement where 1 process sets a value as environment variable and I read that value in python using
os.environ
As per python doc:
This mapping is captured the first
time the os module is imported,
typically during Python startup as
part of processing site.py. Changes to
the environment made after this time
are not reflected in os.environ,
except for changes made by modifying
os.environ directly.
My problem is the process set/change the variable every time it calls the python script.
Please tell me a way to read the changed value.
Thanks,
I guess you can use os.getenv() to get the value of an environment variable any time, and this will reflect the most up-to-date state.
Update: note that there is no such thing as one "global" environment, at least not on Linux. Quoting Wikipedia:
In all Unix and Unix-like systems, each process has its own private set of environment variables. By default, when a process is created it inherits a duplicate environment of its parent process, except for explicit changes made by the parent when it creates the child.
Therefore, if you launch (fork) two processes from the same parent process (such as bash), and change an environment variable in one of the processes, the other process won't see it because it uses another copy of the environment of the parent process. Similarly, if you change the environment in the parent process after having launched the child processes, the child processes won't see the change because they have already created their private copy of the environment.
If your process sets/updates an environment variable and then calls the Python script, you would see the updated value in your Python script. But if these are parallel processes and the environment variable gets modified when the Python script is running then the updates to the environemnt variable is not seen in the Python script.