How do I set environment variables in one script and after piping the output to another script, I need certain environment variable to be set in first script and used in second script.
I am using python scripts.
python script1.py | python script2.py
in script1.py , I am doing
os.environ["some_key"] = some_value
and in script2.py , I am trying to read it as
saved_value = os.environ["some_key"]
When I try to run the above scripts as mentioned above,
Traceback (most recent call last):
File "script2.py", line 11, in <module>
filename = os.environ["some_key"]
File "/usr/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
Is there anyway this can be corrected ? I specifically need it to be read from os.environ["some_key"] in script2.py as that file is part of bigger framework and cant change it. I have flexibility with with script1.py
It's not possible to set environment variables of another process - unless you launch it yourself.
If you can, consider changing script1 to launch script2 directly using subprocess.Popen, and supply the env argument.
If you can't do this, you'll need to change the parent process (probably the bash script that launchs the scripts) to, somehow, receive the desired variable from script1 and set it when launching script2.
You won't be able to alter the environment of a sibling process.
You can use a convoluted construct like the one below:
{ export some_key=$(python script1.py) ; } |& python script2.py
where script1.py outputs the environment variable to stdout and writes the stuff to be fed to script2.py to stderr. This will only allow setting one environment variable though.
Example:
$ cat script1.py
import sys
print "some_env_value"
sys.stderr.write("some_piped_value\n")
$ cat script2.py
import os
import sys
print "Passed through enviroment:", os.environ["some_key"]
print "Passed through stdin:",
for line in sys.stdin:
print line
Result:
$ { export some_key=$(python script1.py) ; } |& python script2.py
Passed through enviroment: some_env_value2
Passed through stdin: some_piped_value
Related
I downloaded a script written in Python, called let's say 'myScript.py', but I don't know how to run it.
It has the following structure:
import numpy as np
import argparse
parser = argparse.ArgumentParser(description='manual to this script')
parser.add_argument('--file', type=str, default = None)
parser.add_argument('--timeCor', type=bool, default = False)
parser.add_argument('--iteration', type=str, default = 'Newton')
args = parser.parse_args()
def func1(file):
...
def func2(file):
...
def calculate(data, timeCor=False, iteration='Newton'):
...
if __name__ == "__main__":
print('\n--- Calculating ---\nN file:',args.file)
print('Time correction =',args.timeCor,
'\nIteration strategy =',args.iteration,'\n')
rawdata,data = func2(args.file)
pos = calculate(data,timeCor=args.timeCor,iteration=args.iteration)
for each in pos:
print('Pos:',format(np.uint8(each[0]),'2d'),each[1:-1])
np.savetxt('pos.csv',pos,delimiter=',')
print('--- Save file as "pos.csv" in the current directory ---')
How can I run it from command line? And from another script?
Thank you in advance!
If I understood your question correctly, you are asking about executing this python program from another python script. This can be done by making use of subprocess.
import subprocess
process = subprocess.run([“python3”, “path-to-myScript.py”, “command line arguments here”], capture_output=True)
output = process.stdout.decode() # stdout is bytes
print(output)
If you do not want to provide the command in a list, you can add shell=True as an argument to subprocess.run(), or you can use shlex.split().
If you are on windows, replace python3 with python.
However this solution is not very portable, and on a more general note, if you are not strictly needing to run the script, I would recommend you to import it and call its functions directly instead.
To run the python script from command line:
python myScript.py --arguments
And as #treuss has said, if you are on a Unix system (macOS or Linux) you can add a shebang to the top of the script, but I recommend to use the following instead:
#!/usr/bin/env python3
for portability sake, as the actual path of python3 may vary.
To run the edited program:
chmod +x myScript # to make file executable, and note that there is no longer a .py extension as it is not necessary
./myScript --arguments
Run it by executing:
python myScript.py
Alternatively, on systems which support it (UNIX-Like systems), you can add what's called a she-bang to the first line of the file:
#!/usr/bin/python
Note that /usr/bin/python should be the actual path where your python is located. After that, make the file executable:
chmod u+x myScript.py
and run it either from the current directory:
./myScript.py
or from a different directory with full or relative path:
/path/to/python/scripts/myScript.py
I want to run a python script foo.py from the command line like this
$ foo
Using a shebang in foo.py, for example:
#!/usr/bin/env python
print('this is foo')
allows me to call it like this:
$ ./foo.py
How do I drop the leading ./ and the trailing .py?
First, rename the file from foo.py to foo.
Then, move the file to /usr/local/bin/ or /home/user/.local/bin if the script will only be executed by a single user. Instead, if your script is placed somewhere in the system for example "/path/to/foo", you could add your "/path/to/foo" to the $PATH variable.
After opening a new terminal session. You should be able to execute the script without the "./" and ".py".
By the way "./" means that you want to execute a file in the current working directory. It is always possible to execute a file using a full path of the file, for example "/usr/bin/something_to_run".
Please consider reading about PATH variable here.
In my main Python script, I want to call another python script to run, as follows:
python2 ~/script_location/my_side_script.py \ --input-dir folder1/in_folder \ --output-dir folder1/out_folder/ \ --image-ext jpg \
From inside my Python script, how exactly can I do this?
I will be using both Windows and Ubuntu, but primarily the latter. Ideally would like to be able to do on both.
You could import the script in your main file.
Suppose you have two files: myscript.py and main.py
# myscript.py
print('this is my script!')
# main.py
print('this is my main file')
import myscript
print('end')
The output if you run main.py would be:
this is my main file
this is my script
end
EDIT: If you literally just want to call python2 my_side_script.py --options asdf, you could use the subprocess python module:
import subprocess
stdout = subprocess.check_output(['python2', 'my_side_script.py', '--options', 'asdf'])
print(stdout) # will print any output from your sidescript
I am trying to write a script which is executing couple of things on a Linux server and I would like to use bash for most of the Linux specific commands and only use Python for the most complex stuff, but in order to do that I will need to export some variables from the bash script and use them in the python script and I didn't find a way how I can do that. So I have tried to create two very little scripts to test this functionality:
1.sh is a bash script
#!/bin/bash
test_var="Test Variable"
export test_var
echo "1.sh has been executed"
python 2.sh
2.sh is a Python script:
#!/usr/bin/env python3
print("The python script has been invoked successfully")
print(test_var)
As you can guess when I execute the first script the second fails with the error about unknown variable:
$ ./1.sh
1.sh has been executed
The python script has been invoked successfully
Traceback (most recent call last):
File "2.sh", line 4, in <module>
print(test_var)
NameError: name 'test_var' is not defined
The reason why I am trying to do that is because I am more comfortable with bash and I want to use $1, $2 variables in bash. Is this also possible in Python?
[EDIT] - I have just found out how I can use $1 and $2 it in Python. You need to use sys.argv[1] and sys.argv[2] and import the sys module import sys
To use environment variables from your python script you need to call:
import os
os.environ['test_var']
os.environ is a dictionary with all the environment variables, you can use all the method a dict has. For instance, you could write :
os.environ.get('test_var', 'default_value')
Check python extension it should be .py instead of .sh
1.sh
#!/bin/bash
test_var="Test Variable"
export test_var
echo "1.sh has been executed"
python 2.py
os library will gave you the access the environment variable. Following python code will gave you the required result,
#!/usr/bin/env python3
import os
print("The python script has been invoked successfully")
print(os.environ['test_var'])
Check for reference : How do I access environment variables from Python?
I would like to call a shell script from my python script. I need to pass 3 parameters/arguments to the shell script. I am able to call the shell script (that is in the same directory as that of the python script), but having some issue with the parameter passing
from subprocess import call
// other code here.
line = "Hello"
// Here is how I call the shell command
call (["./myscript.sh", "/usr/share/file1.txt", ""/usr/share/file2.txt", line], shell=True)
In my shell script I have this
#!/bin/sh
echo "Parameters are $1 $2 $3"
...
Unfortunately parameters are not getting passed correctly.
I get this message:
Parameters are
None of the parameter values are passed in the script
call ("./myscript.sh /usr/share/file1.txt /usr/share/file2.txt "+line, shell=True)
When you are using shell=True you can directly pass the command as if passing on shell directly.
Drop shell=True (you might need to make myscript.sh executable: $ chmod +x myscript.sh):
#!/usr/bin/env python
from subprocess import check_call
line = "Hello world!"
check_call(["./myscript.sh", "/usr/share/file1.txt", "/usr/share/file2.txt",
line])
Do not use a list argument and shell=True together.