I am trying to run 3 python programs simultaneously by running a single python program
I am using the following script in a separate python program sample.py
Sample.py:
import subprocess
subprocess.Popen(['AppFlatRent.py'])
subprocess.Popen(['AppForSale.py'])
subprocess.Popen(['LandForSale.py'])
All the three programs including python.py is in the same folder.
Error: OSError: [Errno 2] No such file or directory
Can someone guide me how can i do it using subprocess.Popen method?
The file cannot be found because the current working directory has not been set properly. Use the argument cwd="/path/to/script" in Popen
It's because your script are not in the current directory when you execute sample.py.
If you three script are in the same directory than sample.py, you could use :
import os
import subprocess
DIR = os.path.dirname(os.path.realpath(__file__))
def run(script):
url = os.path.join(DIR, script)
subprocess.Popen([url])
map(run, ['AppFlatRent.py','AppForSale.py', 'LandForSale.py'])
But honestly, if i was you i will do it using a bash script.
There might be shebang missing (#!..) in some of the scripts or executable permission is not set (chmod +x).
You could provide Python executable explicitly:
#!/usr/bin/env python
import inspect
import os
import sys
from subprocess import Popen
scripts = ['AppFlatRent.py', 'AppForSale.py', 'LandForSale.py']
def realpath(filename):
dir = os.path.realpath(os.path.dirname(inspect.getsourcefile(realpath)))
return os.path.join(dir, filename)
# start child processes
processes = [Popen([sys.executable or 'python', realpath(scriptname)])
for scriptname in scripts]
# wait for processes to complete
for p in processes:
p.wait()
The above assumes that script names are given relative to the module.
Consider importing the modules and running corresponding functions concurently using threading, multiprocessing modules instead of running them as scripts directly.
Related
I have two scripts : main.py and worker.py
main.py is located in main folder.
worker.py is located in worker folder.
worker.py read file (parameters.txt) which is located in his folder, in a subfolder called data.
If I execute my worker.py, which is just reading the parameters.txt file, using the path 'data/parameters.txt'. It works great.
If I call the worker.py from the main using
command = ['python', '../worker/worker.py']
subprocess.call(command)
I got an error.
File "../main/main.py", line 11, in getTxt
input_file = open(file_name, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/data/parameters.txt'
I feel like when I call the subprocess from the main.py, it is not creating a instance of the py script, like when I'm executing 'python worker.py' from worker folder, so it mixes the path.
For exemple, I feel like it's not looking for the parameters.txt in the worker folder, but in the main folder.
How to solve this ? Knowing my worker.py is 100% independent from main.py talking about variables and arguments. But need to be call from main.py
This is different than importing or exec the worker.py. I want to execute multiple worker.py at the same time and they automaticaly close themselves when they are done.
Many thanks
subprocess has a cwd variable you can tell it where to execute the script from. This is how I solved your problem:
import subprocess
from os import getcwd, chdir
from os.path import join
CURRENT_DIR: str = getcwd()
chdir(CURRENT_DIR)
LOCATION = join(CURRENT_DIR, "worker")
command = ["python", "worker.py"]
subprocess.call(command, cwd=LOCATION)
worker.py - for testing purpose:
with open("parameters/test.txt") as f:
print(f.read())
Result:
hello!
This was the contents of test.txt for testing.
You may need to change the current working directory
Get current path:
import os
current_path = os.getcwd()
Change the current working directory
os.chdir('../worker')
Run your process:
command = ['python', 'worker.py']
subprocess.call(command)
Finally get back to your previous path
os.chdir(current_path)
I am trying to run a .sh script from python.
I saw that this can be done in various ways such as:
import subprocess
subprocess.call(["./test.sh"])
or
import os
os.system("sh test.sh")
However this assumes that test.sh is in the same folder where you are running the script from. What if I want to run the .sh which is in a specific folder?
I tried the following but with no luck:
import subprocess
subprocess.call(["cd ~/ros_ws", "./intera.sh"])
import subprocess
subprocess.call(["cd ~/ros_ws", "./intera.sh"], shell=True)
Thanks for the help.
The subprocess.call has an cwd function argument (change working directory)
import subprocess
subprocess.call(["./intera.sh"], cwd="~/ros_ws")
My django project fetches credentials from environment variables, now I want to automate this process and store the credentials in the vault(hashivcorp).
I have a python and shell script which fetches data from an API and exports it as environment variables, when I run it using os.system command it runs the shell script but as it runs it in a subprocess, I can't access the variables in the main(parent) process/shell. Only way of doing it by inserting the shell script in the settings.py file.
Is there any way I can do it so that I get those in the main process?
P.s: I did try sourcing, os.system didn't recognise it as a command.
Here's the code I'm running:
import os
os.environ['ENV'] = 'Demo'
os.system('python3 /home/rishabh/export.py')
print(os.environ.get('RDS_DB_NAME'))
output:
None
the python file, shell script works just fine.
One way to do it is to run export.py in the same process, as user1934428 suggested:
import os
import sys
os.environ['ENV'] = 'Demo'
sys.path.append('/home/rishabh/')
import export # runs export.py in the same process
print(os.environ.get('RDS_DB_NAME'))
This assumes there are no __name__ == '__main__' checks inside export.py.
You only need the sys.path line if export.py is in a different directory than your current script.
How would I open a specific file in IDLE through a python script?
I understand that an app could be opened through subprocess:
import subprocess
subprocess.call('C:\\program.exe')
But I can't figure out how to make it open a file.
If it helps, this:
import os.path
import sys
# Enable running IDLE with idlelib in a non-standard location.
# This was once used to run development versions of IDLE.
# Because PEP 434 declared idle.py a public interface,
# removal should require deprecation.
idlelib_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if idlelib_dir not in sys.path:
sys.path.insert(0, idlelib_dir)
from idlelib.pyshell import main
main()
also opens IDLE. I checked, and main() does not take any parameters such as files to open.
I am using Windows 10 with Python 3.6.4.
Any help is greatly appreciated.
Here are 2 ways to open any python file through IDLE
import subprocess
p = subprocess.Popen(["idle.exe", path_to_file])
# ... do other things while idle is running
returncode = p.wait() # wait for notepad to exit
OR:
import subprocess
import os
subprocess.call([path_to_idle, path_to_file])
You can also use these methods to open any file with any installed app (How can I open files in external programs in Python?)
One can run IDLE from a command line on any platform with <python> -m idlelib <IDLE args>, where <python> could be 'python', 'python3', or something line 'py -3.8', depending on the platform. <IDLE args> are defined in the "Command line usage" subsection of the IDLE doc, also available within IDLE as Help => IDLE Help.
A possible 'IDLE arg' is the path of a file to be opened in an editor window. Relative paths are relative to the current working directory, which can be changed with the 'cd' command. A working command line can used quoted or turned into a list for a subprocess.run or subprocess.Popen call. In Python code, the working directory is changed with os.chdir('newdir').
I am working on an auto restart script for an exe but there are config files in the directory I need to run with the exe. If I start the exe from the actual file folder it works fine but when I use the script to run the exe it starts the exe without the configuration files. What can I add to this script to use all files in the directory?
import os, subprocess, time
while True:
print("Starting process...")
p = subprocess.Popen("C:\\Users\\my-pc\\Desktop\\process\\process.exe")
time.sleep(7200)
print("Terminating process...")
p.terminate()
time.sleep(10)
You should set cwd parameter of Popen constructor to the working directory of the process, e.g.:
p = subprocess.Popen("C:\\Users\\my-pc\\Desktop\\process\\process.exe", cwd="C:\\Users\\my-pc\\Desktop\\process")
You may also find useful the official documentation of subprocess.Popen.
You should be able to call Popen with a "cwd" keyword argument that should pickup the configuration files.