For the life of me i can't figure this one out.
I have 2 applications build in python, so 2 projects in different folders, is there a command to say in the first application like run file2 from documents/project2/test2.py ?
i tried something like os.system('') and exec() but that only seems to work if its in the same folder. How can i give a command a path like documents/project2 and then for example:
exec(documents/project2 python test2.py) ?
short version:
Is there a command that runs python test2.py while that test2 is in a completely different file/project?
thnx for all feedback!
There's a number of approaches to take.
1 - Import the .py
If the path to the other Python script can be made relative to your project, you can simply import the .py. This will cause all the code at the 'root' level of the script to be executed and makes functions as well as type and variable definitions available to the script importing it.
Of course, this only works if you control how and where everything is installed. It's the most preferable solution, but only works in limited situations.
import ..other_package.myscript
2 - Evaluate the code
You can load the contents of the Python file like any other text file and execute the contents. This is considered more of a security risk, but given the interpreted nature of Python in normal use not that much worse than an import under normal circumstances.
Here's how:
with open('/path/to/myscript.py', 'r') as f:
exec(f.read())
Note that, if you need to pass values to code inside the script, or out of it, you probably want to use files in this case.
I'd consider this the least preferable solution, due to it being a bit inflexible and not very secure, but it's definitely very easy to set up.
3 - Call it like any other external program
From a Python script, you can call any other executable, that includes Python itself with another script.
Here's how:
from subprocess import run
run('python path/to/myscript.py')
This is generally the preferable way to go about it. You can use the command line to interface with the script, and capture the output.
You can also pipe in text with stdin= or capture the output from the script with stdout=, using subprocess.Popen directly.
For example, take this script, called quote.py
import sys
text = sys.stdin.read()
print(f'In the words of the poet:\n"{text}"')
This takes any text from standard in and prints them with some extra text, to standard out like any Python script. You could call it like this:
dir | python quote.py
To use it from another Python script:
from subprocess import Popen, PIPE
s_in = b'something to say\nright here\non three lines'
p = Popen(['python', 'quote.py'], stdin=PIPE, stdout=PIPE)
s_out, _ = p.communicate(s_in)
print('Here is what the script produced:\n\n', s_out.decode())
Try this:
exec(open("FilePath").read())
It should work if you got the file path correct.
Mac example:
exec(open("/Users/saudalfaris/Desktop/Test.py").read())
Windows example:
exec(open("C:\Projects\Python\Test.py").read())
All the previous posts on this topic deal with specific challenges for their use case. I thought it would be useful to have a post only dealing with the cleanest way to run PowerShell scripts from Python and ask if anyone has an better solution than what I found.
What seems to be the generally accepted solution to get around PowerShell trying to interpret different control characters in your command differently to what's intended is to feed your Powershell command in using a file:
ps = 'powershell.exe -noprofile'
pscommand = 'Invoke-Command -ComputerName serverx -ScriptBlock {cmd.exe \
/c "dir /b C:\}'
psfile = open(pscmdfile.ps1, 'w')
psfile.write(pscommand)
psfile.close()
full_command_string = ps + ' pscmdfile.ps1'
process = subprocess.Popen(full_command_string , shell=True, \
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
When your python code needs to change the parameters for the Powershell command each time you invoke it you end up writing and deleting a lot of temporary files for subprocess.Popen to run. It works perfectly but it's unnecessary and not very clean. It's really nice to be able to tidy up and wanted to get suggestions on any improvements I could make to the solution I found.
Instead of writing a file to disk containing the PS command create a virtual file using the io module. Assuming that the "date" and "server" strings are being fed in as part of a loop or function that contains this code, not including the imports of course:
import subprocess
import io
from string import Template
raw_shellcmd = 'powershell.exe -noprofile '
--start of loop with server and date variables populated--
raw_pslistcmd = r'Invoke-Command -ComputerName $server -ScriptBlock ' \
r'{cmd.exe /c "dir /b C:\folder\$date"}'
pslistcmd_template = Template(raw_pslistcmd)
pslistcmd = pslistcmd_template.substitute(server=server, date=date)
virtualfilepslistcommand = io.BytesIO(pslistcmd)
shellcmd = raw_shellcmd + virtualfilepslistcommand.read()
process = subprocess.Popen(shellcmd, shell=True, stdout=subprocess.PIPE, \
stderr=subprocess.PIPE)
--end of loop--
Arguably the best approach is to use powershell.exe -Command rather than writing the PowerShell command to a file:
pscommand = 'Invoke-Command ...'
process = subprocess.Popen(['powershell.exe', '-NoProfile', '-Command', '"&{' + pscommand + '}"'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Make sure double quotes in the pscommand string are properly escaped.
Note that shell=True is required only in certain edge cases, and should not be used in your scenario. From the documentation:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
After spending a fair amount of time on this.
I think that running powershell commands from python may not make sense to a lot of people, especially people who work exclusively in windows environments. There are numerous clear advantages to python over powershell however so the ability to do all your business logic in python and then selectively execute powershell on remote servers is truly a great thing.
I've now been through several improvements of my "winrmcntl" module which I can't share due to company policy unfortunately but here is my advice to anyone who would like to do something similar. The module should take as input an unmodified PS command or scriptblock as you'd run it if you were typing directly in PS on the destination box. A few tricks:
To avoid permission difficulties, ensure the user running your python script and hence the one running powershell.exe via process.Popen is the user that has the correct permissions on the windows box you're invoke-command is pointing at. We use an enterprise scheduler which has windows vms as agents on which the python code lives which takes care of that.
You will sometimes rarely but still get the odd esoteric exception from powershell land, if they're anything like the one in particular I saw the odd time, microsoft scratch their heads at a little and get you to do time consuming application stack tracing. This is not only time consuming but very difficult to get right because it's resource intensive and you don't know when the exception will next occur. In my opinion, it's much better and easier to parse the output of the exception and retry up to x number of times if a certain text appears in those exceptions. I keep a list of strings in my winrmcntl module which currently contains a single string.
If you want to not have to "massage" the powershell commands as they traverse the python -> windows -> powershell -> powershell stack to make them work as expected on destination boxes, the most consistent method I've found is to write your one liners and scriptblocks alike into a ps_buffer.ps1 file which you then feed to powershell on the source box so that every process.popen looks exactly the same but the content of ps_buffer.ps1 changes with each execution.
powershell.exe ps_buffer.ps1
To keep your python code nice and clean, it's great having your list of powershell one liners in a json file or similar as well as pointers to scriptblocks you want to run saved into static files. You load up your json file as an ordered dict and cycle through issuing commands based on what you're doing.
Can't be overstated, as far as is possible try to be on the latest stable version of PS but more than that, it's imperative to be on the same version on client and server.
"scriptblock" and "server" are the values fed to this module or function
import subprocess
from string import Template
scriptblock = 'Get-ChildItem' #or a PS scriptblock as elaborate as you need
server = 'serverx'
psbufferfile = os.path.join(tempdir, 'pscmdbufferfile_{}.ps1'.format(server))
fullshellcmd = 'powershell.exe {}'.format(psbufferfile)
raw_pscommad = 'Invoke-Command -ComputerName $server -ScriptBlock {$scriptblock}'
pscmd_template = Template(raw_pscommand)
pscmd = pscmd_template.substitute(server=server, scriptblock=scriptblock)
try:
with open(psbufferfile, 'w') as psbf:
psbf.writelines(pscmd)
....
try:
process = subprocess.Popen(fullshellcmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = process.communicate()
....
I am building a python utility which automates sysadmin type tasks. Part of the tool involves writing scripts and then calling them with powershell from the python interface. An example of such code is this:
def remote_ps_session():
target = raw_input("Enter your target hostname: ")
print "Creating target.ps1 file to establish connection"
pstarget = open("pstarget.ps1", "w")
pstarget.write("$target = New-Pssession " + target + "\n")
pstarget.write("Enter-PSSession $target" + "\n")
pstarget.close()
print "File created. Initiating Connection to remote host..."
os.system("powershell -noexit -ExecutionPolicy Unrestricted " + "C:\path\to\my\file\pstarget.ps1")
I would like to do two things which I think can be answered with the same method, I've just yet to find out what is best (importing vs variables vs initial setup definitions and so on)
For simplicity we'll say the utility is in C:\utility and the powershell functions are in a functions folder one level deeper: C:\utility\functions
I want to be able to specify a location for 1) where the script (the file that is written) is saved to and then 2) refer to that location when the os.system call is made. I want this to be able to run on most/any modern Windows system.
My thoughts on possibilities are:
When the script launches get the current directory and save that as a variable, if I need to go back a directory take that variable and remove everything after the last \ and so on. Doesn't seem ideal.
On the first launch of the file prompt for system locations to put in variables. For instance have it prompt 'where do you want your log files?' 'where do you want your output files?' 'where do you want your generated scripts?' These could then be referred to as variables but would break if they ever moved folders and may not be easy to 'fix' for a user.
I imagine there is some way to refer to current directories and navigate to ..\parallel folder to where I am executing from. ....\2 folders up, but that also seems like it might be messy. I've yet to see what a standard/best practice for managing this is.
Edit: based on some comments I think __file__ might be the place to start looking. I'm going to dig in to this some but any examples (for example: __file__/subfoldernameor whatever the usage would be would be cool.
Python has a lib dedicated to path manipulation os.path, so anytime you need filesystem paths manipulation take a look at it.
As for your particular questions, run the following example, to see how you can use the functions from this lib:
test.py
import os
# These two should basicly be the same,
# but `realpath` resolves symlinks
this_file_absolute_path = os.path.abspath(__file__)
this_file_absolute_path1 = os.path.realpath(__file__)
print(this_file_absolute_path)
print(this_file_absolute_path1)
this_files_directory_absolute_path = os.path.dirname(this_file_absolute_path)
print(this_files_directory_absolute_path)
other_script_file_relative_path = "functions/some.ps"
print(other_script_file_relative_path)
other_script_file_absolute_path = os.path.join(this_files_directory_absolute_path,
other_script_file_relative_path)
print(other_script_file_absolute_path)
print("powershell -noexit -ExecutionPolicy Unrestricted %s" %
other_script_file_absolute_path)
You should get output similar to this:
/proj/test_folder/test.py
/home/user/projects/test_folder/test.py
/proj/test_folder
functions/some.ps
/proj/test_folder/functions/some.ps
powershell -noexit -ExecutionPolicy Unrestricted /proj/test_folder/functions/some.ps
I want to run a command with os.system but i get an error
c:/fe ' is not recognized as an internal or external command, operable program or batch file
The code I use is
import os
os.system('"C:\\fe re\\python.exe" program "c:\\test now\\test.txt" http://site.to.explore')
It will work if I only run:
import os
os.system('"C:\\fe re\\python.exe" program -h')
Or if I have no space in the python path like this
import os
os.system('C:\\fere\\python.exe program "c:\\test now\\test.txt" http://site.to.explore')
But if I have two pairs of double-quotes in the command both in python path and in txt path I get an error...
os.system has some serious drawbacks, especially with space in filenames and w.r.t. security. I suggest you look into the subprocess module and particularly subprocess.check_call, which is much more powerful. You could then do e.g.
import subprocess
subprocess.check_call(["c:\\fe re\\python.exe", "program", etcetc...])
Of course, make sure to take great care not to have user-determined variables in these calls unless the user is already running the script herself from the command line with the same privileges.
I don't agree with the subprocess module being the accepted answer.
I can make the same call to the system in ANY other language and not have one single issue. This is broken in python.
Here is my simple solution though to getting around this:
os.chdir("C:\\Program Files\\7-Zip")
os.system("7z.exe e \"{}\" -o\"{}\"\n".format(os.path.join(directory, file), self.out))
You simply change to the directory of your executable, then run the os.system command as normal.
I just started working on Python, and I have been trying to run an outside executable from Python.
I have an executable for a program written in Fortran. Let’s say the name for the executable is flow.exe. And my executable is located in C:\Documents and Settings\flow_model. I tried both os.system and popen commands, but so far I couldn't make it work. The following code seems like it opens the command window, but it wouldn't execute the model.
# Import system modules
import sys, string, os, arcgisscripting
os.system("C:/Documents and Settings/flow_model/flow.exe")
How can I fix this?
If using Python 2.7 or higher (especially prior to Python 3.5) you can use the following:
import subprocess
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Runs the command described by args. Waits for command to complete, then returns the returncode attribute.
subprocess.check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Runs command with arguments. Waits for command to complete. If the return code was zero then returns, otherwise raises CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute
Example: subprocess.check_call([r"C:\pathToYourProgram\yourProgram.exe", "your", "arguments", "comma", "separated"])
In regular Python strings, the \U character combination signals a
extended Unicode code point escape.
Here is the link to the documentation: http://docs.python.org/3.2/library/subprocess.html
For Python 3.5+ you can now use run() in many cases: https://docs.python.org/3.5/library/subprocess.html#subprocess.run
Those whitespaces can really be a bother. Try os.chdir('C:/Documents\ and\ Settings/') followed by relative paths for os.system, subprocess methods, or whatever...
If best-effort attempts to bypass the whitespaces-in-path hurdle keep failing, then my next best suggestion is to avoid having blanks in your crucial paths. Couldn't you make a blanks-less directory, copy the crucial .exe file there, and try that? Are those havoc-wrecking space absolutely essential to your well-being...?
The simplest way is:
import os
os.startfile("C:\Documents and Settings\flow_model\flow.exe")
It works; I tried it.
I'd try inserting an 'r' in front of your path if I were you, to indicate that it's a raw string - and then you won't have to use forward slashes. For example:
os.system(r"C:\Documents and Settings\flow_model\flow.exe")
Your usage is correct. I bet that your external program, flow.exe, needs to be executed in its directory, because it accesses some external files stored there.
So you might try:
import sys, string, os, arcgisscripting
os.chdir('c:\\documents and settings\\flow_model')
os.system('"C:\\Documents and Settings\\flow_model\\flow.exe"')
(Beware of the double quotes inside the single quotes...)
Use subprocess, it is a smaller module so it runs the .exe quicker.
import subprocess
subprocess.Popen([r"U:\Year 8\kerbal space program\KSP.exe"])
By using os.system:
import os
os.system(r'"C:/Documents and Settings/flow_model/flow.exe"')
Try
import subprocess
subprocess.call(["C:/Documents and Settings/flow_model/flow.exe"])
If it were me, I'd put the EXE file in the root directory (C:) and see if it works like that. If so, it's probably the (already mentioned) spaces in the directory name. If not, it may be some environment variables.
Also, try to check you stderr (using an earlier answer by int3):
import subprocess
process = subprocess.Popen(["C:/Documents and Settings/flow_model/flow.exe"], \
stderr = subprocess.PIPE)
if process.stderr:
print process.stderr.readlines()
The code might not be entirely correct as I usually don't use Popen or Windows, but should give the idea. It might well be that the error message is on the error stream.
in python 2.6 use string enclosed inside quotation " and apostrophe ' marks. Also a change single / to double //.
Your working example will look like this:
import os
os.system("'C://Documents and Settings//flow_model//flow.exe'")
Also You can use any parameters if Your program ingest them.
os.system('C://"Program Files (x86)"//Maxima-gcl-5.37.3//gnuplot//bin//gnuplot -e "plot [-10:10] sin(x),atan(x),cos(atan(x)); pause mouse"')
finally You can use string variable, as an example is plotting using gnuplot directly from python:
this_program='C://"Program Files (x86)"//Maxima-gcl-5.37.3//gnuplot//bin//gnuplot'
this_par='-e "set polar; plot [-2*pi:2*pi] [-3:3] [-3:3] t*sin(t); pause -1"'
os.system(this_program+" "+this_par)
import os
path = "C:/Documents and Settings/flow_model/"
os.chdir(path)
os.system("flow.exe")
Note added by barlop
A commenter asked why this works. Here is why.
The OP's problem is os.system("...") doesn't work properly when there is a space in the path. (Note os.system can work with ('"...."') but anyhow)
Had the OP tried their program from a cmd prompt they'd have seen the error clearly.
C:\carp>type blah.py
import os
os.system(R"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")
C:\carp>python blah.py
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
C:\carp>
So it's fine for os.system("calc.exe") (there calc.exe is in the path environment variable). Or for os.system(R"c:\windows\system32\calc.exe"). There's no space in that path.
C:\>md "aa bb cc"
C:\>copy c:\windows\system32\calc.exe "c:\aa bb cc\cccalc.exe"
1 file(s) copied.
This works (Given file "c:\aa bb cc\cccalc.exe" )
import os
os.chdir(R"c:\aa bb cc")
os.system("cccalc.exe")
Other options are subprocess.run and subprocess.popen.
Is that trying to execute C:\Documents with arguments of "and", "Settings/flow_model/flow.exe"?
Also, you might consider subprocess.call().
There are loads of different solutions, and the results will strongly depend on:
the OS you are using: Windows, Cygwin, Linux, MacOS
the python version you are using: Python2 or Python3x
As I have discovered some things that are claimed to work only in Windows, doesn't, probably because I happen to use Cygwin which is outsmarting the OS way to deal with Windows paths. Other things only work in pure *nix based OS's or in Python2 or 3.
Here are my findings:
Generally speaking, os.system() is the most forgiving method.
os.startfile() is the least forgiving. (Windows only && if you're lucky)
subprocess.Popen([...]) not recommended
subprocess.run(winView, shell=True) the recommended way!
Remembering that using subprocess for anything may pose a security risk.
Try these:
import os, subprocess
...
winView = '/cygdrive/c/Windows/explorer.exe %s' % somefile
...
# chose one of these:
os.system(winView)
subprocess.Popen(['/cygdrive/c/Windows/explorer.exe', 'somefile.png'])
subprocess.run(winView, shell=True)
Q: Why would you want to use explorer in Windows?
A: Because if you just want to look at the results of some new file, explorer will automatically open the file with whatever default windows program you have set for that file type. So no need to re-specify the default program to use.
That's the correct usage, but perhaps the spaces in the path name are messing things up for some reason.
You may want to run the program under cmd.exe as well so you can see any output from flow.exe that might be indicating an error.
for the above question this solution works.
just change the path to where your executable file is located.
import sys, string, os
os.chdir('C:\\Downloads\\xpdf-tools-win-4.00\\xpdf-tools-win-4.00\\bin64')
os.system("C:\\Downloads\\xpdf-tools-win-4.00\\xpdf-tools-win-4.00\bin64\\flowwork.exe")
'''import sys, string, os
os.chdir('C:\\Downloads\\xpdf-tools-win-4.00\\xpdf-tools-win-4.00\\bin64')
os.system(r"C:\\Downloads\\xpdf-tools-win-4.00\\xpdf-tools-win-4.00\bin64\\pdftopng.exe test1.pdf rootimage")'''
Here test1.pdf rootimage is for my code .