I have a nonpython program I am running with python using the os.system command, but I put this command inside a function. The program I want to run with os.system is supposed to give me an output file, and I need that output for processing, also I need that output to be actually written in the directory I am sending it to.
I wrote my function is the following general format
def myFunction(infile):
os.system('myProgram '+infile+' '+outfileName)
outfile = numpy.loadtxt(outfileName)
return outfile
However, the output of myProgram (outfileName) isn't being written to my directory and numpy can't therefore load it. Is there a way to store globally outputs of programs I run using os.system when it's inside a function?
Assuming myProgram is working correctly, this is likely happening because myProgram does not know the python path, so the file is simply being written somewhere else. Try using the full paths and see if that works.
Assuming infile and outfileName are relative paths in your current working directory, you could do:
def myFunction(infile):
cmd = 'myProgram ' + os.path.join(os.getcwd(), infile)
cmd += ' ' + os.path.join(os.getcwd(), outfileName))
os.system(cmd)
outfile = numpy.loadtxt(outfileName)
return outfile
Related
I'm trying to run a script that works without issue when I run using in console, but causes issue if I try to run it from another directory (via IPython %run <script.py>)
The issue comes from this line, where it references a folder called "Pickles".
with open('Pickles/'+name+'_'+date.strftime('%y-%b-%d'),'rb') as f:
obj = pickle.load(f)
In Console:
python script.py <---works!
In running IPython (Jupyter) in another folder, it causes a FileNotFound exception.
How can I make any path references within my scripts more robust, without putting the whole extended path?
Thanks in advance!
Since running in the console the way you show works, the Pickles directory must be in the same directory as the script. You can make use of this fact so that you don't have to hard code the location of the Pickles directory, but also don't have to worry about setting the "current working directory" to be the directory containing Pickles, which is what your current code requires you to do.
Here's how to make your code work no matter where you run it from:
with open(os.path.join(os.path.dirname(__file__), 'Pickles', name + '_' + date.strftime('%y-%b-%d')), 'rb') as f:
obj = pickle.load(f)
os.path.dirname(__file__) provides the path to the directory containing the script that is currently running.
Generally speaking, it's a good practice to always fully specify the locations of things you interact with in the filesystem. A common way to do this as shown here.
UPDATE: I updated my answer to be more correct by not assuming a specific path separator character. I had chosen to use '/' only because the original code in the question already did this. It is also the case that the code given in the original question, and the code I gave originally, will work fine on Windows. The open() function will accept either type of path separator and will do the right thing on Windows.
You have to use absolute paths. Also to be cross platform use join:
First get the path of your script using the variable __file__
Get the directory of this file with os.path.dirname(__file__)
Get your relative path with os.path.join(os.path.dirname(__file__), "Pickles", f"{name}_{date.strftime('%y-%b-%d')}")
it gives you:
with open(os.path.join(os.path.dirname(__file__), "Pickles", f"{name}_{date.strftime('%y-%b-%d')}"), 'rb') as f:
obj = pickle.load(f)
I have the below python script which takes a user input that is eventually used by the program to read a particular file.
I want to execute the python program from batch script and pass the file_name in the batch script. Can someone please help?
file_name = input("Input File Name to Compare: ")
path = ("outward\\" + file_name)
import sys
file_name = sys.argv[1]
path = "outward\\" + file_name
and you pass it to your script like:
$ python script.py filename.ext
To run the Python program from the batch script and pass the filename into the batch script, you need to use sys, so:
import sys
sys.argv is automatically a list of strings representing arguments on the command line. You can use this as an input for your program. Represents the first command line argument (like a string) given to the script in question.
The lists are indexed by numbers based on zero, so you can get the individual items using the syntax [0]. To get the script name, you need to get the first argument after the script for a filename, so:
filename = sys.argv[1]
path = "outward\\" + file_name
Next you need to pass it to your script like this:
$ python your_script.py filename.ext
(not to be confused with .ext with .text)
Your complete code for the solution to the question, quite simply, will be:
import sys
filename = sys.argv[1]
path = "outward\\" + file_name
and
$ python your_script.py filename.ext
You could use the sys.argv Python function.
The idea would be to get the input from the user in the batch file, then execute the Python program from the batch file while passing the user input as a command line argument to the python file. Example:
batchfile.bat
#echo off
set /p file="Enter Filename: "
python /path/to/program/pythonprogram.py %file%
pythonprogram.py
import sys
sys.argv[0] = file_name
path = ("outward\\" + file_name)
Now, when you execute the batch file, it will prompt for user input of a filename. Then, it will execute the Python program while passing the filename as a command-line argument to the Python file. Then using the sys.argv function you can collect the argument.
I have a program that relies on user input to enter files for the program to open in Python 2.7.11. I have all of those files in a sub-directory called TestCases within the original directory Detector, but I can't seem to access the files in TestCases when running the program from the super-directory. I tried to use os.path.join but to of no avail. Here is my code:
import os.path
def __init__(self):
self.file = None
os.path.join('Detector', 'TestCases')
while self.file == None:
self.input = raw_input('What file to open? ')
try:
self.file = open(self.input, 'r')
except:
print "Can't find file."
My terminal when I run the program goes as follows:
>>> What file to open? test.txt # From the TestCases directory
>>> Can't find file.
>>> What file to open? ...
Am I using os.path.join incorrectly? I thought it was supposed to link the two directories so that files could be accessed from the sub-directory while running the program from the super-directory.
You are using os.path.join('Detector', 'TestCases'), that should return 'Detector/TestCases', but you aren't storing that variable anywhere.
I suppose that you are in Detector directory and you want to open files in TestCases. I that case you can use path join (It concatenates its arguments and RETURNS the result):
import os.path
file = None
while not file:
input = raw_input('What file to open? ')
try:
filepath = os.path.join('TestCases', input)
file = open(filepath, 'r')
except IOError:
print "Can't find " + input
I have stored the result of os.path.join so you could see that it doesn't change the directory, it just concatenates its arguments, maybe you was thinking that function will change the directory, you can do it with os.chdir.
Try it first in a simple script or in the terminal, it will save many headaches.
The documentation about os.path.join
Join one or more path components intelligently. The return value is the concatenation of path...
It seems like you expect it to set some kind of PATH variable or affect the current working directory. For a first start it should be sufficient to add something like this to your code:
open(os.path.join("TestCases",self.input), 'r')
1. Introduction
I have a bunch of files in netcdf format.
Each file contain the meteorology condition of somewhere in different period(hourly data).
I need to extract the first 12 h data for each file. So I select to use NCO(netcdf operator) to deal with.
NCO works with terminal environment. With >ncks -d Time 0,11 input.nc output.nc, I can get one datafile called out.ncwhich contain the first 12h data of in.nc.
2. My attempt
I want to keep all the process inside my ipython notebook. But I stuck on two aspects.
How to execute terminal code in python loop
How to transfer the string in python into terminal code.
Here is my fake code for example.
files = os.listdir('.')
for file in files:
filename,extname = os.path.splitext(file)
if extname == '.nc':
output = filename + "_0-12_" + extname
## The code below was my attempt
!ncks -d Time 0,11 file output`
3. Conclusion
Basically, my target was letting the fake code !ncks -d Time 0,11 file output coming true. That means:
execute netcdf operator directly in python loop...
...using filename which is an string in python environment.
Sorry for my unclear question. Any advice would be appreciated!
You can use subprocess.check_output to execute external program:
import glob
import subprocess
for fn in glob.iglob('*.nc'):
filename, extname = os.path.splitext(fn)
output_fn = filename + "_0-12_" + extname
output = subprocess.call(['ncks', '-d', 'Time', '0,11', fn, output_fn])
print(output)
NOTE: updated the code to use glob.iglob; you don't need to check extension manually.
You may also check out pynco which wraps the NCO with subprocess calls, similar to #falsetru's answer. Your application may look something like
nco = Nco()
for fn in glob.iglob('*.nc'):
filename, extname = os.path.splitext(fn)
output_fn = filename + "_0-12_" + extname
nco.ncks(input=filename, output=output_fn, dimension='Time 0,11')
I am working on Linux with python 2.7.x and I am running some programs python through terminal. I want the certain output should be written in a file located at different directory than my working directory. So I wrote this piece of code. However, what is happening is file All.txt is being created in current directory instead of the desired directory. Can someone help me where I went wrong?
ResultDir = '/pr/p1/ap11/'
os.system('cd ' + ResultDir)
Outputname1 = 'All.txt'
Output1 = open(Outputname1, 'a')
Output1.write('hello' +'\n')
Output1.close()
Changing the current directory with os.system will not affect the Python process that’s running. Just open the file with its full path directly:
with open('/pr/p1/ap11/All.txt', 'a') as output:
output.write('hello\n')