Script A.py (python 3.0) does some work and creates a output.csv file.
Script B.py (python 2.5) takes output.csv as input and does some work.
Example A.py:
Outfile = open (path_to_csv, "w+")
Outfile.write(string_to_be_written)
Outfile.close
subprocess.call(["path/to/B.py", path_to_csv, args+])
Example B.py:
csv_file = open(path_to_csv, "rb")
csv_reader = csv.reader(csv_file)
for row in csv_reader:
print row
Now when I run B.py directly, it works perfectly and prints each row of the csv.
But doesn't read anything when called from A.py with subprocess.call. It doesnt throw any errors while reading either.
I have tried using os.system and subprocess.Popen but they encounter the same problem. What am I missing here?
Outfile.write = open (path_to_csv, "w+") is probably a typo, next line could work as is. You probably mean Outfile = open (path_to_csv, "w+")
Then the problem is that you don't actually close your file:
Outfile.close
accomplishes nothing. You need:
Outfile = open (path_to_csv, "w+")
Outfile.write(string_to_be_written)
Outfile.close()
else the file stays open/unflushed. On Windows, you cannot open it again (locked), on Linux it may contain unflushed data.
The most annoying thing as that when a finishes, the file is properly closed and you see the proper csv output, so it's misleading as why it doesn't work.
Better: use a with block to close file on exiting the block
with open (path_to_csv, "w+") as Outfile:
Outfile.write(string_to_be_written)
# now call the subprocess outside the with block: the file is closed
subprocess.call(["path/to/B.py", path_to_csv, args+])
Even better: rewrite both scripts so output of the first one is consumed by input of the second one (avoids writing a temporary file just to pass data)
Related
I am trying to copy the content of one file to another.
The script successfully copies content to the file but when I try to run READ command with the output file to print the output, it is blank.
from sys import argv
script, inputFile, outputFile = argv
inFile = open(inputFile)
inData = inFile.read()
outFile = open(outputFile, 'w+')
outFile.write(inData)
print("The new data is:\n",outFile.read())
inFile.close()
outFile.close()
After the write operation the file pointer is at the end of file so you'd need to reset it to the start. Also, the filesystem IO buffers may not have been flushed at that point (you haven't closed the file yet)...
Simple solution: close the outFile and reopen it for reading.
As a side note: always make sure you DO close your files whatever happens, specially when writing, else you may end up with corrupted data. The simplest way is the with statement:
with open(...) as infile, (...) as outfile:
outfile.write(infile.read())
# at this point both files have been automagically closed
You forgot to return to the beginning of outFile after writing to it.
So inserting outFile.seek(0) should fix your issues.
After you are done writing, file pointer is at the end of the file, so no data is there. Reposition pointer to start of the file.
I've written a program in which I need 3 arrays (lists) to write to a txt file. When I run the code on my computer, the txt file is empty. I sent it to a friend, who ran it and the program populated the txt file on his computer.
I have next to no experience coding and need the txt file for a homework assignment.
Note: It did run earlier today on my computer, although one of the arrays did not write to the file. However, when I ran it again later (after adding additional print statements earlier in the code for error checking), it again wasn't writing to the txt file. What could cause this type of behavior? My code for writing to the txt file is as follows:
import csv
.....
MyFile = open('BattSim.txt', 'w')
wr = csv.writer(MyFile)
wr.writerow(RedArmy)
wr.writerow(BlueArmy)
wr.writerow(BattleTime)
MyFile.close
Did you run this in an interactive interpreter (or in a non-CPython interpreter or otherwise crash in some weird way)? If so, the problem is that you didn't actually flush/close the file; you referenced the close method without calling it. You wanted MyFile.close() (with parens to call).
Alternatively, you use a with statement to get guaranteed close behavior (even if an exception is thrown midway):
import csv
.....
# Use with statement to autoclose, and newline='' to follow csv module rules
with open('BattSim.txt', 'w', newline='') as MyFile:
wr = csv.writer(MyFile)
wr.writerow(RedArmy)
wr.writerow(BlueArmy)
wr.writerow(BattleTime)
outfile = open(inputfile, 'w')
outfile.write(argument)
outfile.flush()
os.fsync(outfile)
outfile.close
This is the code snippet. I am trying to write something into a file in python. but when we open the file, nothing is written into it. Am i doing anything wrong?
You are not calling the outfile.close method.
No need to flush here, just call close properly:
outfile = open(inputfile, 'w')
outfile.write(argument)
outfile.close()
or better still, use the file object as a context manager:
with open(inputfile, 'w') as outfile:
outfile.write(argument)
This is all presuming that argument is not an empty string, and that you are looking at the right file. If you are using a relative path in inputfile what absolute path is used depends on your current working directory and you could be looking at the wrong file to see if something has been written to it.
Try with
outfile.close()
note the brackets.
outfile.close
would only return the function-object and not really do anything.
You wont see the data you have written into it until you flush or close the file. And in your case, you are not flushing/closing the file properly.
* flush the file and not stdout - So you should invoke it as outfile.flush()
* close is a function. So you should invoke it as outfile.close()
So the correct snippet would be
outfile = open(inputfile, 'w')
outfile.write(argument)
outfile.flush()
outfile.close()
If you are simply planning to write to a file using a python script as show below:
#!/usr/bin/python
count = 1
fo = open('DbCount.txt', 'w')
fo.write(str(count))
#fo.flush()
fo.close()
The Dbcount.txt file which was placed in the same folder as the script(attempting to modify the Dbcount.txt). i dont see any change in the txt file and no error is shown by the interpreter, its very strange, any help ?
first of all, always use the with statement variant, that will always close the file, even on errors:
#!/usr/bin/python
count = 1
with open('DbCount.txt', 'w') as fo:
fo.write(str(count))
then the 'w' overwrites your file each time you write to it. If you want to append, use 'a'.
About your specific problem, did you look only in the directory of your script, or in the current directory you're calling the script from? As you wrote your code, the file's path you write to is relative to where you execute your code from.
try:
import os
count = 1
with open(os.path.join(os.path.dirname(__file__), 'DbCount.txt'), 'w') as fo:
fo.write(str(count))
then it should output DbCount.txt in the same path as your script.
I have a python script that runs a subprocess to get some data and then process it. What I'm trying to achieve is have the data written to a file, and then use the data from the file to do the processing (the reason is that the subprocess is slow, but can change based on the date, time, and parameters I use, and I need to run the script frequently)
I've tried various methods, including opening the file as w+ and trying to seek to the beginning after the write is done, but nothing seems to work - the file is written, but when I try to read back from it (using file.readline()) i get EOF back.
This is what I'm essentially trying to accomplish:
myFile = open(fileName, "w")
p = subprocess.Popen(args, stdout=myFile)
myFile.flush() # force the file to disk
os.fsync(myFile) # ..
myFile.close()
myFile = open(fileName, "r")
while myFile.readline():
pass # do stuff
myFile.close()
But even though the file is correctly written (after the script runs, i can see the contents of the file), readline never returns a valid line. Like I said I also tried using the same file object, and doing seek(0) on it, to no luck. This only worked when opening the file as r+, which fails when the file doesn't already exist.
Any help would be appreciated. Also if there's a cleaner way to do this, i'm open to it :)
PS: I realize I can Popen and stdout to a pipe, read from the pipe and then write line by line the data to the file as I do that, but I'm trying to separate the creation of the data file from the reading.
The subprocess almost certainly isn't finishing before you try to read from the file. In fact, it's likely that the subprocess isn't even writing anything before you try to read from the file. For true separation you're going to have to have the subprocess write to a temporary file then replace the file you read from, so that you either read the previous version or the new version but never get to see the partially-written file from the new version.
You can do this in a number of ways; the easiest would be to change the subprocess, but I don't know if that's an option for you here. Alternatively, you can wrap it in your own separate script to manage the files. You probably don't want to call the subprocess in the script that analyses the output file either; you'll want a cronjob or something to regenerate periodically.
This should work as is provided the subprocess is finishing in time (see James's answer).
If you want to wait for it to finish, add p.wait() after the Popen invocation.
What is your actual while loop, though? while myFile.readline() makes it seem as you're not actually saving the line for anything. Try this:
myFile = open(fileName, "r")
print myFile.readlines()
myFile.close()
Or, if you want to interactively examine the state of your program:
myFile = open(fileName, "r")
import pdb; pdb.set_trace()
myFile.close()
Then you can do things like print myFile.readlines() after it stops.
#James Aylett pointed me to the right path, it appears that my problem was that subprocess.Popen wasn't finished running when I call .flush().
The solution, is to call p.wait() right after the subprocess.Popen call, to allow for the underlying command to finish. After doing that, .flush does the right thing (since all the data is there), and I can proceed to read from the file.
So the above code becomes:
myFile = open(fileName, "w")
p = subprocess.Popen(args, stdout=myFile)
p.wait() # <-- Missing line
myFile.flush() # force the file to disk
os.fsync(myFile) # ..
myFile.close()
myFile = open(fileName, "r")
while myFile.readline():
pass # do stuff
myFile.close()
And then it all works!