Current program:
#!/usr/bin/python
lookup = 'Loop time'
with open('log.lammps') as myFile:
found = []
for num, line in enumerate(myFile, 1):
if lookup in line:
print 'found at line:', num
found.append(num)
print found
a = int(found[0])
b = int(found[1])
c = int(found[2])
d = int(found[3])
lookup = 'Memory usage per processor ='
with open('log.lammps') as myFile:
found2 = []
for num, line in enumerate(myFile, 1):
if lookup in line:
print 'found at line:', num
found2.append(num)
print found2
e = int(found2[0])
f = int(found2[1])
g = int(found2[2])
h = int(found2[3])
x = 0
while x < len(found):
a = found[x]
e = found2[x]
print "Some thing useful!"
for w,line in enumerate(open("log.lammps")):
if w >= e and w < a :
print line,
x += 1
How do i make it store the last while function into a document. I want to it make a file with the data that this program prints out so that I can have matlab read that file.
The easiest solution is to redirect the output to a file. Let script.py be your python script:
./script.py > out.txt
Otherwise you can redirect the stdout to a file directly in python.
#!/usr/bin/python
import sys
sys.stdout = open('out.txt', 'w')
lookup = 'Loop time'
...
Now every print statement will be redirected to out.txt
One very fast way is to replace sys.stdout with an open file descriptor.
f = open("file.txt","w")
old_stdout = sys.stdout
sys.stdout = f
while x < len(found):
a = found[x]
e = found2[x]
print "Some thing useful!"
for w,line in enumerate(open("log.lammps")):
if w >= e and w < a :
print line,
x += 1
sys.stdout = old_stdout
f.close()
This will automatically redirect anything you print to file.txt instead of stdout. It's also a little hackish, though. It would be a better idea just to open the write file and then replace print ... with f.write(...) wherever you see it, since you don't have any functions that do complex printing.
(I'm assuming you only want the last while loop written to a file, as your question stated.)
Open a file at the start and write the data as you go:
with open('output.txt',"w") as out:
Wherever you print data just write it to a file also:
print 'found at line: {}'.format(num)
out.write('found at line: {}\n'.format(num)) etc..
Just start your code like:
with open('output.txt',"w") as out:
with open('log.lammps') as myFile:
# continue with your code
The important thing about sys.stdout is that you have to import sys instead of from sys import stdout
i.e. this won't redirect your output to a file:
>>> from sys import stdout
>>> stdout = open('/dev/null', 'w')
>>> print 1
1 # not working
Related
Could someone please explain why this code won't write to a text file?
I run the code but a text file is not created and also cannot be opened using f.open("data.txt","r")
f = open("data.txt", "w+")
n = 1
s = 0
for n in range(1, 999999):
s += 1/n**2
print(s, end="\r")
x = s*6
pisquare = math.sqrt(x)
f.write("Pi is ", pisquare)
f.close()
The recommended way to open and write to a file is as follow:
with open(file_path, 'w') as file:
file.write(f'Pi is {pisquare}')
We use the context manager with so the file closes automatically when done with .write(). This prevents memory corruption when your program exits prematurely I believe.
However, as you have probably noticed, your problem comes from this line:
f.write("Pi is ", pisquare)
You are giving .write() two arguments rather than one string.
import math
f = open("data.txt", "w+")
n = 1
s = 0
for n in range(1, 999999):
s += 1/n**2
print(s, end="\r")
x = s*6
pisquare = math.sqrt(x)
f.write("Pi is " + str(pisquare))
f.close()
I am able to create the text file. Please check for it in your current directory. But if I understand your code correctly, this is what you are looking for -
import math
n = 1
s = 0
with open("data.txt", "w+") as f:
for n in range(1, 9999):
s += 1/n**2
print(s, end="\r")
x = s*6
pisquare = math.sqrt(x)
f.write(" Pi is " + str(pisquare) + "\n")
I want to create python script which can modify code in that script itself using Python Language Services or using any other way.
e.g. A script which keep track of its count of successfull execution
import re
COUNT = 0
def updateCount():
# code to update second line e.g. COUNT = 0
pass
if __name__ == '__main__':
print('This script has run {} times'.format(COUNT))
updateCount()
On successful execution of this script code should get changed to
import re
COUNT = 1
def updateCount():
# code to update second line e.g. COUNT = 0
pass
if __name__ == '__main__':
print('This script has run {} times'.format(COUNT))
updateCount()
Simple approach came to my mind was to open __file__ in write mode and do requried modification using reguler expessions etc. But that did not work I got exception io.UnsupportedOperation: not readable. Even if this approach would be working then it would be very risky because it can spoil my whole script. so I am looking for solution using Python Language Services.
Yes, you can use the language services to achieve self-modification, as in following example:
>>> def foo(): print("original foo")
>>> foo()
original foo
>>> rewrite_txt="def foo(): print('I am new foo')"
>>> newcode=compile(rewrite_text,"",'exec')
>>> eval(newcode)
>>> foo()
I am new foo
So, by new dynamically generated code you can replace stuff contained in the original source file, without modifying the file itself.
A python script is nothing more than a text file. So, you are able to open it as an external file and read & write on that. (Using __file__ variable you can get the exact name of your script):
def updateCount():
fin = open(__file__, 'r')
code = fin.read()
fin.close()
second_line = code.split('\n')[1]
second_line_parts = second_line.split(' ')
second_line_parts[2] = str(int(second_line_parts[2])+1)
second_line = ' '.join(second_line_parts)
lines = code.split('\n')
lines[1] = second_line
code = '\n'.join(lines)
fout = open(__file__, 'w')
fout.write(code)
fout.close()
#kyriakosSt's answer works but hard-codes that the assignment to COUNT must be on the second line, which can be prone to unexpected behaviors over time when the line number changes due to the source being modified for something else.
For a more robust solution, you can use lib2to3 to parse and update the source code instead, by subclassing lib2to3.refactor.RefactoringTool to refactor the code using a fixer that is a subclass of lib2to3.fixer_base.BaseFix with a pattern that looks for an expression statement with the pattern 'COUNT' '=' any, and a transform method that updates the last child node by incrementing its integer value:
from lib2to3 import fixer_base, refactor
COUNT = 0 # this should be incremented every time the script runs
class IncrementCount(fixer_base.BaseFix):
PATTERN = "expr_stmt< 'COUNT' '=' any >"
def transform(self, node, results):
node.children[-1].value = str(int(node.children[-1].value) + 1)
return node
class Refactor(refactor.RefactoringTool):
def __init__(self, fixers):
self._fixers = [cls(None, None) for cls in fixers]
super().__init__(None)
def get_fixers(self):
return self._fixers, []
with open(__file__, 'r+') as file:
source = str(Refactor([IncrementCount]).refactor_string(file.read(), ''))
file.seek(0)
file.write(source)
Demo: https://repl.it/#blhsing/MushyStrangeClosedsource
This will edit the module level variables defined before _local_config. Later, process an update to the dictionary, then replace the line when iterating over the source file with the new _local_config values:
count = 0
a = 0
b = 1
c = 1
_local_config = dict(
filter(
lambda elem: (elem[0][:2] != "__") and (str(elem[1])[:1] != "<"),
globals().items(),
),
)
# do some stuff
count += 1
c = a + b
a = b
b = c
# update with new values
_local_config = dict(
filter(
lambda elem: elem[0] in _local_config.keys(),
globals().items(),
)
)
# read self
with open(__file__, "r") as f:
new_file = ""
for line in f.read().split("\n"):
for k, v in _local_config.items():
search = f"{k} = "
if search == line[: len(k) + 3]:
line = search + str(v)
_local_config.pop(k)
break
new_file += line + "\n"
# write self
with open(__file__, "w") as f:
f.write(new_file[:-1])
Essentially what I am attempting to do is read 'n' number of lines from a file and then write them to a separate file. This program essentially should take a file that has 100 lines and separate that file into 50 separate files.
def main():
from itertools import islice
userfile = raw_input("Please enter the file you wish to open\n(must be in this directory): ")
file1 = open(userfile, "r+")
#print "Name: ", file1.name
#print "Closed or not", file1.closed
#print "Opening mode: ", file1.mode
#print "Softspace flag: ", file1.softspace
jcardtop = file1.read(221);
#print jcardtop
n = 2
count = 0
while True:
next_n_lines = list(islice(file1,n))
print next_n_lines
count = count + 1
fileout = open(str(count)+ ".txt", "w+")
fileout.write(str(jcardtop))
fileout.write(str(next_n_lines))
fileout.close()
break
if not next_n_lines:
break
I do have the file printing as well to show what is in the variable next_n_lines.
*['\n', "randomtext' more junk here\n"]
I would like it instead to look like
randomtext' more junk here
Is this a limitatoin of the islice function? Or am I missing a portion of the syntax?
Thanks for your time!
Where you call str() or print, you want to ''.join(next_n_lines) instead:
print ''.join(next_n_lines)
and
fileout.write(''.join(next_n_lines))
You can store the flattened string in a variable if you don't want to call join twice.
Did you mean something like this?
f = open(userfile,"r")
start = 4
n_lines = 100
for line in f.readlines()[start:(start + n_lines)]:
print line
#do stuff with line
or maybe this rough, yet effective code:
f = open(userfile,"r")
start = 4
end = start + 100
count = start
while count != end:
for line in f.readlines()[count:(count + 2)]:
fileout = open(str(count)+ ".txt", "w+")
fileout.write(str(line))
fileout.close()
count = count + 2
I'm in trouble here. I need to read a file. Txt file that contains a sequence of records, check the records that I want to copy them to a new file.
The file content is like this (this is just an example, the original file has more than 30 000 lines):
AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460
99999|35|436 #end register
00000|46|316 #begin register
03000|SP|467
99999|33|130 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file
The records that begin with 03000 and have the characters 'TO' must be written to a new file. Based on the example, the file should look like this:
AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460
99999|35|436 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file
Code:
file = open("file.txt",'r')
newFile = open("newFile.txt","w")
content = file.read()
file.close()
# here I need to check if the record exists 03000 characters 'TO', if it exists, copy the recordset 00000-99999 for the new file.
I did multiple searches and found nothing to help me.
Thank you!
with open("file.txt",'r') as inFile, open("newFile.txt","w") as outFile:
outFile.writelines(line for line in inFile
if line.startswith("03000") and "TO" in line)
If you need the previous and the next line, then you have to iterate inFile in triads. First define:
def gen_triad(lines, prev=None):
after = current = next(lines)
for after in lines:
yield prev, current, after
prev, current = current, after
And then do like before:
outFile.writelines(''.join(triad) for triad in gen_triad(inFile)
if triad[1].startswith("03000") and "TO" in triad[1])
import re
pat = ('^00000\|\d+\|\d+.*\n'
'^03000\|TO\|\d+.*\n'
'^99999\|\d+\|\d+.*\n'
'|'
'^AAAAA\|\d+\|\d+.*\n'
'|'
'^ZZZZZ\|\d+\|\d+.*')
rag = re.compile(pat,re.MULTILINE)
with open('fifi.txt','r') as f,\
open('newfifi.txt','w') as g:
g.write(''.join(rag.findall(f.read())))
For files with additional lines between lines beginning with 00000, 03000 and 99999, I didn't find simpler code than this one:
import re
pat = ('(^00000\|\d+\|\d+.*\n'
'(?:.*\n)+?'
'^99999\|\d+\|\d+.*\n)'
'|'
'(^AAAAA\|\d+\|\d+.*\n'
'|'
'^ZZZZZ\|\d+\|\d+.*)')
rag = re.compile(pat,re.MULTILINE)
pit = ('^00000\|.+?^03000\|TO\|\d+.+?^99999\|')
rig = re.compile(pit,re.DOTALL|re.MULTILINE)
def yi(text):
for g1,g2 in rag.findall(text):
if g2:
yield g2
elif rig.match(g1):
yield g1
with open('fifi.txt','r') as f,\
open('newfifi.txt','w') as g:
g.write(''.join(yi(f.read())))
file = open("file.txt",'r')
newFile = open("newFile.txt","w")
content = file.readlines()
file.close()
newFile.writelines(filter(lambda x:x.startswith("03000") and "TO" in x,content))
This seems to work. The other answers seem to only be writing out records that contain '03000|TO|' but you have to write out the record before and after that as well.
import sys
# ---------------------------------------------------------------
# ---------------------------------------------------------------
# import file
file_name = sys.argv[1]
file_path = 'C:\\DATA_SAVE\\pick_parts\\' + file_name
file = open(file_path,"r")
# ---------------------------------------------------------------
# create output files
output_file_path = 'C:\\DATA_SAVE\\pick_parts\\' + file_name + '.out'
output_file = open(output_file_path,"w")
# create output files
# ---------------------------------------------------------------
# process file
temp = ''
temp_out = ''
good_write = False
bad_write = False
for line in file:
if line[:5] == 'AAAAA':
temp_out += line
elif line[:5] == 'ZZZZZ':
temp_out += line
elif good_write:
temp += line
temp_out += temp
temp = ''
good_write = False
elif bad_write:
bad_write = False
temp = ''
elif line[:5] == '03000':
if line[6:8] != 'TO':
temp = ''
bad_write = True
else:
good_write = True
temp += line
temp_out += temp
temp = ''
else:
temp += line
output_file.write(temp_out)
output_file.close()
file.close()
Output:
AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460
99999|35|436 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file
Does it have to be python? These shell commands would do the same thing in a pinch.
head -1 inputfile.txt > outputfile.txt
grep -C 1 "03000|TO" inputfile.txt >> outputfile.txt
tail -1 inputfile.txt >> outputfile.txt
# Whenever I have to parse text files I prefer to use regular expressions
# You can also customize the matching criteria if you want to
import re
what_is_being_searched = re.compile("^03000.*TO")
# don't use "file" as a variable name since it is (was?) a builtin
# function
with open("file.txt", "r") as source_file, open("newFile.txt", "w") as destination_file:
for this_line in source_file:
if what_is_being_searched.match(this_line):
destination_file.write(this_line)
and for those who prefer a more compact representation:
import re
with open("file.txt", "r") as source_file, open("newFile.txt", "w") as destination_file:
destination_file.writelines(this_line for this_line in source_file
if re.match("^03000.*TO", this_line))
code:
fileName = '1'
fil = open(fileName,'r')
import string
##step 1: parse the file.
parsedFile = []
for i in fil:
##tuple1 = (1,2,3)
firstPipe = i.find('|')
secondPipe = i.find('|',firstPipe+1)
tuple1 = (i[:firstPipe],\
i[firstPipe+1:secondPipe],\
i[secondPipe+1:i.find('\n')])
parsedFile.append(tuple1)
fil.close()
##search criterias:
searchFirst = '03000'
searchString = 'TO' ##can be changed if and when required
##step 2: used the parsed contents to write the new file
filout = open('newFile','w')
stringToWrite = parsedFile[0][0] + '|' + parsedFile[0][1] + '|' + parsedFile[0][2] + '\n'
filout.write(stringToWrite) ##to write the first entry
for i in range(1,len(parsedFile)):
if parsedFile[i][1] == searchString and parsedFile[i][0] == searchFirst:
for j in range(-1,2,1):
stringToWrite = parsedFile[i+j][0] + '|' + parsedFile[i+j][1] + '|' + parsedFile[i+j][2] + '\n'
filout.write(stringToWrite)
stringToWrite = parsedFile[-1][0] + '|' + parsedFile[-1][1] + '|' + parsedFile[-1][2] + '\n'
filout.write(stringToWrite) ##to write the first entry
filout.close()
I know that this solution may be a bit long. But it is quite easy to understand. And it seems an intuitive way to do it. And I have already checked this with the Data that you have provided and it works perfectly.
Please tell me if you need some more explanation on the code. I will definitely add the same.
I tip (Beasley and Joran elyase) very interesting, but it only allows to get the contents of the line 03000. I would like to get the contents of the lines 00000 to line 99999.
I even managed to do here, but I am not satisfied, I wanted to make a more cleaner.
See how I did:
file = open(url,'r')
newFile = open("newFile.txt",'w')
lines = file.readlines()
file.close()
i = 0
lineTemp = []
for line in lines:
lineTemp.append(line)
if line[0:5] == '03000':
state = line[21:23]
if line[0:5] == '99999':
if state == 'TO':
newFile.writelines(lineTemp)
else:
linhaTemp = []
i = i+1
newFile.close()
Suggestions...
Thanks to all!
I'm having some difficulties while parsing a binary STL file with Python (2.7.1 32-bit and Windows 7 64). The file is a about 450k in size, but my parser suddenly stops working after parsing 244 triangles out of ~8600 with en exception of struct.unpack:
Exception unpack requires a string argument of length 12
The cursor position in the file is line 33, row 929. But the line contains about 3400 characters. So it doesn't seem to be newline problem.
This is the code:
import struct
normals = []
points = []
triangles = []
bytecount = []
fb = [] # debug list
def unpack (f, sig, l):
s = f.read (l)
fb.append(s)
return struct.unpack(sig, s)
def read_triangle(f):
n = unpack(f,"<3f", 12)
p1 = unpack(f,"<3f", 12)
p2 = unpack(f,"<3f", 12)
p3 = unpack(f,"<3f", 12)
b = unpack(f,"<h", 2)
normals.append(n)
l = len(points)
points.append(p1)
points.append(p2)
points.append(p3)
triangles.append((l, l+1, l+2))
bytecount.append(b[0])
def read_length(f):
length = struct.unpack("#i", f.read(4))
return length[0]
def read_header(f):
f.seek(f.tell()+80)
def write_as_ascii(outfilename):
f = open(outfilename, "w")
f.write ("solid "+outfilename+"\n")
for n in range(len(triangles)):
f.write ("facet normal {} {} {}\n".format(normals[n][0],normals[n][1],normals[n][2]))
f.write ("outer loop\n")
f.write ("vertex {} {} {}\n".format(points[triangles[n][0]][0],points[triangles[n][0]][1],points[triangles[n][0]][2]))
f.write ("vertex {} {} {}\n".format(points[triangles[n][1]][0],points[triangles[n][1]][1],points[triangles[n][1]][2]))
f.write ("vertex {} {} {}\n".format(points[triangles[n][2]][0],points[triangles[n][2]][1],points[triangles[n][2]][2]))
f.write ("endloop\n")
f.write ("endfacet\n")
f.write ("endsolid "+outfilename+"\n")
f.close()
def main():
infilename = r"cupHemis46_28.stl"
outfilename = r"cupHemis46_28_ascii_test.stl"
try:
f = open ( infilename, "r")
read_header(f)
l = read_length(f)
try:
while True:
read_triangle(f)
except Exception, e:
print "Exception",e[0]
print len(normals), len(points), len(triangles), l
write_as_ascii(outfilename)
except Exception, e:
print e
if __name__ == '__main__':
main()
The unpack function (not from struct) collects all the strings which will be written to a file. When I compare both file they seem equal, up to the file position where unpack stops working. I opened the binary file with Notepad++, the next character is a "SUB".
Are there any restrictions of unpack that I'm not aware of regarding file size or limitation in characters or something? Is there something wrong with my code? Thanks in advance.
Your unpack function calls f.read twice. I suspect you've walked off the end of the file.
You'll also have trouble with reading the file in text mode on Windows. Any incidental occurrences of \r\n will be read in as \n. Make the following change to avoid this problem.
f = open(infilename, "rb")