Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I am looping through a file and I need to insert a text block above a line when it matches a particular string. Please help!
convBlockMoved = False
for line in outfile:
if(line.startswith('mappingSchemeAxis') and not convBlockMoved):
for convLine in conversionBlocks:
print convLine, #Print this above line
convBlockMoved = True
Note: conversionBlocks is a String array
Not a Python answer, but sed can do this in one line.
The file:
$ cat > so.txt
foo
bar
baz
qux
Insert before line baz:
$ sed -i '/baz/i11\n22\n33' so.txt
The result:
$ cat so.txt
foo
bar
11
22
33
baz
qux
So if your file is not huge, you can read all the lines at once and then work with lists. An example using the insert method of lists would be:
def main():
lines = []
with open('input.txt') as f:
lines = f.readlines()
ins_at = find_occurences_of('mappingSchemeAxis', lines)
for i in ins_at:
lines.insert(i,'HELLO WORLD\n')
with open('input.txt', 'w') as f:
f.writelines(lines)
def find_occurences_of(needle, haystack):
ret = []
for i, line in enumerate(haystack):
if line.startswith(needle):
ret.append(i)
return ret
if __name__ == '__main__':
main()
Basically, you are reading a list of strings and you want to put a new list element above the current one under some conditions.
What I would suggest you (if the file is not too big) is to append the lines from your input to an output list, appending the text you want before each lines that matche your conditions. Something like the following
for line in infile.readlines ():
if line.startswith ('mappingSchemeAxis'):
outcontent.append ('xxxxx')
outcontent.append (line)
for line in outcontent:
print (line) # here you want to write the content to the output file
I posted that a bit late :D
Try this:
def replace_import_in_f(f_in, pattern, plus):
with open(f_in) as f:
in_str = f.read()
in_str = re.sub(pattern, pattern + plus + "\n", in_str)
with open(f_in, "w") as f:
f.write(in_str)
Pattern must be the entire line that you would add the new line above.
Note: This is perfect for medium file due to the f.write() of the whole file content. (Test with python 3.4)
[UPDATE]
More complicated but to handle big files, use of coroutine to write to temp file during line processing. If not error replace the temp file.
import tempfile, os
def write_file(pattern="", plus=""):
with tempfile.NamedTemporaryFile(delete=False) as fin:
yield fin
while True:
line = (yield)
if pattern:
if line.startswith(pattern):
fin.write(bytes(plus, 'UTF-8'))
fin.write(bytes(line, 'UTF-8'))
def copy_file(path_in, path_out):
with open(path_in) as fin, open(path_out, "w") as fout:
for line in fin:
fout.write(line)
def read_and_file(fname, pattern="", plus=""):
try:
with open(fname) as fh:
# generator creation
gen_write = write_file(pattern, plus)
# get the tempfile
fout = next(gen_write)
for line in fh:
# send line
gen_write.send(line)
except (IOError, OSError) as e:
print(e)
else:
fout.close()
if os.name == "nt":
copy_file(fout.name, fname)
else:
os.rename(fout.name, fname)
Related
def getlink():
with open('findlink.txt') as infile, open('extractlink.txt', 'w') as outfile:
copy = False
for line in infile:
if "](" in line.strip():
copy = True
if copy:
outfile.write(line)
if ")" in line.strip():
copy = False
print("extractlink written.")
def part3():
with open ('findlink.txt', 'w') as findlink:
findlink.write("[Testing](Test)")
print("findlink written and closed.")
getlink()
def run_bot():
getlink() #Already have findlink.txt written
When part3() is activated, the text is written to findlink.txt as expected, but when getlink() is activated, the extractlink.txt is never written to.
I've gathered my current code from a post back in 2013/2016, does anyone have any ideas why this may not be working?
The current goal is to have "Test" from findlink copied to extractlink.txt, not the entire line.
Someone edited your post to fix this, but you should understand that it is unnecessary to explicitly close a file when using a "with" statement AKA a context manager, because they handle this automatically for you.
A simple parser :
def getlink():
with open('findlink.txt') as infile, open('extractlink.txt', 'w') as outfile:
for line in infile:
begin = line.find("](")+2
end = line.find(")")
if 0 <= begin < end:
outfile.write(line[begin:end]+'\n')
According to this [tutorial][1] https://www.tutorialspoint.com/python/string_strip.htm
line.strip("text") removes text from the string.
So when you call line.strip() == "](" you are comparing line.strip("") with "](" which is always false unless the input line was just "](". So this condition is not really doing anything.
To get the Test from the file, I used line.find() which gives the index of the character in the string.
print line.find("](")
print line.find(")")
print line[line.find("](")+2: line.find(")")]
output:
8
14
Test
So then you could just do this for getlink().
def getlink():
with open('findlink.txt') as infile, open('extractlink.txt', 'w') as outfile:
for line in infile:
outfile.write(line[line.find("](")+2: line.find(")")])
print("extractlink written.")
infile.close()
outfile.close()
extractlink.txt:
Test
This is just one simple solution. You could implement it differently, but you may want to use line.find() instead of line.strip().
I'm just starting to learn python and have a textfile that looks like this:
Hello
World
Hello
World
And I want to add the numbers '55' to the beggining and end of every string that starts with 'hello'
The numbers '66' to the beggining and every of every string that starts with 'World'
etc
So my final file should look like this:
55Hello55
66World66
55Hello55
66World66
I'm reading the file in all at once, storing it in a string, and then trying to append accordingly
fp = open("test.txt","r")
strHolder = fp.read()
print(strHolder)
if 'Hello' in strHolder:
strHolder = '55' + strHolder + '55'
if 'World' in strHolder:
strHolder = '66' + strHolder + '66'
print(strHolder)
fp.close()
However, my string values '55' and '66' are always being added to the front of the file and end of the file, not the front of a certain string and to the end of the string, where I get this output of the string:
6655Hello
World
Hello
World
5566
Any help would be much appreciated.
You are reading the whole file at once with .read().
You can read it line by line in a for loop.
new_file = []
fp = open("test.txt", "r")
for line in fp:
line = line.rstrip("\n") # The string ends in a newline
# str.rstrip("\n") removes newlines at the end
if "Hello" in line:
line = "55" + line + "55"
if "World" in line:
line = "66" + line + "66"
new_file.append(line)
fp.close()
new_file = "\n".join(new_file)
print(new_file)
You could do it all at once, by reading the whole file and splitting by "\n" (newline)
new_file = []
fp = open("text.txt")
fp_read = fp.read()
fp.close()
for line in fp_read.split("\n"):
if "Hello" # ...
but this would load the whole file into memory at once, while the for loop only loads line by line (So this may not work for larger files).
The behaviour of this is that if the line has "Hello" in it, it will get "55" before and after it (even if the line is " sieohfoiHellosdf ") and the same for "World", and if it has both "Hello" and "World" (e.g. "Hello, World!" or "asdifhoasdfhHellosdjfhsodWorldosadh") it will get "6655" before and after it.
Just as a side note: You should use with to open a file as it makes sure that the file is closed later.
new_file = []
with open("test.txt") as fp: # "r" mode is default
for line in fp:
line = line.rstrip("\n")
if "Hello" in line:
line = "55" + line + "55"
if "World" in line:
line = "66" + line + "66"
new_file.append(line)
new_file = "\n".join(new_file)
print(new_file)
You need to iterate over each line of the file in order to get the desired result. In your code you are using .read(), instead use .readlines() to get list of all lines.
Below is the sample code:
lines = []
with open("test.txt", "r") as f:
for line in f.readlines(): # < Iterate over each line
if line.startswith("Hello"): # <-- check if line starts with "Hello"
line = "55{}55".format(line)
elif line.startswith("World"):
line = "66{}66".format(line)
lines.append(line)
print "\n".join(lines)
Why to use with? Check Python doc:
The ‘with‘ statement clarifies code that previously would use try...finally blocks to ensure that clean-up code is executed. In this section, I’ll discuss the statement as it will commonly be used. In the next section, I’ll examine the implementation details and show how to write objects for use with this statement.
The ‘with‘ statement is a control-flow structure whose basic structure is:
with expression [as variable]: with-block
The expression is evaluated, and it should result in an object that supports the context management protocol (that is, has enter() and exit() methods).
once you have read the file:
read_file = read_file.replace('hello','55hello55')
It'll replace all hellos with 55hello55
and use with open(text.txt, 'r' ) as file_hndler:
To read a text file, I recommend the following way which is compatible with Python 2 & 3:
import io
with io.open("test", mode="r", encoding="utf8") as fd:
...
Here, I make the assumption that your file use uft8 encoding.
Using a with statement make sure the file is closed at the end of reading even if a error occurs (an exception). To learn more about context manager, take a look at the Context Library.
There are several ways to read a text file:
read the whole file with: fd.read(), or
read line by line with a loop: for line in fd.
If you read the whole file, you'll need to split the lines (see str.splitlines. Here are the two solutions:
with io.open("test", mode="r", encoding="utf8") as fd:
content = fd.read()
for line in content.splilines():
if "Hello" in line:
print("55" + line + "55")
if "World" in line:
print("66" + line + "66")
Or
with io.open("test", mode="r", encoding="utf8") as fd:
for line in content.splilines():
line = line[:-1]
if "Hello" in line:
print("55" + line + "55")
if "World" in line:
print("66" + line + "66")
If you need to write the result in another file you can open the output file in write mode and use print(thing, file=out) as follow:
with io.open("test", mode="r", encoding="utf8") as fd:
with io.open("test", mode="w", encoding="utf8") as out:
for line in content.splilines():
line = line[:-1]
if "Hello" in line:
print("55" + line + "55", file=out)
if "World" in line:
print("66" + line + "66", file=out)
If you use Python 2, you'll need the following directive to use the print function:
from __future__ import print_function
I want to delete some specific lines in a file.
The part I want to delete is enclosed between two lines (that will be deleted too), named STARTING_LINE and CLOSING_LINE. If there is no closing line before the end of the file, then the operation should stop.
Example:
...blabla...
[Start] <-- # STARTING_LINE
This is the body that I want to delete
[End] <-- # CLOSING_LINE
...blabla...
I came out with three different ways to achieve the same thing (plus one provided by tdelaney's answer below), but I am wondering which one is the best. Please note that I am not looking for a subjective opinion: I would like to know if there are some real reasons why I should choose one method over another.
1. A lot of if conditions (just one for loop):
def delete_lines(filename):
with open(filename, 'r+') as my_file:
text = ''
found_start = False
found_end = False
for line in my_file:
if not found_start and line.strip() == STARTING_LINE.strip():
found_start = True
elif found_start and not found_end:
if line.strip() == CLOSING_LINE.strip():
found_end = True
continue
else:
print(line)
text += line
# Go to the top and write the new text
my_file.seek(0)
my_file.truncate()
my_file.write(text)
2. Nested for loops on the open file:
def delete_lines(filename):
with open(filename, 'r+') as my_file:
text = ''
for line in my_file:
if line.strip() == STARTING_LINE.strip():
# Skip lines until we reach the end of the function
# Note: the next `for` loop iterates on the following lines, not
# on the entire my_file (i.e. it is not starting from the first
# line). This will allow us to avoid manually handling the
# StopIteration exception.
found_end = False
for function_line in my_file:
if function_line.strip() == CLOSING_LINE.strip():
print("stop")
found_end = True
break
if not found_end:
print("There is no closing line. Stopping")
return False
else:
text += line
# Go to the top and write the new text
my_file.seek(0)
my_file.truncate()
my_file.write(text)
3. while True and next() (with StopIteration exception)
def delete_lines(filename):
with open(filename, 'r+') as my_file:
text = ''
for line in my_file:
if line.strip() == STARTING_LINE.strip():
# Skip lines until we reach the end of the function
while True:
try:
line = next(my_file)
if line.strip() == CLOSING_LINE.strip():
print("stop")
break
except StopIteration as ex:
print("There is no closing line.")
else:
text += line
# Go to the top and write the new text
my_file.seek(0)
my_file.truncate()
my_file.write(text)
4. itertools (from tdelaney's answer):
def delete_lines_iter(filename):
with open(filename, 'r+') as wrfile:
with open(filename, 'r') as rdfile:
# write everything before startline
wrfile.writelines(itertools.takewhile(lambda l: l.strip() != STARTING_LINE.strip(), rdfile))
# drop everything before stopline.. and the stopline itself
try:
next(itertools.dropwhile(lambda l: l.strip() != CLOSING_LINE.strip(), rdfile))
except StopIteration:
pass
# include everything after
wrfile.writelines(rdfile)
wrfile.truncate()
It seems that these four implementations achieve the same result. So...
Question: which one should I use? Which one is the most Pythonic? Which one is the most efficient?
Is there a better solution instead?
Edit: I tried to evaluate the methods on a big file using timeit. In order to have the same file on each iteration, I removed the writing parts of each code; this means that the evaluation mostly regards the reading (and file opening) task.
t_if = timeit.Timer("delete_lines_if('test.txt')", "from __main__ import delete_lines_if")
t_for = timeit.Timer("delete_lines_for('test.txt')", "from __main__ import delete_lines_for")
t_while = timeit.Timer("delete_lines_while('test.txt')", "from __main__ import delete_lines_while")
t_iter = timeit.Timer("delete_lines_iter('test.txt')", "from __main__ import delete_lines_iter")
print(t_if.repeat(3, 4000))
print(t_for.repeat(3, 4000))
print(t_while.repeat(3, 4000))
print(t_iter.repeat(3, 4000))
Result:
# Using IF statements:
[13.85873354100022, 13.858520206999856, 13.851908310999988]
# Using nested FOR:
[13.22578497800032, 13.178281234999758, 13.155530822999935]
# Using while:
[13.254994718000034, 13.193942980999964, 13.20395484699975]
# Using itertools:
[10.547019549000197, 10.506679693000024, 10.512742852999963]
You can make it fancy with itertools. I'd be interested in how timing compares.
import itertools
def delete_lines(filename):
with open(filename, 'r+') as wrfile:
with open(filename, 'r') as rdfile:
# write everything before startline
wrfile.writelines(itertools.takewhile(lambda l: l.strip() != STARTING_LINE.strip(), rdfile))
# drop everything before stopline.. and the stopline itself
next(itertools.dropwhile(lambda l: l.strip() != CLOSING_LINE.strip(), rdfile))
# include everything after
wrfile.writelines(rdfile)
wrfile.truncate()
I am very new to programming and the python language.
I know how to open a file in python, but the question is how can I open the file as a parameter of a function?
example:
function(parameter)
Here is how I have written out the code:
def function(file):
with open('file.txt', 'r') as f:
contents = f.readlines()
lines = []
for line in f:
lines.append(line)
print(contents)
You can easily pass the file object.
with open('file.txt', 'r') as f: #open the file
contents = function(f) #put the lines to a variable.
and in your function, return the list of lines
def function(file):
lines = []
for line in f:
lines.append(line)
return lines
Another trick, python file objects actually have a method to read the lines of the file. Like this:
with open('file.txt', 'r') as f: #open the file
contents = f.readlines() #put the lines to a variable (list).
With the second method, readlines is like your function. You don't have to call it again.
Update
Here is how you should write your code:
First method:
def function(file):
lines = []
for line in f:
lines.append(line)
return lines
with open('file.txt', 'r') as f: #open the file
contents = function(f) #put the lines to a variable (list).
print(contents)
Second one:
with open('file.txt', 'r') as f: #open the file
contents = f.readlines() #put the lines to a variable (list).
print(contents)
Hope this helps!
Python allows to put multiple open() statements in a single with. You comma-separate them. Your code would then be:
def filter(txt, oldfile, newfile):
'''\
Read a list of names from a file line by line into an output file.
If a line begins with a particular name, insert a string of text
after the name before appending the line to the output file.
'''
with open(newfile, 'w') as outfile, open(oldfile, 'r', encoding='utf-8') as infile:
for line in infile:
if line.startswith(txt):
line = line[0:len(txt)] + ' - Truly a great person!\n'
outfile.write(line)
# input the name you want to check against
text = input('Please enter the name of a great person: ')
letsgo = filter(text,'Spanish', 'Spanish2')
And no, you don't gain anything by putting an explicit return at the end of your function. You can use return to exit early, but you had it at the end, and the function will exit without it. (Of course with functions that return a value, you use the return to specify the value to return.)
def fun(file):
contents = None
with open(file, 'r') as fp:
contents = fp.readlines()
## if you want to eliminate all blank lines uncomment the next line
#contents = [line for line in ''.join(contents).splitlines() if line]
return contents
print fun('test_file.txt')
or you can even modify this, such a way it takes file object as a function arguement as well
Here's a much simpler way of opening a file without defining your own function in Python 3.4:
var=open("A_blank_text_document_you_created","type_of_file")
var.write("what you want to write")
print (var.read()) #this outputs the file contents
var.close() #closing the file
Here are the types of files:
"r": just to read a file
"w": just to write a file
"r+": a special type which allows both reading and writing of the file
For more information see this cheatsheet.
def main():
file=open("chirag.txt","r")
for n in file:
print (n.strip("t"))
file.close()
if __name__== "__main__":
main()
the other method is
with open("chirag.txt","r") as f:
for n in f:
print(n)
I want to create a text file and add data to it, line by line. If a data line already exists in the file, it should be ignored. Otherwise, it should be appended to the file.
You are almost certainly better to read the file and write a new changed version. In most circumstances it will be quicker, easier, less error-prone and more extensible.
If your file isn't that big, you could just do something like this:
added = set()
def add_line(line):
if line not in added:
f = open('myfile.txt', 'a')
f.write(line + '\n')
added.add(line)
f.close()
But this isn't a great idea if you have to worry about concurrency, large amounts of data being stored in the file, or basically anything other than something quick and one-off.
I did it like this,
def retrieveFileData():
"""Retrieve Location/Upstream data from files"""
lines = set()
for line in open(LOCATION_FILE):
lines.add(line.strip())
return lines
def add_line(line):
"""Add new entry to file"""
f = open(LOCATION_FILE, 'a')
lines = retrieveFileData()
print lines
if line not in lines:
f.write(line + '\n')
lines.add(line)
f.close()
else:
print "entry already exists"
if __name__ == "__main__":
while True:
line = raw_input("Enter line manually: ")
add_line(line)
if line == 'quit':
break