We Keep Coding

Adding a new string to the end of a specific line in a text file

I'm new to python hence I am unable to implement the solutions I've found online in order to fix my problem.
I am trying to add a specific string to the end of a specific line to a textfile. As I understand text commands, I must overwrite the file if I don't want to append to the end of it. So, my solution is as follows:
ans = 'test'
numdef = ['H',2]
f = open(textfile, 'r')
lines = f.readlines()
f.close()
f = open(textfile, 'w')
f.write('')
f.close()
f = open(textfile, 'a')
for line in lines:
if int(line[0]) == numdef[1]:
if str(line[2]) == numdef[0]:
k = ans+ line
f.write(k)
else:
f.write(line)
Basically, I am trying to add variable ans to the end of a specific line, the line which appears in my list numdef. So, for example, for
2 H: 4,0 : Where to search for information : google
I want
2 H: 4,0 : Where to search for information : google test
I have also tried using line.insert() but to no avail.
I understand using the 'a' function of the open command is not so relevant and helpful here, but I am out of ideas. Would love tips with this code, or if maybe I should scrap it and rethink the whole thing.
Thank you for your time and advice!

When you use the method
lines = f.readlines()
Python automatically adds "\n" to the end of each line.
Try instead of :
k = line+ans
The following:
k = line.rstrip('\n') + ans
Good luck!

Try this. You don't have an else case if it meets the first requirement but not the other.
ans = 'test'
numdef = ['H',2]
f = open(textfile, 'r')
lines = f.readlines()
f.close()
f = open(textfile, 'w')
f.write('')
f.close()
f = open(textfile, 'a')
for line in lines:
if int(line[0]) == numdef[1] and str(line[2]) == numdef[0]:
k = line.replace('\n','')+ans
f.write(k)
else:
f.write(line)
f.close()
Better way:
#initialize variables
ans = 'test'
numdef = ['H',2]
#open file in read mode, add lines into lines
with open(textfile, 'r') as f:
lines=f.readlines()
#open file in write mode, override everything
with open(textfile, 'w') as f:
#in the list comprehension, loop through each line in lines, if both of the conditions are true, then take the line, remove all newlines, and add ans. Otherwise, remove all the newlines and don't add anything. Then combine the list into a string with newlines as separators ('\n'.join), and write this string to the file.
f.write('\n'.join([line.replace('\n','')+ans if int(line[0]) == numdef[1] and str(line[2]) == numdef[0] else line.replace('\n','') for line in lines]))

Changing a text file and making a bigger text file in python

I have a tab separated text file like these example:
infile:
chr1 + 1071396 1271396 LOC
chr12 + 1101483 1121483 MIR200B
I want to divide the difference between columns 3 and 4 in infile into 100 and make 100 rows per row in infile and make a new file named newfile
and make the final tab separated file with 6 columns. The first 5 columns would be like infile, the 6th column would be (5th column)_part number (number is 1 to 100).
This is the expected output file:
expected output:
chr1 + 1071396 1073396 LOC LOC_part1
chr1 + 1073396 1075396 LOC LOC_part2
.
.
.
chr1 + 1269396 1271396 LOC LOC_part100
chr12 + 1101483 1101683 MIR200B MIR200B_part1
chr12 + 1101683 1101883 MIR200B MIR200B_part2
.
.
.
chr12 + 1121283 1121483 MIR200B MIR200B_part100
I wrote the following code to get the expected output but it does not return what I expect.
file = open('infile.txt', 'rb')
cont = []
for line in file:
cont.append(line)
newfile = []
for i in cont:
percent = (i[3]-i[2])/100
for j in percent:
newfile.append(i[0], i[1], i[2], i[2]+percent, i[4], i[4]_'part'percent[j])
with open('output.txt', 'w') as f:
for i in newfile:
for j in i:
f.write(i + '\n')
Do you know how to fix the problem?

Try this:
file = open('infile.txt', 'rb')
cont = []
for line in file:
cont.append(list(filter(lambda x: not x.isspace(), line.split(' ')))
newfile = []
for i in cont:
diff= (int(i[3])-int(i[2]))/100
left = i[2]
right = i[2] + diff
for j in range(100):
newfile.append(i[0], i[1], left, right, i[4], i[4]_'part' + j)
left = right
right = right + diff
with open('output.txt', 'w') as f:
for i in newfile:
for j in i:
f.write(i + '\n')
In your code for i in cont youre loop over the string and i is a char and not string.
To fix that i split the line and remove spaces.

Here are some suggestions:
when you open the file, open it as a text file, not a binary file.
open('infile.txt','r')
now, when you read it line by line, you should strip the newline character at the end by using strip(). Then, you need to split your input text line by tabs into a list of strings, vs a just a long string containing your line, by using split('\t'):
line.strip().split('\t')
now you have:
file = open('infile.txt', 'r')
cont = []
for line in file:
cont.append(line.strip().split('\t))
now cont is a list of lists, where each list contains your tab separated data. i.e.
cont[1][0] = 'chr12'.
You will probably able to take it from here.

Others have answered your question with respect to your own code, I thought I would leave my attempt at solving your problem here.
import os
directory = "C:/Users/DELL/Desktop/"
filename = "infile.txt"
path = os.path.join(directory, filename)
with open(path, "r") as f_in, open(directory+"outfile.txt", "w") as f_out: #open input and output files
for line in f_in:
contents = line.rstrip().split("\t") #split line into words stored as a string 'contents'
diff = (int(contents[3]) - int(contents[2]))/100
for i in range(100):
temp = (f"{contents[0]}\t+\t{int(int(contents[2])+ diff*i)}\t{contents[3]}\t{contents[4]}\t{contents[4]}_part{i+1}")
f_out.write(temp+"\n")
This code doesn't follow python style convention well (excessively long lines, for example) but it works. The line temp = ... uses fstrings to format the output string conveniently, which you could read more about here.

How do append characters on either side of certain words on every line in a file in python?

Say I have a file my_file, and I want to search for a certain word x on every line of the file, and if the word exists, attach my variable y to the left and right side of the word. Then I want replace the old line with the new, modified line in my_new_file. How do I do this? So far I have:
output = open(omy_new_file, "w")
for line in open(my_file):
if (" " + x + "") in line:

You can try this:
y = "someword"
x = "target_string"
lines = [i.strip('\n') for i in open('filename.txt')]
final_lines = ["{}{}{}".format(y, i, y) if x in i else i for i in lines]
f = open(omy_new_file, "w")
for i in final_lines:
f.write("{}\n".format(i))
f.close()

with open('inputfile.txt', 'r') as infile:
with open('outfile.txt', 'w') as outfile:
for line in infile.readlines():
outfile.write(line.replace('string', y + 'string' + y)

Try This:
with open("my_file", "r") as my_file:
raw_data = my_file.read()
# READ YOUR FILE
new_data = raw_data.split("\n")
for line in new_data:
if "sd" in line:
my_new_line = "y" + line + "y"
raw_data = raw_data.replace(line, my_new_line)
print(raw_data)

It's tough to replace a line in a file while reading it, for the same reason that it's tough to safely modify a list as you iterate over it.
It's much better to read through the file, collect a list of lines, then overwrite the original. If the file is particularly large (such that it would be infeasible to hold it all in memory at once), you can write to disk twice.
import tempfile
y = "***"
your_word = "Whatever you're filtering by"
with tempfile.TemporaryFile(mode="w+") as tmpf:
with open(my_file, 'r') as f:
for line in f:
if your_word in line:
line = f"{y}{line.strip()}{y}\n"
tmpf.write(line) # write to the temp file
tmpf.seek(0) # move back to the beginning of the tempfile
with open(my_file, 'w') as f:
for line in tmpf: # reading from tempfile now
my_file.write(line)

How can I compare files quicker in Python?

Is there any way to make this script faster? I'm using one file to compare another file to print lines, if second column are equal.
import csv
output =[]
a = open('/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Phase1_missing.vcf', 'r')
list1 = a.readlines()
reader1 = a.read()
b = open('/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Neandertais.vcf', 'r')
list2 = b.readlines()
reader2 = b.read()
f3 = open('/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Neandertais_and_YRI.vcf', 'w')
for line1 in list1:
separar = line1.split("\t")
gene = separar[2]
for line2 in list2:
separar2 = line2.split("\t")
gene2 = separar2[2]
if gene == gene2:
print line1
f3.write(line1)
Input example (for both files):
1 14107321 rs187821037 C T 100 PASS AA=C;SNPSOURCE=LOWCOV,EXOME;AN=2184;AVGPOST=0.9996;VT=SNP;THETA=0.0006;RSQ=0.7640;LDAF=0.0006;AC=1;ERATE=0.0003;AF=0.0005;AFR_AF=0.0020;STATUS=sample_dropout
1 14107321 rs187821037 C T 100 PASS AA=C;SNPSOURCE=LOWCOV,EXOME;AN=2184;AVGPOST=0.9996;VT=SNP;THETA=0.0006;RSQ=0.7640;LDAF=0.0006;AC=1;ERATE=0.0003;AF=0.0005;AFR_AF=0.0020;STATUS=sample_dropout
1 14107321 rs187821037 C T 100 PASS AA=C;SNPSOURCE=LOWCOV,EXOME;AN=2184;AVGPOST=0.9996;VT=SNP;THETA=0.0006;RSQ=0.7640;LDAF=0.0006;AC=1;ERATE=0.0003;AF=0.0005;AFR_AF=0.0020;STATUS=sample_dropout
The command line below works equally for same purpose in bash:
awk 'FNR==NR {a[$3]; next} $3 in a' Neandertais.vcf Phase1_missing.vcf > teste.vcf
How can I improve this Python script?

If you store your lines in dictionaries that are keyed by the column that you are interested in, you can easily use Python's built-in set functions (which run at C speed) to find the matching lines. I tested a slightly modified version of this (filenames changed, and changed split('\t') to split() because of stackoverflow formatting) and it seems to work fine:
import collections
# Use 'rb' to open files
infn1 = '/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Phase1_missing.vcf'
infn2 = '/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Neandertais.vcf'
outfn = '/home/lucas/Doutorado/Projeto Eduardo/Exoma Neandertal/Listas_eduardo/Neandertais_and_YRI.vcf'
def readfile(fname):
'''
Read in a file and return a dictionary of lines, keyed by the item in the second column
'''
results = collections.defaultdict(list)
# Read in binary mode -- it's quicker
with open(fname, 'rb') as f:
for line in f:
parts = line.split("\t")
if not parts:
continue
gene = parts[2]
results[gene].append(line)
return results
dict1 = readfile(infn1)
dict2 = readfile(infn2)
with open(outfn, 'wb') as outf:
# Find keys that appear in both files
for key in set(dict1) & set(dict2):
# For these keys, print all the matching
# lines in the first file
for line in dict1[key]:
print(line.rstrip())
outf.write(line)

Counting to 100,000 and writing that to a file

I haven't used Python for a while but I decided to create a program today to help me with some work I am trying to do. I am trying to create a program that writes the numbers 1-100,000 with the symbol | after each but can't seem to strip the file after I create it so it shows like this: 1|2|3|4.
My Code:
a = 0
b = "|"
while a < 100000:
a += 1 # Same as a = a + 1
new = (a,b)
f = open("export.txt","a") #opens file with name of "export.txt"
f.write(str(new))
f.close()
infile = "export.txt"
outfile = "newfile.txt"
delete_list = ["(","," "'"]
fin = open(infile)
fout = open(outfile, "w+")
for line in fin:
for word in delete_list:
line = line.replace(word, "")
fout.write(line)
fin.close()
fout.close()
export.txt:
newfile.txt:

It looks like you're doing a lot of work unnecessarily.
If all you want is a file that has the numbers 0-99999 with | after each, you could do:
delim = "|"
with open('export.txt', 'w') as f:
for a in xrange(100):
f.write("%d%s" % (a, delim))
I'm not sure what the purpose of the second file is, but, in general, to open one file to read from and a second to write to, you could do:
with open('export.txt', 'r') as fi:
with open('newfile.txt', 'w') as fo:
for line in fi:
for word in line.split('|'):
print(word)
fo.write(word)
Note that there are no newlines in the original file, so for line in fi is actually reading the entire contents of "export.txt" -- this could cause issues.

Try this for writing your file:
numbers = []
for x in range(1,100001):
numbers.append(str(x))
f = open('export.txt', 'w')
f.write('|'.join(numbers))
f.close()