I am trying to take a file with multiple entries and write it to a new file. The new file should contain stripped and split by comma data in separate list. Once I have done that I want to calculate the z score using row 3(observed) and row 4(expected). The z score formula I am using is Zi = (Observed i - Expected i )/sqrt(Expected i ).Then I want to add the Zscores to the list of data in the input file, which is where I am having trouble. I am using output_file = open(outpath,"w") but nothing is being written to the output file.
example input_file data:
Ashe,1853282.679,1673876.66,1,2 Alleghany,1963178.059,1695301.229,0 ,1 Surry,2092564.258,1666785.835,5 ,6 Currituck,3464227.016,1699924.786,1 ,1 Northampton,3056933.525,1688585.272,9 ,3 Hertford,3180151.244,1670897.027,7 ,3 Camden,3403469.566,1694894.58,0 ,1 Gates,3264377.534,1704496.938,0 ,1 Warren,2851154.003,1672865.891,4 ,2
my code:
import os
from math import sqrt
def calculateZscore(inpath,outpath):
"Z score calc"
input_file = open(inpath,"r")
lines = input_file.readlines()
output_file = open(outpath,"w")
county = []
x_coor = []
y_coor = []
observed = []
expected = []
score = 0
result = 0
for line in lines:
row = line.split(',')
county.append(row[0].strip())
x_coor.append(row[1].strip())
y_coor.append(row[2].strip())
observed.append(int(row[3].strip()))
expected.append(int (row[4].strip()))
o = observed
e = expected
length_o = len(o)
length_e = len(e)
score = 0
for i in range(length_o):
score += (o[i] - e[i])
result += (score/(sqrt(e[i])))
def main():
"collects data for code "
workingDirec = raw_input("What is the working directory?")
original_file = raw_input("The input filename is?")
full_original = os.path.join(workingDirec,original_file)
chi_square = raw_input("The name of the chi-squared stats table file is?")
full_chi_square = os.path.join(workingDirec,chi_square)
output_file = raw_input ("What is the output filename?")
full_output = os.path.join(workingDirec,output_file)
calculateZscore(full_original,full_output)
Any guidance would be appreciated
In order to write to a file object, you need to call fileobj.write() for a file object opened in write ("w"), append ("a"), read/write ("r+"), or append/write modes ("a+").
You need to call at some point at the end of your function:
output_file.write(str(result)) # want character representation
For example:
for i in range(length_o):
score += (o[i] - e[i])
result += (score/(sqrt(e[i])))
output_file.write(str(result))
Then, at the end, you should probably close the file
output_file.close()
(This is just in case you are calling a specific module many times, it's good practice to close all the file objects you open. Typically this is done by "with open(filename, mode) as e".
The Problem is access mode in your code input_file = open(inpath,"r") you are opening file in read only mode you need to give write permission too.
r Opens a file for reading only
Here are some of the permission mode, regarding writing into file. You can give whichever suits you
file = open('myfile.txt', 'r+')
r+ Opens a file for both reading and writing. The file pointer placed at the beginning of the file.
file = open('myfile.txt', 'w+')
w+ Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
file = open('myfile.txt', 'a+')
a+ Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
Then You need to write in that file too..
One simple workaround you can implement,
file = open('myfile.txt', 'r+')
file.write( "Python \n");
file.close()
Related
I want to save the wimp mass only from the text file below into another txt file for plotting purposes. I have a lot of other .txt files to read WIMP_Mass from. I try to use np.loadtxt but cannot since there is string there. Can you guys suggest the code to extract Wimp_Mass, and the output value to be appended to .txt file without deleting the old values.
Omegah^2: 2.1971043635736895E-003
x_f: 25.000000000000000
Wimp_Mass: 100.87269924860568 GeV
sigmav(xf): 5.5536288606920853E-008
sigmaN_SI_p: -1.0000000000000000 GeV^-2: -389000000.00000000 pb
sigmaN_SI_n: -1.0000000000000000 GeV^-2: -389000000.00000000 pb
sigmaN_SD_p: -1.0000000000000000 GeV^-2: -389000000.00000000 pb
sigmaN_SD_n: -1.0000000000000000 GeV^-2: -389000000.00000000 pb
Nevents: 0
smearing: 0
%:a1a1_emep 24.174602466963883
%:a1a1_ddx 0.70401899013937730
%:a1a1_uux 10.607701601533348
%:a1a1_ssx 0.70401807105204617
%:a1a1_ccx 10.606374255125269
%:a1a1_bbx 0.70432127586224602
%:a1a1_ttx 0.0000000000000000
%:a1a1_mummup 24.174596692050287
%:a1a1_tamtap 24.172981870222447
%:a1a1_vevex 1.3837949256836950
%:a1a1_vmvmx 1.3837949256836950
%:a1a1_vtvtx 1.3837949256836950
You can use regex for this, please find below code:
import re
def get_wimp_mass():
# read content of file
with open("txt.file", "r") as f:
# read all lines from file into a list
data_list = f.readlines()
# convert data from list to string
data = "\n".join(data_list)
# use regex to fetch the data
wimp_search = re.search(r'Wimp_Mass:\s+([0-9\.]+)', data, re.M | re.I)
if wimp_search:
return wimp_search.group(1)
else:
return "Wimp mass not found"
if __name__ == '__main__':
wimp_mass = get_wimp_mass()
print(wimp_mass)
You can use basic regex if you want to extract value the code would go something like this:
import re
ans=re.findall("Wimp_Mass:[\ ]+([\d\.]+)",txt)
ans
>>['100.87269924860568']
If you wanted a more general code to extract everything, it could be
re.findall("([a-zA-Z0-9\^:\%]+)[\ ]+([\d\.]+)",txt)
you might have to add in some edge cases, though
Here is a simple solution:
with open('new_file.txt', 'w') as f_out: # not needed if file already exists
pass
filepaths = ['file1.txt', 'file2.txt'] # take all files
with open('new_file.txt', 'a') as f_out:
for file in filepaths:
with open(file, 'r') as f:
for line in f.readlines():
if line.startswith('Wimp_Mass'):
_, mass, _ = line.split()
f_out.write(mass) # writing current mass
f_out.write('\n') # writing newline
It first creates a new text file (remove if file exists). Then you need to enter all file paths (or just names if in same directory). The new_file.txt is opened in append mode and then for each file, the mass is found and added to the new_file.
EDIT: The solution was simply changing how I was opening the file (thanks Primusa), not how I was replacing the information.
I am attempting to overwrite parts of a file, but my code does not work. When it runs, it completely erases everything in the file, leaving nothing behind.
Here is my code:
for fname in store:
with open(fname + ".txt", "w+") as f:
for line in f:
c = store[fname]
x = (c-1)%len(line.split(","))
y = m.floor((c-1)/len(line.split(",")))
for n in range(y):
f.readline()
ldata = line.strip("\n").split(",")
for i in range(len(ldata)):
ldata[i] = int(ldata[i])
if w == None:
ldata[x] += 1
elif w == True:
ldata[x] += 2
elif w == False:
ldata[x] -= 1
#s = line.find("\n") - 1
#f.truncate(s)
f.write(str(ldata) + "\n")
Key:
fname: a string variable corresponding to a file name without the file type.
store: a dictionary containing file name keys as string and integer values.
f: a file containing multiple lines of integer lists.
c: an integer variable used to determine the integer to be accessed.
x and y: variables with values set as the column and row (respectively) of the integer to be accessed in the file.
w: a variable stored as either a boolean or None, used to determine whether the accessed integer should increase or decrease, and by how much.
s: a currently unused integer variable used to truncate part of the file.
Example of use:
Say I have a file Foo.txt, which has the following information stored in it:
0,2,3,7
11,3,6,4
I want to increase the "6" value by 2, so I run the code with w set to True and add "Foo" : 7 to store, since "6" is the 7th number in the file (regardless of list length).
What should happen:
Foo.txt is modified and now contains:
0,2,3,7
11,3,8,4
What actually happens:
Foo.txt still exists, but now contains:
Which is to say it is empty.
What is wrong with my code? Am I handling the file incorrectly, the variable calculations, syntax, or something else entirely?
with open(fname + ".txt", "w+") as f:
Opening a file in mode "w+" truncates the file, which means it deletes everything inside of it. All of the operations you do after this statement are on an empty file.
What I would suggest would be opening the file in read mode:
with open(fname + ".txt", "r") as f:
Loading the file into memory, making your changes, and then opening the file in "w+" mode and putting the file back down.
Lets do this on the foo example:
with open("foo.txt", 'r') as f: #open in read mode
a = f.read().split(',') #breaking it apart so i can locate the 6th number easier
a[6] = str(int(a[6]) + 2) #make my modifications
a = ','.join(a) #convert it back to string
with open("foo.txt", 'w+') as f: #open in write and delete everything
f.write(a) #write my modified version of the file
Note this is a very basic example so it doesn't take into account the newlines.
I am very new with python. I have a .txt file and want to convert it to a .csv file with the format I was told but could not manage to accomplish. a hand can be useful for it. I am going to explain it with screenshots.
I have a txt file with the name of bip.txt. and the data inside of it is like this
I want to convert it to csv like this csv file
So far, what I could do is only writing all the data from text file with this code:
read_files = glob.glob("C:/Users/Emrehana1/Desktop/bip.txt")
with open("C:/Users/Emrehana1/Desktop/Test_Result_Report.csv", "w") as outfile:
for f in read_files:
with open(f, "r") as infile:
outfile.write(infile.read())
So is there a solution to convert it to a csv file in the format I desire? I hope I have explained it clearly.
There's no need to use the glob module if you only have one file and you already know its name. You can just open it. It would have been helpful to quote your data as text, since as an image someone wanting to help you can't just copy and paste your input data.
For each entry in the input file you will have to read multiple lines to collect together the information you need to create an entry in the output file.
One way is to loop over the lines of input until you find one that begins with "test:", then get the next line in the file using next() to create the entry:
The following code will produce the split you need - creating the csv file can be done with the standard library module, and is left as an exercise. I used a different file name, as you can see.
with open("/tmp/blip.txt") as f:
for line in f:
if line.startswith("test:"):
test_name = line.strip().split(None, 1)[1]
result = next(f)
if not result.startswith("outcome:"):
raise ValueError("Test name not followed by outcome for test "+test_name)
outcome = result.strip().split(None, 1)[1]
print test_name, outcome
You do not use the glob function to open a file, it searches for file names matching a pattern. you could open up the file bip.txt then read each line and put the value into an array then when all of the values have been found join them with a new line and a comma and write to a csv file, like this:
# set the csv column headers
values = [["test", "outcome"]]
current_row = []
with open("bip.txt", "r") as f:
for line in f:
# when a blank line is found, append the row
if line == "\n" and current_row != []:
values.append(current_row)
current_row = []
if ":" in line:
# get the value after the semicolon
value = line[line.index(":")+1:].strip()
current_row.append(value)
# append the final row to the list
values.append(current_row)
# join the columns with a comma and the rows with a new line
csv_result = ""
for row in values:
csv_result += ",".join(row) + "\n"
# output the csv data to a file
with open("Test_Result_Report.csv", "w") as f:
f.write(csv_result)
This program must change the last parameter on a line based on the starting code.
File now:
312|fotelja snesko|bela|15|2900|fotelja|False
621|digimon tabure|crna|25|850|tabure|False
Code is:>> 312
File after:
312|fotelja snesko|bela|15|2900|fotelja|True
621|digimon tabure|crna|25|850|tabure|False
My work right now.
parameter = input("\nCode is >> ")
with open("komad_namestaja.txt", "r") as fileNAME:
allDATA = fileNAME.readlines()
for linija in allDATA:
linija = linija.split("|")
if parameter == linija[0]:
linija[6] = "True"
With this I read every line in the file and found the line that I need. I just need to change False to True in that line. How can I change it?
As, your file is not too large(let's say 1 GB), no need of a temporary file. Below solution can help you. The idea is to:
Convert the file contents to a list
Split the innerLists
modify the list based on your condition
form the innerLists
finally writing it back to the same file.
open mode should be r+, so that it can both read and write to the file.
Let's see the code below:
with open('komad_namestaja.txt','r+') as f:
data = f.readlines()
output_data=[]
for i in data:
lst=i.split("|")
if(lst[0]=='312'):
lst[-1]='True\n'
output_data.append("|".join(lst))
f.seek(0)
f.writelines(output_data)
What you need to do is open a temporary file, write your result to that file and then replace the old file with your new file:
fh, abs_path = mkstemp() # Create a temporary file
with open(abs_path,'w') as new_file:
with open("komad_namestaja.txt", "r") as old_file:
for line in old_file:
linija = linija.split("|")
if parameter == linija[0]:
linija[6] = "True"
new_file.write(linija)
close(fh)
remove("komad_namestaja.txt") # Remove original file
move(abs_path, "komad_namestaja.txt") # Move new file to old file location
I am currently having problems displaying a file correctly once i have written a dictionary to the file. For this program the input file needs to have the format:
ID: Date: Dayskept: ProductName e.g. 1:12/12/2011:12:A
This is fine the first time I read the example file into a dictionary, but once i save the dictionay into a new file and try to open this file i get the output:
1:"date":12/12/2011, "life":12, "name":A
Is there an easy way to format the data in the dictionary before it is written to file?
Thanks for any advice given
def loadProduct(fileName):
global cheeseDictionary
f = open(fileName,"r")
line = f.readline() # Reads line from file
while line:
line = line[:-1]
data = split(line,":") # Splits line when there is a colon
cheeseDictionary[data[0]] = {"date":data[1], "life":data[2], "name":data[3]} # Stores each split item
line = f.readline() # Next line
f.close()
print cheeseDictionary
def saveProduct(fileName):
global cheeseDictionary
f = open(fileName,"w")
pickle.dump(cheeseDictionary, f)
f.close()
Since you have a specific format you want, you're going to need to write code to emit that format. (I don't know what you're trying to do with pickle in there, that produces a binary format that doesn't bear any resemblance to what you say you're getting.)
For instance, you could redefine saveProduct like so:
def saveProduct(fileName, cheeseDictionary):
f = open(fileName, "w")
for i in sorted(cheeseDictionary.keys()):
v = cheeseDictionary[i]
f.write("%s:%s:%s:%s\n" % (i, v["date"], v["life"], v["name"]))