I am trying to write a script that opens files based on dictionary values. For each key/value it opens a file based on that value's name and assigns that file to the name of the value (I think it's going wrong here). So far it opens the files for me, with the right names, so that works. However, I think the name I assign the open(file) function to is wrong, since the rest of my function does not open the files anymore, and I can't close them.
Example of my script:
filelist=[]
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
for value in codeconv.values():
value=open("c:\Biochemistry\Pythonscripts\Splittest\split"+value+".txt", 'w')
filelist.append(value)
"Here I do something with the files"
for f in filelist:
f.close()
So the problem is, how do I assign the open(file) function to the correct name? (Here r1p1d and r2p1d for example) And how do I later call those again?
The error I get now is:
AttributeError: 'str'object has no attribute 'close'
on the f.close() line.
EDIT: It now works as I want it to, using the following code: (I also included the 'here I do something' part now just for clarity)
result=open("C:\\Biochemistry\\Pythonscripts\\Illuminaresults.txt", "r")
filelist=[]
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
opened_files={}
for key, value in codeconv.items():
filename="c:\Biochemistry\Pythonscripts\Splittest\split"+value+".txt"
file=open(filename, 'w')
opened_files.update({key: file})
for line in result:
if line[0]==">":
lastline=line
if line[0:3] in codeconv and len(line)==64:
f=opened_files[line[0:3]]
f.write(lastline+line)
else: continue
for f in opened_files.values():
f.close()
result.close()
I got another problem now though when I try to write the next part of my script, but that's probably something for another question, as it gives a Windowserror not related to this part. Thanks for the help all!
If i understand you well, you want to create some variable dynamically from the dictionary so that you can assign them to the opened files, is that it ???!!!!
I will suggest to do it using another dictionary like this:
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
opened_files = {}
for key, value in codeconv.items():
file_name = "c:\Biochemistry\Pythonscripts\Splittest\split%s.txt" % value
file=open(file_name, 'w')
opened_files.update({key: file})
you can now access your opened files from the dictionary like this:
f = opened_files['agt']
f.read()
....
and for you latter code do it like this:
for f in opened_files.values():
f.close()
I'd caution against reusing "value" in that first loop, for clarity and future bugs that might be introduced there.
But, the code you posted actually works, so obviously there's something else going on that we can't diagnose here. What you need to do is inspect the contents of filelist before you try to close the file handles in it. That will probably point you towards the answer.
Perhaps you want to append tuples to filelist so the name is associated with the file object
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
for value in codeconv.values():
f=open("c:\Biochemistry\Pythonscripts\Splittest\split"+value+".txt", 'w')
filelist.append((value, f))
"Here I do something with the files"
for name, f in filelist:
f.close()
I can't see a problem in your code, maybe it's not in the part pasted? (in filelist, or the snipped bit).
But anyway, I would rewrite it to avoid using the same variable, value, for two different things:
filelist = []
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
for file_id in codeconv.values():
f = open("c:\Biochemistry\Pythonscripts\Splittest\split"+file_id+".txt", 'w')
filelist.append(f)
"Here I do something with the files"
for f in filelist:
if isinstance(f, str):
print "WARNING: Expected file handle, got string:", f
else:
f.close()
(I also added a bit of troubleshooting code)
The files are already open - your filelist object contains open file objects that you can iterate over (for example, with for line_of_text in filelist[0]:) or call other functions of (see dir(file) for other members).
You can defer opening the file by assigning a lambda and calling it later, for example:
for value in codeconv.values():
value2 = lambda: open(complete_filename)
filelist.append(value2)
my_file_object = filelist[0]()
You may prefer to store these in a dictionary:
for value in codeconv.itervalues():
filedict[value] = lambda: open(complete_filename)
my_file_object = filedict["r1p1d"]()
Or if you really want to create new variables (I strongly recommend not doing this, but since you asked):
for value in codeconv.itervalues():
globals()[value] = open(complete_filename)
# or
#globals()[value] = lambda: open(complete_filename)
# if you prefer
Finally, you close the files as you already are (substituting filedict.itervalues() for filelist if you use a dictionary instead).
(Obviously you need to replace complete_filename in the above examples with however you calculate the actual filename. I shouldn't need to say this, but I've been stung too often by leaving out these sorts of details.)
You're not assigning the function, you are assigning the return value. Which appearantly is a string in this case instead of a file object, which is a bit surprising. Are you doing anything with the filelist in the "here I do something" part?
[edit]
ok, sounds like you want a dict between key/fileobject (with example usage).
codeconv={"agt":"r1p1d", "aga":"r2p1d"}
filedict = {key, open("c:\Biochemistry\Pythonscripts\Splittest\split"+value+".txt", 'w')
for key, value in codeconv.iteritems()}
#Here I do something with the files
filedict["agt"].write("foo")
filedict["aga"].write("bar")
for f in filedict.values():
f.close()
Related
I'm trying to write a function such as the one below except it it'd take the list and filename as parameter. It's annoyingly basic and yet it escapes me.
def write_list_to_file():
wordlst = ["No", "Time", "for", "this"]
file = open(filename, 'w')
for items in wordlst:
file.writelines(items + '\n')
file.close()wordlst
Given that this works, this:
def write_list_to_file(wordlst, filename):
wordlst = ["No", "Time", "for", "this"]
with open(filename+".txt",'w') as file:
for items in wordlst:
file.writelines(items + '\n')
Should too. Except that calling the function in the fashion of write_list_to_file(wordlst, Idk) returns no file. Fully aware that the list remains static I've tried a single parameter function in the same fashion, I.e :
def makefile(filename):
file = open(filename + ".txt", 'w')
file.close()
makefile("Really")
This yields no file either somehow. Please do ignore the list's elements, i've been on this a lot longer than i care to admit and i couldn't find something that helps solve this particular issue. I've found countless solutions to make this happen, but not in the shape of a function taking any list and any file as input. In every case the exit code shows no error, so i'd at least expect a file to be create yet cant find any.
TLDR: tryin to make a write_list_to_file(wordlst,filename) function, must be missing stupidly obvious, help appreciated.
Edit: approved, the indention issue was only in this post though, the code was indented properly, made this in a hurry
Edit2: cf comments
Your code examples do not have the right indentation, which is important when writing python code.
Futhermore, writelines() takes in a list of strings. You iterate through your list and want to save each element on a seperate line - write() would be the right function for that.
Last, make sure to pass your filename as a string.
This code should work:
def write_list_to_file(wordlst, filename):
with open(filename + '.txt', 'w') as file:
for item in wordlst:
file.write(item + '\n')
write_list_to_file(['element1', 'element2', 'element3'], 'tempfile')
I'm trying to take a dictionary and write it to a file. Each line of output is suppose to contained the key and its value separated by a single space. So far I have the following:
def save(diction):
savefile = open("save.txt", "w")
for line in diction:
values = line.split()
savefile.write(values)
savefile.close()
I'm not sure if this is writing to the file or if it even saves the file with the .close() function. Any advice?
The values are being written to the file on savefile.write(values), however the method with which you are opening and closing the file is a bit dangerous. If you encounter an error the file may never be closed. It's better to use with to ensure that the file will automatically be closed when leaving the with block, whether by normal execution or on error. Also, you probably mean to iterate through diction.items()?
def save(diction):
with open("save.txt", "w") as savefile:
for key, value in diction.items():
values = "%s %s\n" % (key, value)
savefile.write(values)
If you try to call this function, it will give you an exception, like this:
----> 7 savefile.write(values)
TypeError: must be str, not list
… and then quit. So no, it's not writing anything, because there's an error in your code.
What does that error mean? Well, obviously something in that line is a list when it should be a str. It's not savefile, it's not write, so it must be values. And in fact, values is a list, because that's what you get back from split. (If it isn't obvious to you, add some debugging code into the function to print(values), or print(type(values)) and run your code again.)
If you look up the write method in the built-in help or on the web, or re-read the tutorial section Methods of File Objects, you will see that it does in fact require a string.
So, how do you write out a list?
You have to decide what you want it to look like. If you want it to look the same as it does in the print statement, you can just call str to get that:
savefile.write(str(values))
But usually, you want it to be something that looks nice to humans, or something that's easy to parse for later scripts you write.
For example, if you want to print out each values as a line made up of 20-character-wide columns, you could do something like this:
savefile.write(''.join(format(value, '<20') for value in values) + '\n')
If you want something you can parse later, it's usually better to use one of the modules that knows how to write (and read) specific formats—csv, json, pickle, etc.
The tutorial chapter on Input and Output is worth reading for more information.
def save_dict(my_dict):
with open('save.txt', 'w') as f:
for key in my_dict.keys():
f.write(key + ' ' + my_dict[key] + '\n')
if __name__ == "__main__":
my_dict = {'a': '1',
'b': '2'}
save_dict(my_dict)
def save(diction):
with open('save.txt', 'w') as savefile:
for key, value in diction.items():
savefile.write(str(key)+ ' ' + str(value) + '\n')
I'm exploring python and tried to sort all files from a directory by last modified and then write the list into a txt file.
import time
import os
i=1
a="path"
def getfiles(dirpat):
b = [s for s in os.listdir(dirpat)
if os.path.isfile(os.path.join(dirpat, s))]
b.sort(key=lambda s: os.path.getmtime(os.path.join(dirpat, s)))
return b
lyst=[]
testfile='c://test.txt'
lyst=getfiles(a)
for x in range (0,len(lyst)):
print lyst[x]
fileHandle = open (testfile, 'w' )
fileHandle.write ("\n".join(str(lyst[x])))
fileHandle.close()
It printed perfectly and sorted by date also
example1.pdf
example3.docx
example4.docx
exmaple2.docx
example 5.doc
But when I opened the file, it just had the last entry and displayed it like this
e
x
a
... and so on
Just can't figure out where the problem lies. If I remove "\n".join it just prints me the last entry.
Thanks in advance,
Nils
Correct the join(), e.g:
'\n'.join(str(path) for path in list)
And please rename the "list" variable, because list is a built-in data type in Python.
import os, os.path
a="path"
def getfiles(dirpat):
b = [s for s in os.listdir(dirpat)
if os.path.isfile(os.path.join(dirpat, s))]
b.sort(key=lambda s: os.path.getmtime(os.path.join(dirpat, s)))
return b
outfile='c://test.txt'
with open(outfile, 'w') as fileHandle:
lines = getfiles(a)
for line in lines:
print line
fileHandle.write(line)
Avoid using meaningless one character variable names. I also did not touch your getfiles() function. I did, however, rename file and list as those are both the names of built in functions, which you're hiding when you use those names.
You also only need to open the file once, not once per line. You were truncating the file on every write. Using with makes sure the file handle gets closed even if you have an error.
Edit: If you don't need to print out each line before writing it, you can just have one line inside the with block: fileHandle.writelines(getfiles(a)).
You are opening and overwriting the file contents in each iteration of the loop.
Pass 'a' to the open(path) call to append to the file, or simply open it once outside the loop and close it outside the loop.
Because you convert each entry in the list to str the join operates on each
str because they also count as iterables and therefore a \n is put between
each character, not every item in the list. Changing the line to fileHandle.write ('\n'.join(str(path) for path in list))) will fix this just like BasicWolf wrote.
Do this when writing the file:
fileHandle = open (file, 'w' )
for listItem in list:
print listItem
fileHandle.write (str(listItem) + "\n")
fileHandle.close()
I'm trying to replace a string in all the files within the current directory. for some reason, my temp file ends up blank. It seems my .write isn't working because the secondfile was declared outside its scope maybe? I'm new to python, so still climbing the learning curve...thanks!
edit: I'm aware my tempfile isn't being copied currently. I'm also aware there are much more efficient ways of doing this. I'm doing it this way for practice. If someone could answer specifically why the .write method fails to work here, that would be great. Thanks!
import os
import shutil
for filename in os.listdir("."):
file1 = open(filename,'r')
secondfile = open("temp.out",'w')
print filename
for line in file1:
line2 = line.replace('mrddb2.','shpdb2.')
line3 = line2.replace('MRDDB2.','SHPDB2.')
secondfile.write(line3)
print 'file copy in progress'
file1.close()
secondfile.close()
Just glancing at the thing, it appears that your problem is with the 'w'.
It looks like you keep overwriting, not appending.
So you're basically looping through the file(s),
and by the end you've only copied the last file to your temp file.
You'll may want to open the file with 'a' instead of 'w'.
Your code (correctly indented, though I don't think there's a way to indent it so it runs but doesn't work right) actually seems right. Keep in mind, temp.out will be the replaced contents of only the last source file. Could it be that file is just blank?
Firstly,
you have forgotten to copy the temp file back onto the original.
Secondly:
use sed -i or perl -i instead of python.
For instance:
perl -i -pe 's/mrddb2/shpdb2/;s/MRDDB2/SHPDB2/' *
I don't have the exact answer for you, but what might help is to stick some print lines in there in strategic places, like print each line before it was modified, then again after it was modified. Then place another one after the line was modified just before it is written to the file. Then just before you close the new file do a:
print secondfile.read()
You could also try to limit the results you get if there are too many for debugging purposes. You can limit string output by attaching a subscript modifier to the end, for example:
print secondfile.read()[:n]
If n = 100 it will limit the output to 100 characters.
if your code is actually indented as showed in the post, the write is working fine. But if it is failing, the write call may be outside the inner for loop.
Just to make sure I wasn't really missing something, I tested the code and it worked fine for me. Maybe you could try continue for everything but one specific filename and then check the contents of temp.out after that.
import os
for filename in os.listdir("."):
if filename != 'findme.txt': continue
print 'Processing', filename
file1 = open(filename,'r')
secondfile = open("temp.out",'w')
print filename
for line in file1:
line2 = line.replace('mrddb2.','shpdb2.')
line3 = line2.replace('MRDDB2.','SHPDB2.')
print 'About to write:', line3
secondfile.write(line3)
print 'Done with', filename
file1.close()
secondfile.close()
Also, as others have mentioned, you're just clobbering your temp.out file each time you process a new file. You've also imported shutil without actually doing anything with it. Are you forgetting to copy temp.out back to your original file?
I noticed sometimes it will not print to file if you don't have a file.close after file.write.
For example, this program never actually saves to file, it just makes a blank file (unless you add outfile.close() right after the outfile.write.)
outfile=open("ok.txt","w")
fc="filecontents"
outfile.write(fc.encode("utf-8"))
while 1:
print "working..."
#OP, you might also want to try fileinput module ( this way, you don't have to use your own temp file)
import fileinput
for filename in os.listdir("."):
for line in fileinput.FileInput(filename,inplace=1):
line = line.strip().replace('mrddb2.','shpdb2.')
line = line.strip().replace('MRDDB2.','SHPDB2.')
print line
set "inplace" to 1 for editing the file in place. Set to 0 for normal print to stdout
Not really too sure how to word this question, therefore if you don't particularly understand it then I can try again.
I have a file called example.txt and I'd like to import this into my Python program. Here I will do some calculations with what it contains and other things that are irrelevant.
Instead of me importing this file, going through it line-by-line and extracting the information I want.. can Python do it instead? As in, if I structure the .txt correctly (whether it be key / value pairs seperated by an equals on each line), is there a current Python 'way' where it can handle it all and I work with that?
with open("example.txt") as f:
for line in f:
key, value = line.strip().split("=")
do_something(key,value)
looks like a starting point if I understand you correctly. You need Python 2.6 or 3.x for this.
Another place to look is the csv module that can parse comma-separated value files - and you can tell it to use = as a separator instead. This will abstract away some of the "manual work" in that previous example - but it seems your example doesn't especially need that kind of abstraction.
Another idea:
with open("example.txt") as f:
d = dict([line.strip().split("=") for line in f])
Now that's concise and pythonic :)
for line in open("file")
key, value = line.strip().split("=")
key=key.strip()
value=value.strip()
do_something(key,value)
There's also another method - you can create a valid python file (let it be a list, dict definition or whatever else), read its content using
f = open('file.txt', r)
content = f.read() #assuming file isn't too long
And then just parse it:
parsedContent = eval(content)
You can pass any environment to eval (see docs), so it might not have access to your globals and locals. This is evil and wrong, but in small program that won't be distributed and won't get 'file.txt' from network or from so called malicious user - you can use it.