We Keep Coding

Python Programing - Os Module - Replace

What I Want
I have written a code that opens a file (currentcode) gets it's text finds it in another file (text.txt) and replaces currentcode with a new int.
My Code
import os
currentcode = open('currentcode.txt','r+')
code = currentcode.read()
print('Choose File: ')
print('1: File One > ')
file = input('Enter Number Of File: ')
file = 'C:/text.txt'
old_text = code
new_text = str(int(code) + 1)
print('Opened File')
f1 = open(file, 'r')
f2 = open(file, 'w')
f2.write(replace(old_text, new_text))
currentcode.write(new_text)
f1.close()
f2.close()
Output After Running
When I Run This Code I Get:
Choose File:
1: File One >
Enter Number Of File: 1
Opened File
Traceback (most recent call last):
File "C:\Users\DanielandZoe\Desktop\scripys\Replace.py", line 18, in <module>
f2.write(replace(old_text, new_text))
NameError: name 'replace' is not defined

NameError: name 'replace' is not defined
That means python couldn't find a module, class or function called 'replace'.
If you want to replace text on a file, you need to get its contents as a string, not as a file-like object (like you're doing now), then you replace the contents using the replace method on your string and finally, write the contents to the desired file.

string.replace() is a method for strings:
https://docs.python.org/2/library/string.html#string.replace
what is in f2? Not a string. You should read the lines of the file.

Splitting database throwing errno 9 bad file descriptor with CON file

I've done some research and from what I can tell this normally happens when a file is closed before it's finished being used?
But that doesn't make sense for what is happening here.
Here is my code:
import csv
dicto = {}
name = ""
with open(input("enter filepath here: "), "r") as mainfile:
reader = csv.reader(mainfile)
for row in reader:
name = row[8].lstrip("'")
name = name.lstrip("\xa0")
name1 = name
name = name.upper()
if not name[:3] in dicto:
dicto[name[:3]] = [name[:3]+".js", 0]
with open(dicto[name[:3]][0], "w") as file1: #here is the problem line
file1.write("tags=[")
else:
dicto[name[:3]][1] += 1
if name[:1] == "#":
print(name)
with open(dicto[name[:3]][0], "a") as file2:
if dicto[name[:3]][1]>0:
file2.write('various spam')
else:
file2.write('various eggs')
for key in dicto.keys():
with open(dicto[key][0], "a") as file3:
file3.write("\n];")
I'm running through a large database and splitting it up into JS files which are named after the first three letters of the data's label. It seems to run ok at first (there are 44k entries to go through so it takes a few seconds to finish). Overall I currently have 309 files generated, although none are complete. However Once it gets to the combo "CON" an error occurs:
Traceback (most recent call last):
File "C:\Users\SarbickiN\Documents\Codes\Python\schools\schools.py", line 16, in <module>
with open(dicto[name[:3]][0], "w") as file1:
OSError: [Errno 9] Bad file descriptor: 'CON.js'
and this shuts down the programme. Is there any reason why this would have happened? I've made a comment next to the line causing the issue.
Edit: solution (or lack thereof)
CON is a reserved name for files in windows along with a few others so would need to be replaced with something else. Check here for more details.

Self answering
CON is a reserved name for files in windows along with a few others so would need to be replaced with something else. Check here for more details.

Python: data to file then data from text file to list - TypeError: must be str, not bytes

I'm a beginner in programming and have decided to teach myself Python. After a few days, i've decided to code a little piece. I's pretty simple:
date of today
page i am at (i'm reading a book)
how i feel
then i add the data in a file. every time i launch the program, it adds a new line of data in the file
then i extract the data to make a list of lists.
truth is, i wanted to re-write my program in order to pickle a list and then unpickle the file. However, as i'm coping with an error i can't handle, i really really want to understand how to solve this. Therefore i hope you will be able to help me out :)
I've been struggling for the past hours on this apparently a simple and stupid problem. Though i don't find the solution. Here is the error and the code:
ERROR:
Traceback (most recent call last):
File "dailyshot.py", line 25, in <module>
SaveData(todaysline)
File "dailyshot.py", line 11, in SaveData
mon_pickler.dump(datatosave)
TypeError: must be str, not bytes
CODE:
import pickle
import datetime
def SaveData(datatosave):
with open('journey.txt', 'wb') as thefile:
my_pickler = pickle.Pickler(thefile)
my_pickler.dump(datatosave)
thefile.close()
todaylist = []
today = datetime.date.today()
todaylist.append(today)
page = input('Page Number?\n')
feel = input('How do you feel?\n')
todaysline = today.strftime('%d, %b %Y') + "; " + page + "; " + feel + "\n"
print('Thanks and Good Bye!')
SaveData(todaysline)
print('let\'s make a list now...')
thefile = open('journey.txt','rb')
thelist = [line.split(';') for line in thefile.readlines()]
thefile.close()
print(thelist)
Thanks a looot!

Ok so there are a few things to comment on here:
When you use a with statement, you don't have to explicitly close the file. Python will do that for you at the end of the with block (line 8).
You don't use todayList for anything. You create it, add an element and then just discard it. So it's probably useless :)
Why are you pickling string object? If you have strings just write them to the file as is.
If you pickle data on write you have to unpickle it on read. You shouldn't write pickled data and then just read the file as a plain text file.
Use a for append when you are just adding items to the file, w will overwrite your whole file.
What I would suggest is just writing a plain text file, where every line is one entry.
import datetime
def save(data):
with open('journey.txt', 'a') as f:
f.write(data + '\n')
today = datetime.date.today()
page = input('Page Number: ')
feel = input('How do you feel: ')
todaysline = ';'.join([today.strftime('%d, %b %Y'), page, feel])
print('Thanks and Good Bye!')
save(todaysline)
print('let\'s make a list now...')
with open('journey.txt','r') as f:
for line in f:
print(line.strip().split(';'))

Are you sure you posted the right code? That error can occur if you miss out the "b" when you open the file
eg.
with open('journey.txt', 'w') as thefile:
>>> with open('journey.txt', 'w') as thefile:
... pickler = pickle.Pickler(thefile)
... pickler.dump("some string")
...
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
TypeError: must be str, not bytes
The file should be opened in binary mode
>>> with open('journey.txt', 'wb') as thefile:
... pickler = pickle.Pickler(thefile)
... pickler.dump("some string")
...
>>>

Converting a python code for Mac to Windows

I am new to python and I have a question about a piece of python code that creates a cleaned up output file from a model output file. This code was written for a Mac user, but now I want to run it in Windows. But it gives an error message. Could you help me in converting this code so I can use it in Windows? Thanks.
import sys
if len(sys.argv) > 1:
fileName = sys.argv[1]
else:
print "selected_um_b.out" #insert file name here
sys.exit()
f = open(fileName)
counter = 0
fw = open(fileName+".cleaned", 'w')
for line in f:
line = line.strip()
counter = counter + 1
if counter <= 4:
fw.write(line+"\n");
continue
values = line.split("\t")
if (values[4].strip() == "-99" or values[5].strip() == "0"): continue
fw.write("\t".join(values)+"\n")
f.close()
Update
The error message is:
Traceback (most recent call last): File
"C:\trial_batch\clean_output.py", line 7, in sys.exit()
SystemExit

The program expects a filename on the command line when you execute it. It appears you did not provide one, so the program exited (the sys.exit() call terminates the program).
How are you trying to use it? If you just want to convert one file, put the file and the Python script into the same directory. Replace lines 3 through 7 with filename = "yourfilename.typ" (do not indent the line); it will read the file ("yourfilename.typ" in my example) and write an output file with 'cleaned' in the filename.

Slow python file I:O; Ruby runs better than this; Got the wrong language?

Please advise - I'm going to use this asa learning point. I'm a beginner.
I'm splitting a 25mb file into several smaller file.
A Kindly guru here gave me a Ruby sript. It works beautifully fast. So, in order to learn I mimicked it with a python script. This runs like a three-legged cat (slow). I wonder if anyone can tell me why?
My python script
##split a file into smaller files
###########################################
def splitlines (file) :
fileNo=0001
outFile=open("C:\\Users\\dunner7\\Desktop\\Textomics\\Media\\LexisNexus\\ele\\newdocs\%s.txt" % fileNo, 'a') ## open file to append
fh = open(file, "r") ## open the file for reading
mylines = fh.readlines() ### read in lines
for line in mylines: ## for each line
if re.search("Copyright ", line): # if the line is equal to the regex
outFile.close() ## close the file
fileNo +=1 #and add one to the filename, starting to read lines in again
else: # otherwise
outFile=open("C:\\Users\\dunner7\\Desktop\\Textomics\\Media\\LexisNexus\\ele\\newdocs\%s.txt" % fileNo, 'a') ## open file to append
outFile.write(line) ## then append it to the open outFile
fh.close()
The guru's Ruby 1.9 script
g=0001
f=File.open(g.to_s + ".txt","w")
open("corpus1.txt").each do |line|
if line[/\d+ of \d+ DOCUMENTS/]
f.close
f=File.open(g.to_s + ".txt","w")
g+=1
end
f.print line
end

There are many reasons why your script is slow -- the main reason being that you reopen the outputfile for almost every line you write. Since the old file gets implicitly closed on opening a new one (due to Python garbage collection), the write buffer is flushed for every single line you write, which is quite expensive.
A cleaned up and corrected version of your script would be
def file_generator():
file_no = 1
while True:
f = open(r"C:\Users\dunner7\Desktop\Textomics\Media"
r"\LexisNexus\ele\newdocs\%s.txt" % file_no, 'a')
yield f
f.close()
file_no += 1
def splitlines(filename):
files = file_generator()
out_file = next(files)
with open(filename) as in_file:
for line in in_file:
if "Copyright " in line:
out_file = next(files)
out_file.write(line)
out_file.close()

I guess the reason your script is so slow is that you open a new file descriptor for each line. If you look at your guru's ruby script, it closes and opens the output file only if your separator matches.
In contrast to that, your python script opens a new file descriptor for every line you read (and btw, does not close them). Opening a file requires talking to the kernel, so this is relatively slow.
Another change I would suggest is to change
fh = open(file, "r") ## open the file for reading
mylines = fh.readlines() ### read in lines
for line in mylines: ## for each line
to
fh = open(file, "r")
for line in fh:
With this change, you do not read the whole file into memory, but only block after block. Although it should not matter with a 25MiB file, it will hurt you with big files and is good practice (and less code ;)).

The Python code might be slow due to regex and not IO. Try
def splitlines (file) :
fileNo=0001
outFile=open("newdocs/%s.txt" % fileNo, 'a') ## open file to append
reg = re.compile("Copyright ")
for line in open(file, "r"):
if reg.search("Copyright ", line): # if the line is equal to the regex
outFile.close() ## close the file
outFile=open("newdocs%s.txt" % fileNo, 'a') ## open file to append
fileNo +=1 #and add one to the filename, starting to read lines in again
outFile.write(line) ## then append it to the open outFile
Several notes
Always use / instead of \ for path name
If regex is used repeatedly, compile it
Do you need re.search? or re.match?
UPDATE:
#Ed. S: point taken
#Winston Ewert: code updated to be closer to the original Ruby code

rosser,
Don't use names of built-in objects as identifiers in a code (file, splitlines)
The following code respects the effect of your own code: an out_file is closed without the line containing 'Copyright ' that constitutes the signal of closing
The use of the function writelines() is intended to obtain a faster execution than with a repetition of out_file.write(line)
The if li: block is there to trigger the closing of out_file in case the last line of the read file doesn't contains 'Copyright '
def splitfile(filename, wordstop, destrep, file_no = 1, li = []):
with open(filename) as in_file:
for line in in_file:
if wordstop in line:
with open(destrep+str(file_no)+'.txt','w') as f:
f.writelines(li)
file_no += 1
li = []
else:
li.append(line)
if li:
with open(destrep+str(file_no)+'.txt','w') as f:
f.writelines(li)