I am trying to read and write to a text file . The reading and writing parts works fine but the actual file does not update until after the program has executed...I understand that this is because the data is being stored in a buffer and is being uploaded after.so I came across this How come a file doesn't get written until I stop the program?
and tried the .flush, os.sync, etc: this did not have an affect though..maybe I'm not seeing something.
Note that the .flush does not work in the Postdata sub ... I think it's because of the way that subroutine is coded.
Read does as it is expected.
Post takes an index and a line index and edits that specific text at that position.
def Getdata(Index,lineindex):#indexed so can say get data at index 3 and it will return it
Datafile = open("Trade data/3rd file", "a+")
Linetoget = linecache.getline('Trade data/Databaseforbot', lineindex).split("|")
Traddetail = Linetoget[Index]
print(Traddetail)
return Traddetail
def Postdata(index,lineindex,data):#will work fine the first time, but run it as PostdataV1(3,2) it will convert
Getdata(3,2)
with fileinput.FileInput('Trade data/Databaseforbot', inplace=True, backup='.bak') as file:
entireline= linecache.getline('Trade data/Databaseforbot', lineindex)
splitted = entireline.split("|")
Traddetail = splitted[index]
Newline = entireline.replace(Traddetail, str(index+1)+"*"+data)
for line in file:
print(line.replace(entireline, Newline), end='')
#os.fsync(file)
file.close()
Getdata(3, 2)
Postdata(3,2,"QW")
Getdata(3, 2)
The data file stores this data:
1|https://app.libertex.com/products/stock/BA/|3*45#4|4*0|5*0|6*0|7*0|8*0|9*Up|CDwindow-5C5C0883A51583A013B50FDC5A1798B7
2|https://app.libertex.com/products/energetics/NG/|3*56#5|4*0|5*0|6*0|7*0|8*0|9*Up|CDwindow-5C5C0883A51583A013B50FDC5A1798B7
3|https://app.libertex.com/products/metal/XAUUSD/|3*45#4|4*0|5*0|6*0|7*0|8*0|9*Up|CDwindow-5C5C0883A51583A013
Is there a way to live update the file so I can call other parts of the code to read the data from the file...I will be using something like getch to run other stuff...I don't mind if I have to pause postng data while reading... I tried doing a second file that the data eg: filex which is read from in Getdata() and the post data first writes to filey then copy everything to filex, but that did not work either.
Also there wil be around maybe 10-50 lines in the text file, if that helps.
Related
so first and foremost i wanted to connect multiple lines into one and add ","
so in example
line1
line2
line2
to
line1,line2,line3
i managed to make it work with this script right here
filelink = input("Enter link here ")
fix = file = open(filelink, "r")
data=open(filelink).readlines()
for n,line in enumerate(data):
if line.startswith("line"):
data[n] = "\n"+line.rstrip()
else:
data[n]=line.rstrip()
print(','.join(data))
HOWEVER in the terminal itself it shows it executed perfectly but in the text file itself it's still remains the same no connected lines and no commas
side note. i would love some explanations how does the loop work and what "enumerate" stands for and why specifically this code i tried googling each one separately and understand the code but i didn't manage to find what i was looking for if anyone keen to explain the code line by line shortly i would be very appreciative
Thanks in advance <3
This is somewhat superfluous:
fix = file = open(filelink, "r")
That's assigning two names to the same file object, and you don't even use fix, so at least drop that part.
For handling files, you would be better using a context manager. That means that you can open a resource and they will automatically get closed for you once you're done (usually).
In any case, you opened in read mode with open(filelink, "r") so you'll never change the file contents. print(','.join(data)) will probably show you what you expect, but print() writes to stdout and the change will only be in your terminal. You will not modify the base file with this. But, I think you're sufficiently close that I'll try close the missing connection.
In this case, you need to:
open the file first in read mode to pull the data out.
Do the transform in python to the data
Open the file again in write mode (which wipes the existing contents)
Write the transformed data
So, like this:
filelink = input("Enter link here ")
with open(filelink) as infile: # context manager, by default in "r" mode
data = [item.strip() for item in infile.readlines()]
data = ','.join(data)
# Now write it back out
with open(filelink, "w") as outfile:
outfile.write(data)
so, my program will replace the old data with new data which is the program will put in same .csv file. and after i run the program it is not replaced. below is my code.
TEXTFILE = open("data.csv", "w")
for i in book_list:
TEXTFILE.write("{},{},{},{}".format(i[0],i[1],i[2],i[3]))
TEXTFILE.close()
book_list is the list that save new data to stored
the result i got:
k,k,45,c
a,a,65,r
d,s,65,r
as,as,65,r
df,df6,65,r
as,as,6,r
as,as,46,r
as,as,45,r
as,as,56,rk,k,45,r
a,a,65,r
d,s,65,r
as,as,65,r
df,df6,65,r
as,as,6,r
as,as,46,r
as,as,45,r
as,as,56,r
it stored to csv with combining old and new content.
well the origininal file is looks like this:
k,k,45,r
a,a,65,r
d,s,65,r
as,as,65,r
df,df6,65,r
as,as,6,r
as,as,46,r
as,as,45,r
as,as,56,r
idk how to explain. but I expect that the result will change one line with new data in the fourth row. for example, the previous line is k,k,45,r (line 1). and the program will change it become k,k,45,c that way
hope you all can help me :)
Try using .truncate() , if called after opening file it destroys it's content.
TEXTFILE = open("data.csv", "w")
TEXTFILE.truncate()
for i in book_list:
TEXTFILE.write("{},{},{},{}".format(i[0],i[1],i[2],i[3]))
TEXTFILE.close()
I hope I understood you correctly.
I am working on a project that requires me to read a file with a .dif extension. Dif stands for data information exchange. The file opens nicely in Open Office Calc. Then you can easily save as a csv file, however when I open in Python all I get are random characters that don't make sense. Here is the last code that I tried just to see if I could read.
txt = open('C:\myfile.dif', 'rb').read()
print txt
I would even be open to programatically converting the file to csv first. before opening if someone knows how to do that. As always, any help is much appreciated. Below is a partial screenshot of what I get when I run the code.
Hadn't heard of this file format. Went and got a sample here.
I tested your method and it works fine:
>>> content = open(r"E:\sample.dif", 'rb').read()
>>> print (content)
b'TABLE\r\n0,1\r\n"EXCEL"\r\nVECTORS\r\n0,8\r\n""\r\nTUPLES\r\n0,3\r\n""\r\nDATA\r\n0,0\r\n""\r\n-1,0\r\nBOT\r\n1,0\r\n"Welcome to File Extension FYI Center!"\r\n1,0\r\n""\r\n1,0\r\n""\r\n-1,0\r\nBOT\r\n1,0\r\n""\r\n1,0\r\n""\r\n1,0\r\n""\r\n-1,0\r\nBOT\r\n1,0\r\n"ID"\r\n1,0\r\n"Type"\r\n1,0\r\n"Description"\r\n-1,0\r\nBOT\r\n0,1\r\nV\r\n1,0\r\n"ASP"\r\n1,0\r\n"Active Server Pages"\r\n-1,0\r\nBOT\r\n0,2\r\nV\r\n1,0\r\n"JSP"\r\n1,0\r\n"JavaServer Pages"\r\n-1,0\r\nBOT\r\n0,3\r\nV\r\n1,0\r\n"PNG"\r\n1,0\r\n"Portable Network Graphics"\r\n-1,0\r\nBOT\r\n0,4\r\nV\r\n1,0\r\n"GIF"\r\n1,0\r\n"Graphics Interchange Format"\r\n-1,0\r\nBOT\r\n0,5\r\nV\r\n1,0\r\n"WMV"\r\n1,0\r\n"Windows Media Video"\r\n-1,0\r\nEOD\r\n'
>>>
The question is what is in the file and how do you want to handle it. Personally I liked:
with open(r"E:\sample.dif", 'rb') as f:
for line in f:
print (line)
In the first code block, that long line that has a b'' (for bytes!) in front of it can be iterated on \r\n:
b'TABLE\r\n'
b'0,1\r\n'
b'"EXCEL"\r\n'
b'VECTORS\r\n'
b'0,8\r\n'
b'""\r\n'
b'TUPLES\r\n'
b'0,3\r\n'
b'""\r\n'
b'DATA\r\n'
b'0,0\r\n'
.
.
.
b'"Windows Media Video"\r\n'
b'-1,0\r\n'
b'EOD\r\n'
Similar to posting: Replace string in a specific line using python, however results were not forethcomming in my slightly different instance.
I working with python 3 on windows 7. I am attempting to batch edit some files in a directory. They are basically text files with .LIC tag. I'm not sure if that is relevant to my issue here. I am able to read the file into python without issue.
My aim is to replace a specific string on a specific line in this file.
import os
import re
groupname = 'Oldtext'
aliasname = 'Newtext'
with open('filename') as f:
data = f.readlines()
data[1] = re.sub(groupname,aliasname, data[1])
f.writelines(data[1])
print(data[1])
print('done')
When running the above code I get an UnsupportedOperation: not writable. I am having some issue writing the changes back to the file. Based on suggestion of other posts, I edited added the w option to the open('filename', "w") function. This causes all text in the file to be deleted.
Based on suggestion, the r+ option was tried. This leads to successful editing of the file, however, instead of editing the correct line, the edited line is appended to the end of the file, leaving the original intact.
Writing a changed line into the middle of a text file is not going to work unless it's exactly the same length as the original - which is the case in your example, but you've got some obvious placeholder text there so I have no idea if the same is true of your actual application code. Here's an approach that doesn't make any such assumption:
with open('filename', 'r') as f:
data = f.readlines()
data[1] = re.sub(groupname,aliasname, data[1])
with open('filename', 'w') as f:
f.writelines(data)
EDIT: If you really wanted to write only the single line back into the file, you'd need to use f.tell() BEFORE reading the line, to remember its position within the file, and then f.seek() to go back to that position before writing.
The last line of my file is:
29-dez,40,
How can I modify that line so that it reads:
29-Dez,40,90,100,50
Note: I don't want to write a new line. I want to take the same line and put new values after 29-Dez,40,
I'm new at python. I'm having a lot of trouble manipulating files and for me every example I look at seems difficult.
Unless the file is huge, you'll probably find it easier to read the entire file into a data structure (which might just be a list of lines), and then modify the data structure in memory, and finally write it back to the file.
On the other hand maybe your file is really huge - multiple GBs at least. In which case: the last line is probably terminated with a new line character, if you seek to that position you can overwrite it with the new text at the end of the last line.
So perhaps:
f = open("foo.file", "wb")
f.seek(-len(os.linesep), os.SEEK_END)
f.write("new text at end of last line" + os.linesep)
f.close()
(Modulo line endings on different platforms)
To expand on what Doug said, in order to read the file contents into a data structure you can use the readlines() method of the file object.
The below code sample reads the file into a list of "lines", edits the last line, then writes it back out to the file:
#!/usr/bin/python
MYFILE="file.txt"
# read the file into a list of lines
lines = open(MYFILE, 'r').readlines()
# now edit the last line of the list of lines
new_last_line = (lines[-1].rstrip() + ",90,100,50")
lines[-1] = new_last_line
# now write the modified list back out to the file
open(MYFILE, 'w').writelines(lines)
If the file is very large then this approach will not work well, because this reads all the file lines into memory each time and writes them back out to the file, which is very inefficient. For a small file however this will work fine.
Don't work with files directly, make a data structure that fits your needs in form of a class and make read from/write to file methods.
I recently wrote a script to do something very similar to this. It would traverse a project, find all module dependencies and add any missing import statements. I won't clutter this post up with the entire script, but I'll show how I went about modifying my files.
import os
from mmap import mmap
def insert_import(filename, text):
if len(text) < 1:
return
f = open(filename, 'r+')
m = mmap(f.fileno(), os.path.getsize(filename))
origSize = m.size()
m.resize(origSize + len(text))
pos = 0
while True:
l = m.readline()
if l.startswith(('import', 'from')):
continue
else:
pos = m.tell() - len(l)
break
m[pos+len(text):] = m[pos:origSize]
m[pos:pos+len(text)] = text
m.close()
f.close()
Summary: This snippet takes a filename and a blob of text to insert. It finds the last import statement already present, and sticks the text in at that location.
The part I suggest paying most attention to is the use of mmap. It lets you work with files in the same manner you may work with a string. Very handy.