Related
I am writing a Streamlit app that takes in tensor output data from a .txt file, formats it, and both shows information on the data and prints the formatted data back to a new .txt file for later use.
After uploading the txt file to Streamlit and decoding it to a single long string, I alter the string and write it to a new txt file. When I open the txt file, the line spacings are huge, it looks like extra newlines have been put in but when you highlight the text, it is just large line spacings.
As well as this, when I use splitlines() on the string, the array that is returned is empty. This is the case even though the string is not empty and does contain newlines - I think it is to do with the large line spacings, but I am not sure.
The program is split into modules, but the code that is meant to format the file is in just two functions. One adds delimiters and works like this (with Streamlit as st):
def delim(file):
#read the selected file and write it to variable elems as a string
elems = file.decode('utf-8')
#replace the applicable parts of variable elems with the delimiters
elems = elems.replace('e+002', 'e+002, ')
elems = elems.replace('e+003', 'e+003, ')
elems = elems.replace('e+004', 'e+004, ')
elems = elems.replace('e+005', 'e+005, ')
elems = elems.replace('e+006', 'e+006, ')
elems = elems.replace('e+007', 'e+007, ')
elems = elems.replace('e+008', 'e+008, ')
elems = elems.replace('e+009', 'e+009, ')
with open('final_file.txt', 'w') as magma_file:
#write a txt file with the stored, altered text in variable elems
magma_file.write(elems)
#close the writeable file to be safe
magma_file.close()
st.success('Delimiters successfully added')
The second part, where I am getting the empty array, is in a second function. The whole function is not necessary to see the issue, but the part that is not working is here:
def addElem(file):
#create counting variables
counter = 0
linecount = 1
#put file as string in variable checks
checks = file.decode('utf-8')
checks.splitlines()
#check to see if the start of the file is formatted correctly. This is the part giving me strife
if checks[0].rstrip().endswith('5'):
with open('final_file.txt', 'w') as ff:
#iterate through the lines in the file
for line in checks:
counter+=1
# and so on, not relevant to the problem
The variable checks does contain a string after decoding the file, but when I use splitlines() then look inside checks[0], checks[1] etc., they are all empty. I tried commenting out other code, the conditional statement, removing the rstrip() and just seeing what was in the checks array after splitting the string, but it was still nothing. I tried changing splitlines() to split() using various delimiters including \n, but the array remained empty.
This program logic worked perfectly when I was running it locally using a console application interacting directly with the file system, so probably the problem is something to do with how a Streamlit "file like object" works. I read through the docs at Streamlit, but it doesn't give much detail on this.
This program is not for my use, so I can't keep it as a console app. I did ask about this on the Streamlit community a month ago, but so far no one has answered and I am not sure whether it is an unusual problem or just a terrible question.
I am wondering if there is a better way to decode the file to a string, but decoding to unicode doesn't explain the line spacings so I think something else is going on.
I have a database.txt file the first column is for usernames the second passwords and the rest 5 recovery question and answers alternating. I want to allow the user to be able to change the password of their details, without affecting another users username as they may be the same. I have found a way to delete the previous one and append the new line of modified details to the file. However, the is always a string or unknown characters at the start of the appended line. AND other characters are being changed not the second value in the list. Please help me find a way to avoid this.
https://repl.it/repls/NecessaryBoldButtonsYou can find the code here changing it will affect everyone, so please copy it elsewhere.
https://onlinegdb.com/BJbsn9-cL
I just need the password to be changed on a user input not other strings, the reason for all this code is that when changing a person's password another username could be changed.This is the original file
This is what happens afterwards, the second string in the list of the line which where data[0] = "bye" should only be changed to newpass, not all of the others
'''
import linecache
f = open("database.txt" , "r+")
for loop in range(3):
line = f.readline()
data = line.split(",")
if data[1] == "bye":
print(data[1]) #These are to help me understand what is happening
print(data[0])
b = data[0]
newpass = "Hi"
a = data[1]
fn = 'database.txt'
e = open(fn)
output = []
str="happy"
for line in e:
if not line.startswith(str):
output.append(line)
e.close()
print(output)
e = open(fn, 'w')
e.writelines(output)
e.close()
line1 = linecache.getline("database.txt" ,loop+1)
print(line)
password = True
print("Password Valid\n")
write = (line1.replace(a, newpass))
write = f.write(line1.replace(a, newpass))
f.close()
'''
This is the file in text:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3,Recovery4,Answer4,
Recovery5,Answer5,o,o,o,o,o,o,o,o,o,o,
happy,bye,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
Support is very much appreciated
Feel free to change the code as much as you need to, as it is already a mess
Thanks in Advance
This should be pretty easy. The basic idea is:
open input file for reading
open output file for writing
for each line in input file
if password = "happy"
change user name in line
write line to output file
It should be pretty easy to convert that to python.
From comments, and by examining your code, I get the feeling that you're trying to update a line in-place. That is, it looks like your expectation is that given the file "database.txt" that contains this:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3, Recovery4,Answer4,Recovery5,Answer5,
o,o,o,o,o,o,o,o,o,o,
happy,bye,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
When you make the change, your new "database.txt" will contain this:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3, Recovery4,Answer4,Recovery5,Answer5,
o,o,o,o,o,o,o,o,o,o,
happy,Hi,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
You can do that, but you can't do it in-place. You have to write all the lines of the file, including the changed line, to a new temporary file. Then you can delete the old "database.txt" and rename the temporary file.
You can't update a line in a text file, because if you change the length of the line then you'll either end up with extra space at the end of the line you changed (because the new line has fewer characters than the old line), or you'll overwrite the beginning of the next line (the new line is longer than the old line).
The only other option is to load all of the lines into memory and close the file. Then change the line or lines you want to change, in memory. Finally, open the "database.txt" file for writing and output all of the lines from memory to the file.
I am using the following code to upload a file on server using FTP after editing it:
import fileinput
file = open('example.php','rb+')
for line in fileinput.input('example.php'):
if 'Original' in line :
file.write( line.replace('Original', 'Replacement'))
file.close()
There is one thing, instead of replacing the text in its original place, the code adds the replaced text at the end and the text in original place is unchanged.
Also, instead of just the replaced text, it prints out the whole line. Could anyone please tell me how to resolve these two errors?
1) The code adds the replaced text at the end and the text in original place is unchanged.
You can't replace in the body of the file because you're opening it with the + signal. This way it'll append to the end of the file.
file = open('example.php','rb+')
But this only works if you want to append to the end of the document.
To bypass this you may use seek() to navigate to the specific line and replace it. Or create 2 files: an input_file and an output_file.
2) Also, instead of just the replaced text, it prints out the whole line.
It's because you're using:
file.write( line.replace('Original', 'Replacement'))
Free Code:
I've segregated into 2 files, an inputfile and an outputfile.
First it'll open the ifile and save all lines in a list called lines.
Second, it'll read all these lines, and if 'Original' is present, it'll replace it.
After replacement, it'll save into ofile.
ifile = 'example.php'
ofile = 'example_edited.php'
with open(ifile, 'rb') as f:
lines = f.readlines()
with open(ofile, 'wb') as g:
for line in lines:
if 'Original' in line:
g.write(line.replace('Original', 'Replacement'))
Then if you want to, you may os.remove() the non-edited file with:
More Info: Tutorials Point: Python Files I/O
The second error is how the replace() method works.
It returns the entire input string, with only the specified substring replaced. See example here.
To write to a specific place in the file, you should seek() to the right position first.
I think this issue has been asked before in several places, I would do a quick search of StackOverflow.
Maybe this would help?
Replacing stuff in a file only works well if original and replacement have the same size (in bytes) then you can do
with open('example.php','rb+') as f:
pos=f.tell()
line=f.readline()
if b'Original' in line:
f.seek(pos)
f.write(line.replace(b'Original',b'Replacement'))
(In this case b'Original' and b'Replacement' do not have the same size so your file will look funny after this)
Edit:
If original and replacement are not the same size, there are different possibilities like adding bytes to fill the hole or moving everything after the line.
I have a file that has a list of files but it adds \n at the end how can I have python just write the info I need on a new line without getting \n in the way so that way my info will be called X.acc not x.acc\n? Here is my code that writes the file
def add(x):
nl = "\n"
acc = ".acc"
xy = x + acc
exyz = xy
xyz = exyz
xxx = str(xyz)
tf = open('accounts.dat',"a+")
tf.writelines(nl)
tf.writelines(xxx)
tf.close
Here is the code that calls upon the file:
import sys
tf = open('accounts.dat','r')
names = tf.readlines()
u = choicebox(msg="pick something",title = "Choose an account",choices=(names))
counter_file = open(u, 'r+')
content_lines = []
for line in counter_file:
if line == "credits =":
creds = line
else:
False
for line in counter_file:
if 'credits =' in line:
line_components = line.split('=')
int_value = int(line_components[1]) + 1
line_components[1] = str(int_value)
updated_line= "=".join(line_components)
content_lines.append(updated_line)
else:
msgbox(msg=(creds))
content_lines.append(line)
counter_file.seek(0)
counter_file.truncate()
counter_file.writelines(content_lines)
counter_file.close()
thank you for your help and sorry if this is a trival question still new to python :)
Your question doesn't actually make sense, because of what a "line" actually is and what that '\n' character means.
Files don't have an intrinsic concept of lines. A file is just a sequence of bytes. '\n' is the line separator (as Python represents it with universal newlines). If you want your data to show up on different "lines", you must put a line separator between them. That's all that the '\n' character is. If you open up the file in a text editor after you write it, most editors won't explicitly show the newline character by default, because it's already represented by the separation of the lines.
To break down what your code is doing, let's look at the add method, and fix some things along the way.
The first thing add does is name a variable called nl and assign it the newline character. From this, I can surmise that nl stands for "newline", but it would be much better if that was actually the variable name.
Next, we name a variable called acc and assign it the '.acc' suffix, presumably to be used as a file extension or something.
Next, we make a variable called xy and assign it to x + acc. xy is now a string, though I have no idea of what it contains from the variable name. With some knowledge of what x is supposed to be or what these lines represent, perhaps I could rename xy to something more meaningful.
The next three lines create three new variables called exyz, xyz, and xxx, and point them all to the same string that xy references. There is no reason for any of these lines whatsoever, since their values aren't really used in a meaningful way.
Now, we open a file. Fine. Maybe tf stands for "the file"? "text file"? Again, renaming would make the code much more friendly.
Now, we call tf.writelines(nl). This writes the newline character ('\n') to the file. Since the writelines method is intended for writing a whole list of strings, not just a single character, it'll be cleaner if we change this call to tf.write(nl). I'd also change this to write the newline at the end, rather than the beginning, so the first time you write to the file it doesn't insert an empty line at the front.
Next, we call writelines again, with our data variable (xxx, but hopefully this has been renamed!). What this actually does is break the iterable xxx (a string) into its component characters, and then write each of those to the file. Better replace this with tf.write(xxx) as well.
Finally, we have tf.close, which is a reference to the close function of the file object. It's a no-op, because what you presumably meant was to close the file, by calling the method: tf.close(). We could also wrap the file up as a context manager, to make its use a little cleaner. Also, most of the variables aren't necessary: we can use string formatting to do most of the work in one step. All in all, your method could look like this at the end of the day:
def add(x):
with open('accounts.dat',"a+") as output_file:
output_file.write('{0}.acc\n'.format(x))
So you can see, the reason the '\n' appears at the end of every line is because you are writing it between each line. Furthermore, this is exactly what you have to do if you want the lines to appear as "lines" in a text editor. Without the newline character, everything would appear all smashed together (take out the '\n' in my add method above and see for yourself!).
The problem you described in the comment is happening because names is a direct reading of the file. Looking at the readlines documentation, it returns a list of the lines in the file, breaking at each newline. So to clean those names up, you want line 4 of the code you posted to call str.strip on the individual lines. You can do that like this:
names = tf.readlines()
for i in range(len(names)):
names[i] = names[i].strip() # remove all the outside whitespace, including \n
However, it's much cleaner, quicker, and generally nicer to take advantage of Python's list comprehensions, and the fact that file objects are already iterable line-by-line. So the expression below is equivalent to the previous one, but it looks far nicer:
names = [line.strip() for line in tf]
Just change add:
def add(x):
nl = "\n"
acc = ".acc"
xy = x + acc
exyz = xy
xyz = exyz
xxx = str(xyz)
tf = open('accounts.dat',"a+")
tf.writelines(xxx)
tf.writelines(nl) # Write the newline AFTER instead of before the output
tf.close() # close is a function so needs to be called by having () at the end.
See the comments for what has changed.
why dont you just write a function with "\n" at the end of the line.
So no need recall "\n" every time
I did this way-
import os
log_path = r"c:\python27\Logs\log.txt"
if not os.path.exists(r"c:\python27\Logs"):
os.mkdir(r"c:\python27\Logs")
def write_me_log(text):
global log_path
with open(log_path,"a+") as log:
log.write(text+"\n")
write_me_log("Hello this is the first log text with new line")
file = open("accountfile.txt","a")
file.write(username)
file.write(" ")
file.write(password)
file.write(" ")
file.write(age)
#need it to go down a line here so it writes"hello world" on the next line
file.write("hello world")
file.close()``
The Task
I am writing a program in python that running a SAP2000 program by importing a new .s2k file each time into the Sap2000 program, and then a new file is generated from the results of the previous run by the means of exporting the data.
The file is about 1,500 lines containing arbitrary words and numbers. (For a better understanding, see this: http://pastebin.com/8ptYacJz, which is the file I am dealing with.)
I'm required to replace one number in the file.
That number is somewhere in the middle of line 800.
The Question
Does anyone know an efficient way to move down to the middle of line 800 in a file, in order to replace one number?
What I've Tried
Regular expressions did not work, because there can be more then one instance of the same number.
So I came up with the solution of templating the file and writing a new file each time with the number to be changed as a template parameter.
This solution does work but the person insists that I can move the file pointer down to line 800, then over to the middle of the line to replace the number.
Here is the only code I have for the problem that takes the file buffer to a line then back up to the beginning when I try to seek over.
import sys
import os
#open file
f = open("output.$2k")
#this will go to line 883 in text file
count = 0;
while count < 883:
line = f.readline()
count = count+1
#this would seek over to middle of file DOESN'T WORK
f.seek(0,0)
line = f.readline()
print(line)
f.close()
Yes and no. Consider:
f=open('output.$2k','r+')
f.seek(300)
f.write('\n')
f.close()
This script just changes the 300th character in your ascii file to a newline. Now the tricky part is that there is no way to know the length of a line in an ascii file short of reading until you get to a newline. So, locating the particular character in the file at the middle of the 800th line is non-trivial. However, if you can make guarantees (due to the way the file was written) about the line length, you can calculate the position without any problem. Also note that replacing 1 with 100 won't work here. You need to replace 1 character with 1 character.
And just for all the other *NIX users in the world ... please don't put $ in your filename. That's just a nightmare...
OK, i'm not a professional programmer, but my (stupid) approach would be: If it's always line 800, read the file line by line while tracking the line numbers. Write then directly to a new file. Read line 800, change it, write it. Then write the rest. Dumb and not elegant but it should work-unless i miss something which i probably do. And there goes my meager reputation :D
No. Read in the line, manipulate it, then write it out to the new file you've previously opened for writing (and have been writing the other lines to, unmodified).
A first thing:
#this would seek over to middle of file DOESN'T WORK
f.seek(0,0)
this is not true. This seeks to the beginning of the file.
To your actual question:
Does anyone know an efficient way to move down to the middle of line 800 in a file, in order to replace one number?
In general, no. You'd need to rewrite the file. For example like this:
# open the file in read-and-update mode
with open("file", 'r+') as f:
# read all lines
lines = f.readlines()
# update 800'th line
my_line = lines[799].split()
my_line[5] = "%s" % my_number # TODO: put in index of number and updated number
lines[799] = " ".join(my_line)
# truncate and rewrite file
f.truncate(0)
f.writelines(lines)
You can do it, if the starting position of the number in the file is predictable (e.g. number_starting_pos = 1234 from the beginning of the file) and the size of the string representation is also predictable (e.g. 20).
Then you could rewrite the number and make sure you fill up the padding with whitespace again to overwrite any content of the previous entry.
Similar to this:
with open("file", 'r+') as f:
# seek to the number starting position
f.seek(number_starting_pos, 0)
# update number field, assuming width (20), arbitrary space-padding allowed
my_number_string = "%19s " % my_number
# make sure the string is indeed exactly of the specific size (it may be longer)
assert len(my_number_string) == 20, "file writing would fail! aborting!"
f.write(my_number_string)
For this to work, you'd need to have a look at the docs of your SAP-thingy, and see if whitespace indeed not matters.
However, both approaches are based on a lot of assumptions. Depending on your use case it may easily break your code, e.g. if a line is inserted or even a characters is inserted before the number field.