This is my first post on StackOverflow. I'm also a beginner in Python. So I was just tinkering with the open() function, I was making a simple program to replace text in other .txt files. Here is my code:
f = open("file.txt", "r+")
f.truncate(0)
f.write("This text has been replaced.")
f.close()
print("Text replaced")
So, after running this program, the text in "file.txt" is getting changed. However, when I do ctrl + z, it's showing Undo Reload from Disk?, and when you click OK, the text gets back to normal.
How to prevent this? I am using Python 3.9, Pycharm code editor.
Thank you
I think your question might be more about the PyCharm UI editor.
If I understand correctly:
you have "file.txt" open in the PyCharm editor
you run the code above
At this point, the filesystem has the "file.txt" with the updated contents. The PyCharm editor (buffer) still has the old contents.
When you ctrl+z, PyCharm notices the filesystem has been updated and prompts to see how you want to proceed. When you click "OK", PyCharm writes its buffer to the file, which is the original contents.
Related
This question already has answers here:
Convert UTF-8 with BOM to UTF-8 with no BOM in Python
(7 answers)
Closed last year.
I'm opening a plain text file, parsing it, and adding different lines to existing, empty string variables. I add these variables into a new variable that is a multi-line fstring. Trying to write the data to a new text file is not behaving as expected.
Reading the original file works fine. Text is properly parsed, variables populated.
The multi-line fstring variable seems fine. Prints normally. Even tried formatting it different ways which I show below.
When writing to a new file, that's where the strangeness starts. I've tried 2 ways:
Straight coding the open function with w or w+
Adding the above to a function and using that inside main()
The file is saved to disk with the correct name. Trying to double-click open in Finder produces nothing. Right-click to open produces nothing. Trying to move to trash with command+delete gives an error:
It sounds like the file goes to trash, but as the file disappears from the folder a new one is created with the same name in its place.
If I try to open in TextMate via File > Open, it opens as a blank file with no errors.
Since I can't get rid of the file, I have to delete the directory and create the directory again with the same name, or force delete in Terminal using rm. Restarting the system does not help. Relaunching Finder does nothing. Saving text files from other apps works fine. Directory is chmod 755.
If I copy an existing text file into the output directory, rename it to what the file is expected to be named, and let python overwrite the contents, it doesn't work either. The file modification date changes (and I see the file "blink" in Finder) but the contents remain the same. However, the file is not corrupted and opens normally.
If I do the same but delete the text inside of the copied file first, then run the script, python writes no data to the file, I can't open it by double-clicking on it, and I get error -43 again with the odd non-trashing behavior.
The strangest thing is this: if I add another with open() at the end of the script, and open the file that was just created and supposedly written to, and print its contents, the contents print. It's like when the script ends the file contents are being removed or its being corrupted somehow. Tried to close the file inside the script even though it's not needed, but same behavior persists.
Code:
Here's the code for writing:
FORMAT='utf-8'
OUTPUT_DIR = '/Path/To/SaveFolder'
# as a function
def write_to_file(content, fpath, name):
the_file = os.path.join(fpath, name)
with open(the_file, 'w+', encoding=FORMAT) as t:
t.write(content)
def main():
print(f" Writing File...\n")
filename = f"{pcode}_{author}_{title}_text.txt"
write_to_file(multiline_var, OUTPUT_DIR, filename)
# or hard coded in main()
def main():
print(f" Writing File...\n")
filename = f"{pcode}_{author}_{title}_text.txt"
the_file = os.path.join(OUTPUT_DIR, filename)
with open(the_file, 'w+', encoding=FORMAT) as t:
t.write(multiline_var)
I have tried using w w+ wt and wt+ and with and without encoding='utf-8'
Here is an example of multi-line fstring variable:
# using triple quotes
multiline_var = f"""
[PROJ-{pcode}] {full_title} by {author}
{description}
{URL}
{DIVIDER_1}
{TEXT_BLURB}
Some text here and then {SOME_MORE_TEXT}"
{DIVIDER_1}
{SOME_LINK}
"""
# or inside parens
multiline_var = (
f"[PROJ-{pcode}] {full_title} by {author}\n"
f"{description}\n\n"
f"{URL}\n"
f"{DIVIDER_1}\n"
f"{TEXT_BLURB}\n\n"
f"Some text here and then {SOME_MORE_TEXT}\n"
f"{DIVIDER_1}\n\n"
f"{SOME_LINK}"
)
Using exiftool on the text file shows the following, so it looks the data is there but must be corrupted:
File Size : 1797 bytes
File Modification Date/Time : 2021:12:31 15:55:39-05:00
File Access Date/Time : 2021:12:31 15:58:13-05:00
File Inode Change Date/Time : 2021:12:31 15:55:39-05:00
File Permissions : -rw-r--r--
File Type : TXT
File Type Extension : txt
MIME Type : text/plain
MIME Encoding : utf-8
Byte Order Mark : No
Newlines : Unix LF
Line Count : 55
Word Count : 181
Not sure what I'm doing wrong. VScode shows no syntax errors in the script. There are no errors in Terminal when running the script. Have I made some simple mistake in the above code? Maybe the fstring variable is causing a problem?
Thanks to #bnaecker for leading me to the solution to this problem.
It appeared that when creating/writing to a text file with a long name, Python can corrupt it. Not sure why, as I save long names for images with Python image libraries all the time. Using a short name like "MyFile.txt" it worked just fine, but that was a red herring.
I have updated this post with my journey to the final solution for using the long names that are needed for my project, though I'm not sure why the problem exists.
First Attempts:
So far creating using a short name and then renaming to a long one.... attempts have failed. I did notice that python is locking the file it creates and never unlocks it. Not sure if this is the problem. Setting chflags with os.system('chflags nouchg') command does not work, not even with sudo, and not even in the Terminal doing it manually.
Using os.rename() in Python corrupts the file
Using os.system('mv oldFile.txt newFile.txt') corrupts the file
Manually using mv command in Terminal corrupts the file
Manually changing the filename in the Finder does not (wtf?)
I kept looking for workarounds but nothing did the job.
Round 2:
Progress!
After much tinkering, I discovered a hidden character inside the file. I ran cat /path/longfilename.txt in Terminal, selected and copied the output and pasted into VScode. Here is what I saw:
Somehow a hidden character is getting into the project code number.
Pasting it into a Unicode search engine it came up as a ZERO WIDTH NO-BREAK SPACE also known in Unicode as EF BB BF. However, when pasting this symbol into TextMate it shows up as <U+FEFF> which is?...
The Byte Order Mark!
Opening a normal utf-8 text file in a hex editor also shows the files starting with EFBBBF for the BOM.
Now, the text file being read and parsed at first has no blank lines to start the file, so I added a line break, and also tried adding some spaces. This time when writing the file I could open it, however, after sending it to the trash, the same behavior occurred and the file was broken again. It seems that because other corrupted versions were in the trash, it added the symbol back to the file name for some reason.
So what appears to be happening, for whatever reason, when Python opens the text file I'm parsing that has no line break at the top, it seems to be grabbing the BOM from the file and adding that to the first variable which is grabbing the first line of the text file. Since that text is a number code that starts the file name, the BOM symbol is being added to the file name as well as the code inside the text file.
Just... wow
The Current Solution:
I have to leave a blank line at the start of the text file that I'm opening and parsing and a simple line break won't do it. I have no idea why this is. I added some spaces for good measure because randomly the BOM would be added to the variable and filename again. So far (knock on wood) as long as the first line of that initial file has some spaces and then a line break, and previous corrupted files have been deleted from the trash, a long file name can be used for all the files I'm creating and writing to without any problems.
This corruption even persists if I remove the encoding flag from both of the open functions I'm using (one to read and parse, the other to create and write).
If anyone knows why this is happening, please share. I've never seen it mentioned before. I'm not sure if it's a python 3.8 bug, a mac OS bug, the way TextMate wrote the original file, or a combination of these.
Correct Solution:
Thanks to #tripleee for the proper way to handle this, as I don't remember seeing this before, though I haven't been using python for very long.
In order to ignore the BOM, reading in the text file to be parsed with an encoding='utf-8-sig' does the job. Seems to be why it exists. :)
Problem solved.
I started with this example:
cars = ['audi', 'bmw', 'subaru', 'toyota']
for car in cars:
if car == 'bmw':
print(car.upper())
else:
print(car.title())
I deleted this and moved on to a new example, within same py file:
requested_topping = 'mushrooms'
if requested_topping != 'anchovies':
print("Hold the anchovies!")
After I click run on this second code, VS Code still prints the output for the cars example. What could be the matter?
The white dot indicates the file has not been saved.
You can add this in the settings.json file:
"files.autoSave": "afterDelay",
"files.autoSaveDelay": 1000,
to enable the autosave function in the VSCode.
The issue is unlikely to be due to the code you've written. It is likely related to how you're saving your .py file, and how it is being ran.
Are you running the file using the 'play' button / Ctrl+F5 in VSCode, or are you double clicking on the .py file from File Explorer / Finder?
If it's the latter, then perhaps you're not saving the file in VSCode first, try saving the file with Ctrl+S first.
Screenshots would likely go a long way to helping you answer this question.
Command n (New file..), convert language mode to python from plain text and paste the new block and run ?
I have copied a text from my software using pywinauto. Unfortunately, I don't know how to paste that to a text file. The following is the code that I wrote:
The last line of the code is not working as it should not. However, that is what I should do. Can anyone help me to solve this problem?
pywinauto.mouse.double_click(button='left', coords=(820,168))
pywinauto.keyboard.send_keys('^c')
f= open("trial.txt","w+")
f.write(pywinauto.keyboard.send_keys('^v'))```
I see that you're trying to paste the contents of clipboard, but there is no visual area to paste.
f.write() will accept text through a variable or, by passing some text. Invoking Ctrl + V is a GUI operation, which can't replace the text in f.write()
You can use pyperclip module to access the clipboard contents.
import pyperclip
"""yourcode"""
f.write(pyperclip.paste())
f.close()
You can also programatically copy something to system clipboard using pyperclip.
pyperclip.copy("This is a text copied to clipboard from Python script!!")
You can now check the contents by invoking Ctrl + V in some GUI application like notepad.
You can try send it hotkey
pyautogui.hotkey('ctrl','v')
I would like to get the text from a program with Python, for example from notepad. Can I "request" this text, just like from a website? I thought about something like this:
A document in Notepad:
Hello World!
This is a text!
GetText.py:
get_text("notepad.exe")
>>> Hello World!\nThis is a text!
Is this possible?
No, you can't, not directly.
There are various accessibility, etc. APIs you could use to try and "read" the user interface of another program, but that's certainly a lot more involved than just a simple get_text() style call.
(And for Windows Notepad, you can enumerate the Notepad main window's child windows, find the edit/rich-text control it's using and send a WM_GETTEXT message (if my memory serves) and hope it sends you some of the current text back...)
You can open the file in read-mode and simply print each line of the file using a for loop:
a_file = open('notepad.exe', 'r')
for line in a_file:
print(line)
a_file.close() #Make sure you close whatever file you open
If you are using Jupyter Notebook, make sure notepad.exe is in the same directory as you have your notebook opened in.
Side note: If you have experience with the command line (e.g. Linux), you can also open it in a text editor like vim. There, you can more readily see and edit it.
Goal
I am trying to create and edit a temporary file in vim (exactly the same behavior as a commit script in git/hg/svn).
Current code
I found a method to do so in this answer:
call up an EDITOR (vim) from a python script
import sys, tempfile, os
from subprocess import call
EDITOR = os.environ.get('EDITOR','vim')
initial_message = "write message here:"
with tempfile.NamedTemporaryFile(suffix=".tmp") as tmp:
tmp.write(initial_message)
tmp.flush()
call([EDITOR, tmp.name])
tmp.seek(0)
print tmp.read()
The Issue
When I run the above code, the tempfile does not read the changes made in vim. Here is the output after I have added several other lines in vim:
fgimenez#dn0a22805f> ./note.py
Please edit the file:
fgimenez#dn0a22805f>
Now for the interesting (weird) part. If I change my editor to nano or emacs, the script works just fine! So far, this only seems to break when I use vim or textedit.
As another experiment, I tried calling a couple editors in a row to see what happens. The modified code is:
with tempfile.NamedTemporaryFile(suffix=".tmp") as tmp:
tmp.write(initial_message)
tmp.flush()
# CALLING TWO EDITORS HERE, VIM THEN NANO
call(['vim', tmp.name])
raw_input("pausing between editors, just press enter")
call(['nano', tmp.name])
tmp.seek(0)
print tmp.read()
I.e. I edit with vim then nano. What happens is that nano DOES register the changes made by vim, but python doesn't register anything (same result as before):
fgimenez#dn0a22805f> ./note.py
Please edit the file:
fgimenez#dn0a22805f>
BUT, if I edit with nano first, then vim, python still registers the nano edits but not the vim ones!
with tempfile.NamedTemporaryFile(suffix=".tmp") as tmp:
tmp.write(initial_message)
tmp.flush()
# CALLING TWO EDITORS HERE, NANO THEN VIM
call(['nano', tmp.name])
raw_input("pausing between editors, just press enter")
call(['vim', tmp.name])
tmp.seek(0)
print tmp.read()
Ouput from running the program and adding a\nb\nc in nano and d\ne\nf in vim:
fgimenez#dn0a22805f> ./note.py
Please edit the file:
a
b
c
fgimenez#dn0a22805f>
It seems as if using vim or textedit eliminates the ability to append to the file. I'm completely confused here, and I just want to edit my notes in vim...
Edit 1: Clarifications
I am on osx Mavericks
I call vim from the shell (not MacVim) and end the session with ZZ (also tried :w :q)
I'm no Python expert, but it looks like you're keeping the handle to the temp file open while Vim is editing the file, and then attempt to read in the edited contents from the handle. By default, Vim creates a copy of the original file, writes the new contents to another file, and then renames it to the original (see :help 'backupcopy' for the details; other editors like nano apparently don't do it this way). This means that the Python handle still points to the original file (even though it may have already been deleted from the file system, depending on the Vim settings), and you get the original content.
You either need to reconfigure Vim (see :help 'writebackup'), or (better) change the Python implementation to re-open the same temp file name after Vim has exited, in order to get a handle to the new written file contents.
I had the same problem on OS X after my code worked fine on Linux. As Ingo suggests, you can get the latest contents by re-opening the file. To do this, you probably want to create a temporary file with delete=False and then explicitly delete the file when you're done:
import sys, tempfile, os
from subprocess import call
EDITOR = os.environ.get('EDITOR','vim')
initial_message = "write message here:"
with tempfile.NamedTemporaryFile(suffix=".tmp", delete=False) as tmp:
tmp.write(initial_message)
tmp.flush()
call([EDITOR, tmp.name])
tmp.close()
with open(tmp.name) as f:
print f.read()
os.unlink(tmp.name)