My question is about Python build-in open function with a+ mode. First, from this thread, I know for a+ mode, subsequent writes to the file will always end up at the current end of file. I write the following simple program to confirm this fact:
file = 'test0.txt' # any non-empty text file
with open(file, 'a+', encoding='utf-8') as f:
f.seek(0)
f.write('added')
# print(f.tell())
cc = f.readlines() # strange!
print(cc)
Yes, 'added' is appended on the end even if forcing the stream to position at the beginning of the file by seek() method.
I think cc should be [], because the stream is positioned at the end of the file. However, it is wrong! The result is:
cc shows all the text before appending added. Moreover, either switching to comment f.seek(0) or switching to uncomment print(f.tell()) makes things normal: c turns to be [] as expected. For me, this means tell() indeed changes something—not just reporting the position—in this case. I would be very grateful if anyone could tell me the logic behind this.
Such examples are very implementation-dependent and if you are looking to get an exact answer, you have to dive deep into the source code.
Indeed you are trying to get an undesired outcome using the methods in an undesired order. I know I am not providing a solution here as to why this happens, but here are some of the questions asked before and they still don't really have the detailed explanation you seek.
All in all, be my guest, go ahead read the source code, understand what happens under to hood and explain us what the issue is. If you think this would take a lot of time (I know from experience), use the methods in the order as they should be and avoid ambiguities and undesired behaviors. Like #Blckknght says, don't play with fire unless you are willing to put it out.
At the end of the day, this might actually be a security problem or not if inspected closely, though I doubt since you are not overflowing anything or overwriting to unexpected places, as there are various checks and controls for such operations.
Weird file seeking behaviour
Python 3 tell() gets out of sync with file pointer in append+read mode
I have the following code (it changes the string/filepath, replacing the numbers at the end of the filename + the file extension, and replaces that with "#.exr"). I hope I made the problem replicatable below.
I was doing it this way because the filename can be typed in all kinds of ways, for example:
r_frame.003.exr (but also)
r_12_frame.03.exr
etc.
import pyseq
import re
#create render sequence list
selected_file = 'H:/test/r_frame1.exr'
without_extention = selected_file.replace(".exr", "")
my_regex_pattern = r"\d+\b"
sequence_name_with_replaced_number = re.sub(my_regex_pattern, "#.exr" ,without_extention)
mijn_sequences = fileseq.findSequencesOnDisk(sequence_name_with_replaced_number)
If I print the "sequence_name_with_replaced_number" value, this results in the console in:
'H:/test/r_frame#.exr'
When I use that variable inside that function like this:
mijn_sequences = fileseq.findSequencesOnDisk(sequence_name_with_replaced_number)
Then it does not work.
But when I manually replace that last line into:
mijn_sequences = fileseq.findSequencesOnDisk('H:/test/r_frame#.exr')
Then it works fine. (it's the seems like same value/string)
But this is not an viable option, the whole point of the code if to have the computer do this for thousands of frames.
Anybody any idea what might be the cause of this?
I already tried re-converting the variable into a string with str()
I tried other ways like using an f-string, I wasn't sure how to convert it into a raw string since the variable already exists.
After this I will do simple for loop going trough al the files in that sequence. The reason I'm doing this workflow is to delete the numbers before the .exr file extensions and replace them with # signs. (but ognoring all the bumbers that are not at the end of the filename, hence that regex above. Again, the "sequence_name_with_replaced_number" variable seems ok in the console. It spits out: 'H:/test/r_frame#.exr' (that's what I need it to be)
It's fixed!
the problem was correct, every time I did a cut and past from the variable value in the console and treated it as manual input it worked.
Then I did a len() of both values, and there was a difference by 2!
What happend?
The console added the ''
But in the generated variable it had those baked in as extra letters.
i fixed it by adding
cleaned_sequence = sequence_name_with_replaced_number[1:-1]
so 'H:/test/r_frame1.exr' (as the console showed me)
was not the same as 'H:/test/r_frame1.exr' (what I inserted manually, because I added these marks, in the console there are showed automatically)
I have a long-running script that loops over rows in a database. Every so often I'd like it to print how many rows it's processed, but without creating a new line each time. This is essentially what I have:
import sys
mystr = "{} rows complete\r"
for i in range(0, 100):
if i % 10 == 0:
sys.stdout.write(mystr.format(i))
sys.stdout.flush()
When I run this (on Python 2.7.5 64-bit, Windows) I get:
0 rows complete
10 rows complete
20 rows complete
...
100 rows complete
Can anyone think of why the carriage return isn't working? I've seen some similar questions here about Python and \r, but all the answers say to include sys.stdout.flush(), which I have.
Using \r is probably the correct way to go here, but the behaviour depends very much on how the text is being output. Not all terminals will respect a bare carriage return:
cmd.exe and powershell.exe should both output the text the way you expect (Powershell ISE doesn't but that's a red herring here).
Python's own idle will ignore the carriage return, all output comes on one long line.
As you noted, your own editor's command window also ignores \r.
Also, if you try from an interactive window you also get some numbers output at the end of each line: that's because the interactive windows helpfully outputs the result of the call to sys.stdout.flush().
I'm trying to write a simple tool that reads files from disc, does some image processing, and returns the result of the algorithm. Since the program can sometimes take awhile, I like to have a progress bar so I know where it is in the program. And since I don't like to clutter up my command line and I'm on a Unix platform, I wanted to use the '\r' character to print the progress bar on only one line.
But when I have this code here, it prints nothing.
# Files is a list with the filenames
for i, f in enumerate(files):
print '\r%d / %d' % (i, len(files)),
# Code that takes a long time
I have also tried:
print '\r', i, '/', len(files),
Now just to make sure this worked in python, I tried this:
heartbeat = 1
while True:
print '\rHello, world', heartbeat,
heartbeat += 1
This code works perfectly. What's going on? My understanding of carriage returns on Linux was that it would just move the line feed character to the beginning and then I could overwrite old text that was written previously, as long as I don't print a newline anywhere. This doesn't seem to be happening though.
Also, is there a better way to display a progress bar in a command line than what I'm current trying to do?
Try adding sys.stdout.flush() after the print statement. It's possible that print isn't flushing the output until it writes a newline, which doesn't happen here.
Handling of carriage returns in Linux differs greatly between terminal-emulators.
Normally, one would use terminal escape codes that would tell the terminal emulator to move the virtual "carriage" around the screen (think full-screen programs running over BBS lines). The ones I'm aware of are the VT100 escape codes:
\e[A: up
\e[B: down
\e[C: right
\e[D: left
\e[1~: home
\e[4~: end
Where \e is the escape character, \x1b.
Try replacing all \r's with \e[1~
Also see this post
If your terminal is line-buffered, you may need a sys.stdout.flush() to see your printing if you don't issue a linefeed.
I am a totally blind programmer who would like to learn Python. Unfortunately the fact that code blocks are represented with different levels of indentation is a major stumbling block. I was wondering if there were any tools available that would allow me to write code using braces or some other code block delimiter and then convert that format into a properly indented representation that the Python interpreter could use?
There's a solution to your problem that is distributed with python itself. pindent.py, it's located in the Tools\Scripts directory in a windows install (my path to it is C:\Python25\Tools\Scripts), it looks like you'd have to grab it from svn.python.org if you are running on Linux or OSX.
It adds comments when blocks are closed, or can properly indent code if comments are put in. Here's an example of the code outputted by pindent with the command:
pindent.py -c myfile.py
def foobar(a, b):
if a == b:
a = a+1
elif a < b:
b = b-1
if b > a: a = a-1
# end if
else:
print 'oops!'
# end if
# end def foobar
Where the original myfile.py was:
def foobar(a, b):
if a == b:
a = a+1
elif a < b:
b = b-1
if b > a: a = a-1
else:
print 'oops!'
You can also use pindent.py -r to insert the correct indentation based on comments (read the header of pindent.py for details), this should allow you to code in python without worrying about indentation.
For example, running pindent.py -r myfile.py will convert the following code in myfile.py into the same properly indented (and also commented) code as produced by the pindent.py -c example above:
def foobar(a, b):
if a == b:
a = a+1
elif a < b:
b = b-1
if b > a: a = a-1
# end if
else:
print 'oops!'
# end if
# end def foobar
I'd be interested to learn what solution you end up using, if you require any further assistance, please comment on this post and I'll try to help.
I personally doubt that there currently is at the moment, as a lot of the Python afficionados love the fact that Python is this way, whitespace delimited.
I've never actually thought about that as an accessibility issue however. Maybe it's something to put forward as a bug report to Python?
I'd assume that you use a screen reader here however for the output? So the tabs would seem "invisible" to you? With a Braille output, it might be easier to read, but I can understand exactly how confusing this could be.
In fact, this is very interesting to me. I wish that I knew enough to be able to write an app that will do this for you.
I think it's definately something that I'll put in a bug report for, unless you've already done so yourself, or want to.
Edit: Also, as noted by John Millikin There is also PyBraces Which might be a viable solution to you, and may be possible to be hacked together dependant on your coding skills to be exactly what you need (and I hope that if that's the case, you release it out for others like yourself to use)
Edit 2: I've just reported this to the python bug tracker
Although I am not blind, I have heard good things about Emacspeak. They've had a Python mode since their 8.0 release in 1998 (they seem to be up to release 28.0!). Definitely worth checking out.
You should be able to configure your editor to speak the tabs and spaces -- I know it's possible to display whitespace in most editors, so there must be an accessibility option somewhere to speak them.
Failing that, there is pybraces, which was written as a practical joke but might actually be useful to you with a bit of work.
If you're on Windows, I strongly recommend you take a look at EdSharp from:
http://empowermentzone.com/EdSharp.htm
It supports all of the leading Windows screenreaders, it can be configured to speak the indentation levels of code, or it has a built in utility called PyBrace that can convert to and from braces syntax if you want to do that instead, and it supports all kinds of other features programmers have come to expect in our text editors. I've been using it for years, for everything from PHP to JavaScript to HTML to Python, and I love it.
All of these "no you can't" types of answers are really annoying. Of course you can.
It's a hack, but you can do it.
http://timhatch.com/projects/pybraces/
uses a custom encoding to convert braces to indented blocks before handing it off to the interpreter.
As an aside, and as someone new to python - I don't accept the reasoning behind not even allowing braces/generic block delimiters ... apart from that being the preference of the python devs. Braces at least won't get eaten accidentally if you're doing some automatic processing of your code or working in an editor that doesn't understand that white space is important. If you're generating code automatically, it's handy to not have to keep track of indent levels. If you want to use python to do a perl-esque one-liner, you're automatically crippled. If nothing else, just as a safeguard. What if your 1000 line python program gets all of its tabs eaten? You're going to go line-by-line and figure out where the indenting should be?
Asking about it will invariably get a tongue-in-cheek response like "just do 'from __ future __ import braces'", "configure your IDE correctly", "it's better anyway so get used to it" ...
I see their point, but hey, if i wanted to, i could put a semicolon after every single line. So I don't understand why everyone is so adamant about the braces thing. If you need your language to force you to indent properly, you're not doing it right in the first place.
Just my 2c - I'm going to use braces anyway.
I appreciate your problem, but think you are specifying the implementation instead of the problem you need solved. Instead of converting to braces, how about working on a way for your screen reader to tell you the indentation level?
For example, some people have worked on vim syntax coloring to represent python indentation levels. Perhaps a modified syntax coloring could produce something your screen reader would read?
Searching an accessible Python IDE, found this and decided to answer.
Under Windows with JAWS:
Go to Settings Center by pressing JawsKey+6 (on the number row above the letters) in your favorite text editor. If JAWS prompts to create a new configuration file, agree.
In the search field, type "indent"
There will be only one result: "Say indent characters". Turn this on.
Enjoy!
The only thing that is frustrating for us is that we can't enjoy code examples on websites (since indent speaking in browsers is not too comfortable — it generates superfluous speech).
Happy coding from another Python beginner).
I use eclipse with the pydev extensions since it's an IDE I have a lot of experience with. I also appreciate the smart indentation it offers for coding if statements, loops, etc. I have configured the pindent.py script as an external tool that I can run on the currently focused python module which makes my life easier so I can see what is closed where with out having to constantly check indentation.
There are various answers explaining how to do this. But I would recommend not taking this route. While you could use a script to do the conversion, it would make it hard to work on a team project.
My recommendation would be to configure your screen reader to announce the tabs. This isn't as annoying as it sounds, since it would only say "indent 5" rather than "tab tab tab tab tab". Furthermore, the indentation would only be read whenever it changed, so you could go through an entire block of code without hearing the indentation level. In this way hearing the indentation is no more verbose than hearing the braces.
As I don't know which operating system or screen reader you use I unfortunately can't give the exact steps for achieving this.
Edsger Dijkstra used if ~ fi and do ~ od in his "Guarded Command Language", these appear to originate from the Algol68. There were also some example python guarded blocks used in RosettaCode.org.
fi = od = yrt = end = lambda object: None;
class MyClass(object):
def myfunction(self, arg1, arg2):
for i in range(arg1) :# do
if i > 5 :# then
print i
fi
od # or end(i) #
end(myfunction)
end(MyClass)
Whitespace mangled python code can be unambiguously unmangled and reindented if one uses
guarded blocks if/fi, do/od & try/yrt together with semicolons ";" to separate statements. Excellent for unambiguous magazine listings or cut/pasting from web pages.
It should be easy enough to write a short python program to insert/remove the guard blocks and semicolons.