please tell me why my code didn't work the way I thought it would in python 3.4:
r = input("file address")
file = open(r)
for line in file:
if line.endswith("d") is True:
print(line)
my file is a *.txt and contains the following lines:
This line ends with a
this line ends with b
this line ends with c
this line ends with d
I expected python to print out the last one for it has the "d" at the end. However, the program did nothing.
When I change the boolean to "false", the python prints out all the lines. So I guess python sees the boolean "line.endswith("d") is False in its nature. How is that?
Thanks
Your lines contains newlines, so the conditional never evaluates to true.
You should either s.strip() the newline before calling str.endswith() or check for s. endswith("d\n").
It does that because you're not stripping each line. You need to run line = line.strip() for each line which gets rid of the \n (backslash-n) which is what each line actually ends with. So your line: This line ends with a this line ends with b this line ends with c this line ends with d actually ends in \n (backslash-n) but you don't see it because python doesn't print it. (It prints a new line)!
Good Luck!
You just need to check if line.endswith("d"):, it will either evaluate to True or False you don't need to use if line.endswith("d") is True and use line.strip().endswith("d") to remove the \n.
The last char is a "\n" so line.endswith("d") will never be True
Related
I've been testing reading lines from a text file and finally got my program to execute correctly. But I'm not sure why the while loop exits execution.
filepath = 'LastQuizPlan.txt'
with open(filepath) as fp:
line = fp.readline()
count = 1
while line:
print(line.strip)
line=fp.readline()
count+=1
I found the code online from http://stackabuse.com/read-a-file-line-by-line-in-python/ and manipulated it slightly, it works fine but i need to create a write up for it as well and dont know how the while line: works exactly? it works exactly as i want it to work but i dont understand what its expression is.
I've been using the following statement for trying to figure this out
while expression:
statement(s)
but still just dont understand the expression? why does it automatically know that when count is out of the range of lines in "LastQuizPlan.txt", to stop the loop.
also here is the contents of "LastQuizPlan.txt" Unit 1
Electrical stuff
12/06/17
From the Python docs:
if f.readline() returns an empty string, the end of the file has been reached
Since empty strings are evaluated as false in Python, the while condition is no longer true when EOF is reached.
So if we swing over to the documentation for Python for readline we find the following:
f.readline() reads a single line from the file; a newline character
(\n) is left at the end of the string, and is only omitted on the last
line of the file if the file doesn’t end in a newline. This makes the
return value unambiguous; if f.readline() returns an empty string, the
end of the file has been reached, while a blank line is represented by
'\n', a string containing only a single newline.
And if we go over to another piece of the documentation about truthy-ness we find:
Any object can be tested for truth value, for use in an if or while
condition or as operand of the Boolean operations below. The following
values are considered false:
None
False
zero of any numeric type, for example, 0, 0L, 0.0, 0j.
any empty sequence, for example, '', (), [].
Therefore, with these two items combined we have the answer to the problem.
readline keeps returning non-empty strings while there are lines to be read. When it reaches the EOF, it returns an empty string. When the while loop checks the following condition: while '', it treats the empty string '' as a False value and ends execution of the loop.
You can see this doesn't execute anything with the quick Python statement:
while '':
print("Hello")
When executed, this will print nothing because the while body never executes.
I have a large textfile on my computer (location: /home/Seth/documents/bruteforce/passwords.txt) and I'm trying to find a specific string in the file. The list has one word per line and 215,000 lines/words. Does anyone know of simple Python script I can use to find a specific string?
Here's the code I have so far,
f = open("home/seth/documents/bruteforce/passwords.txt", "r")
for line in f.readlines():
line = str(line.lower())
print str(line)
if str(line) == "abe":
print "success!"
else:
print str(line)
I keep running the script, but it never finds the word in the file (and I know for sure the word is in the file).
Is there something wrong with my code? Is there a simpler method than the one I'm trying to use?
Your help is greatly appreciated.
Ps: I'm using Python 2.7 on a Debian Linux laptop.
I'd rather use the in keyword to look for a string in a line. Here I'm looking for the keyword 'KHANNA' in a csv file and for any such existence the code returns true.
In [121]: with open('data.csv') as f:
print any('KHANNA' in line for line in f)
.....:
True
It's just because you forgot to strip the new line char at the end of each line.
line = line.strip().lower()
would help.
Usually, when you read lines out of a file, they have a newline character at the end. Thus, they will technically not be equal to the same string without the newline character. You can get rid of this character by adding the line line=line.strip() before the test for equality to your target string. By default, the strip() method removes all white space (such as newlines) from the string it is called on.
What do you want to do? Just test whether the word is in the file? Here:
print 'abe' in open("passwords.txt").read().split()
Or:
print 'abe' in map(str.strip, open("passwords.txt"))
Or if it doesn't have to be Python:
egrep '^abe$' passwords.txt
EDIT: Oh, I forgot the lower. Probably because passwords are usually case sensitive. But if it really does make sense in your case:
print 'abe' in open("passwords.txt").read().lower().split()
or
print 'abe' in (line.strip().lower() for line in open("passwords.txt"))
or
print 'abe' in map(str.lower, map(str.strip, open("passwords.txt")))
Your script doesn't find the line because you didn't check for the newline characters:
Your file is made of many "lines". Each "line" ends with a character that you didn't account for - the newline character ('\n'1). This is the character that creates a new line - it is what gets written to the file when you hit enter. This is how the next line is created.
So, when you read the lines out of your file, the string contained in each line actually ends with a newline character. This is why your equality test fails. You should instead, test equality against the line, after it has been stripped of this newline character:
with open("home/seth/documents/bruteforce/passwords.txt") as infile:
for line in infile:
line = line.rstrip('\n')
if line == "abe":
print 'success!'
1 Note that on some machines, the newline character is in fact two characters - the carriage return (CR), and line-feed (LF). This terminology comes from back in the day when typewriters had to jump a line-width of space on the paper that was being written to, and that the carriage that contained the paper had to be returned to its starting position. When seen in a line in the file, this appears as '\r\n'
I have a quick and dirty build script that needs to update a couple of lines in a small xml config file. Since the file is so small, I'm using an admittedly inefficient process to update the file in place just to keep things simple:
def update_xml(property, value):
for line in fileinput.input(os.path.join(app_dir, 'my.xml'), inplace=True):
if property is 'version':
line = re.sub(r'(<version>).*?(</version>)', '\1%s\2' % value, line, flags=re.IGNORECASE)
elif property is 'brand':
line = re.sub(r'(<property name="brand" type="string">).*?(</property>)', '\1%s\2' % value, line, flags=re.IGNORECASE)
elif property is 'env':
line = re.sub(r'(<property name="env" type="string">).*?(</property>)', '\1%s\2' % value, line, flags=re.IGNORECASE)
print line
I have 2 problems:
The back references aren't capturing what I expect. Instead of getting <version>a.b.c</version>, for example, I get the version value surrounded by control characters. I've tried doubling up the backslash, removing the formatted print and a couple of other things, but can't get it quite right.
When I write the line back to the file (print line), I get several extra line breaks.
What am I borking up here?
Try to replace "\1%s\2" by "\g<1>%s\g<2>" , it might be the problem..
About the newlines , the print might be adding a second new line on top of the existing one .
you can try: print line, with a comma to suppress the new line char
Use a raw string to avoid \1 and \2 becoming control chars: r'\1%s\2'
Trying to figure out how to write an if cycle to check if a line is empty.
The file has many strings, and one of these is a blank line to separate from the other statements (not a ""; is a carriage return followed by another carriage return I think)
new statement
asdasdasd
asdasdasdasd
new statement
asdasdasdasd
asdasdasdasd
Since I am using the file input module, is there a way to check if a line is empty?
Using this code it seems to work, thanks everyone!
for line in x:
if line == '\n':
print "found an end of line"
x.close()
If you want to ignore lines with only whitespace:
if line.strip():
... do something
The empty string is a False value.
Or if you really want only empty lines:
if line in ['\n', '\r\n']:
... do something
I use the following code to test the empty line with or without white spaces.
if len(line.strip()) == 0 :
# do something with empty line
line.strip() == ''
Or, if you don't want to "eat up" lines consisting of spaces:
line in ('\n', '\r\n')
You should open text files using rU so newlines are properly transformed, see http://docs.python.org/library/functions.html#open. This way there's no need to check for \r\n.
I think is more robust to use regular expressions:
import re
for i, line in enumerate(content):
print line if not (re.match('\r?\n', line)) else pass
This would match in Windows/unix. In addition if you are not sure about lines containing only space char you could use '\s*\r?\n' as expression
In Python, I have just read a line form a text file and I'd like to know how to code to ignore comments with a hash # at the beginning of the line.
I think it should be something like this:
for
if line !contain #
then ...process line
else end for loop
But I'm new to Python and I don't know the syntax
you can use startswith()
eg
for line in open("file"):
li=line.strip()
if not li.startswith("#"):
print line.rstrip()
I recommend you don't ignore the whole line when you see a # character; just ignore the rest of the line. You can do that easily with a string method function called partition:
with open("filename") as f:
for line in f:
line = line.partition('#')[0]
line = line.rstrip()
# ... do something with line ...
partition returns a tuple: everything before the partition string, the partition string, and everything after the partition string. So, by indexing with [0] we take just the part before the partition string.
EDIT:
If you are using a version of Python that doesn't have partition(), here is code you could use:
with open("filename") as f:
for line in f:
line = line.split('#', 1)[0]
line = line.rstrip()
# ... do something with line ...
This splits the string on a '#' character, then keeps everything before the split. The 1 argument makes the .split() method stop after a one split; since we are just grabbing the 0th substring (by indexing with [0]) you would get the same answer without the 1 argument, but this might be a little bit faster. (Simplified from my original code thanks to a comment from #gnr. My original code was messier for no good reason; thanks, #gnr.)
You could also just write your own version of partition(). Here is one called part():
def part(s, s_part):
i0 = s.find(s_part)
i1 = i0 + len(s_part)
return (s[:i0], s[i0:i1], s[i1:])
#dalle noted that '#' can appear inside a string. It's not that easy to handle this case correctly, so I just ignored it, but I should have said something.
If your input file has simple enough rules for quoted strings, this isn't hard. It would be hard if you accepted any legal Python quoted string, because there are single-quoted, double-quoted, multiline quotes with a backslash escaping the end-of-line, triple quoted strings (using either single or double quotes), and even raw strings! The only possible way to correctly handle all that would be a complicated state machine.
But if we limit ourselves to just a simple quoted string, we can handle it with a simple state machine. We can even allow a backslash-quoted double quote inside the string.
c_backslash = '\\'
c_dquote = '"'
c_comment = '#'
def chop_comment(line):
# a little state machine with two state varaibles:
in_quote = False # whether we are in a quoted string right now
backslash_escape = False # true if we just saw a backslash
for i, ch in enumerate(line):
if not in_quote and ch == c_comment:
# not in a quote, saw a '#', it's a comment. Chop it and return!
return line[:i]
elif backslash_escape:
# we must have just seen a backslash; reset that flag and continue
backslash_escape = False
elif in_quote and ch == c_backslash:
# we are in a quote and we see a backslash; escape next char
backslash_escape = True
elif ch == c_dquote:
in_quote = not in_quote
return line
I didn't really want to get this complicated in a question tagged "beginner" but this state machine is reasonably simple, and I hope it will be interesting.
I'm coming at this late, but the problem of handling shell style (or python style) # comments is a very common one.
I've been using some code almost everytime I read a text file.
Problem is that it doesn't handle quoted or escaped comments properly. But it works for simple cases and is easy.
for line in whatever:
line = line.split('#',1)[0].strip()
if not line:
continue
# process line
A more robust solution is to use shlex:
import shlex
for line in instream:
lex = shlex.shlex(line)
lex.whitespace = '' # if you want to strip newlines, use '\n'
line = ''.join(list(lex))
if not line:
continue
# process decommented line
This shlex approach not only handles quotes and escapes properly, it adds a lot of cool functionality (like the ability to have files source other files if you want). I haven't tested it for speed on large files, but it is zippy enough of small stuff.
The common case when you're also splitting each input line into fields (on whitespace) is even simpler:
import shlex
for line in instream:
fields = shlex.split(line, comments=True)
if not fields:
continue
# process list of fields
This is the shortest possible form:
for line in open(filename):
if line.startswith('#'):
continue
# PROCESS LINE HERE
The startswith() method on a string returns True if the string you call it on starts with the string you passed in.
While this is okay in some circumstances like shell scripts, it has two problems. First, it doesn't specify how to open the file. The default mode for opening a file is 'r', which means 'read the file in binary mode'. Since you're expecting a text file it is better to open it with 'rt'. Although this distinction is irrelevant on UNIX-like operating systems, it's important on Windows (and on pre-OS X Macs).
The second problem is the open file handle. The open() function returns a file object, and it's considered good practice to close files when you're done with them. To do that, call the close() method on the object. Now, Python will probably do this for you, eventually; in Python objects are reference-counted, and when an object's reference count goes to zero it gets freed, and at some point after an object is freed Python will call its destructor (a special method called __del__). Note that I said probably: Python has a bad habit of not actually calling the destructor on objects whose reference count drops to zero shortly before the program finishes. I guess it's in a hurry!
For short-lived programs like shell scripts, and particularly for file objects, this doesn't matter. Your operating system will automatically clean up any file handles left open when the program finishes. But if you opened the file, read the contents, then started a long computation without explicitly closing the file handle first, Python is likely to leave the file handle open during your computation. And that's bad practice.
This version will work in any 2.x version of Python, and fixes both the problems I discussed above:
f = open(file, 'rt')
for line in f:
if line.startswith('#'):
continue
# PROCESS LINE HERE
f.close()
This is the best general form for older versions of Python.
As suggested by steveha, using the "with" statement is now considered best practice. If you're using 2.6 or above you should write it this way:
with open(filename, 'rt') as f:
for line in f:
if line.startswith('#'):
continue
# PROCESS LINE HERE
The "with" statement will clean up the file handle for you.
In your question you said "lines that start with #", so that's what I've shown you here. If you want to filter out lines that start with optional whitespace and then a '#', you should strip the whitespace before looking for the '#'. In that case, you should change this:
if line.startswith('#'):
to this:
if line.lstrip().startswith('#'):
In Python, strings are immutable, so this doesn't change the value of line. The lstrip() method returns a copy of the string with all its leading whitespace removed.
I've found recently that a generator function does a great job of this. I've used similar functions to skip comment lines, blank lines, etc.
I define my function as
def skip_comments(file):
for line in file:
if not line.strip().startswith('#'):
yield line
That way, I can just do
f = open('testfile')
for line in skip_comments(f):
print line
This is reusable across all my code, and I can add any additional handling/logging/etc. that I need.
I know that this is an old thread, but this is a generator function that I
use for my own purposes. It strips comments no matter where they
appear in the line, as well as stripping leading/trailing whitespace and
blank lines. The following source text:
# Comment line 1
# Comment line 2
# host01 # This host commented out.
host02 # This host not commented out.
host03
host04 # Oops! Included leading whitespace in error!
will yield:
host02
host03
host04
Here is documented code, which includes a demo:
def strip_comments(item, *, token='#'):
"""Generator. Strips comments and whitespace from input lines.
This generator strips comments, leading/trailing whitespace, and
blank lines from its input.
Arguments:
item (obj): Object to strip comments from.
token (str, optional): Comment delimiter. Defaults to ``#``.
Yields:
str: Next uncommented non-blank line from ``item`` with
comments and leading/trailing whitespace stripped.
"""
for line in item:
s = line.split(token, 1)[0].strip()
if s:
yield s
if __name__ == '__main__':
HOSTS = """# Comment line 1
# Comment line 2
# host01 # This host commented out.
host02 # This host not commented out.
host03
host04 # Oops! Included leading whitespace in error!""".split('\n')
hosts = strip_comments(HOSTS)
print('\n'.join(h for h in hosts))
The normal use case will be to strip the comments from a file (i.e., a hosts file, as in my example above). If this is the case, then the tail end of the above code would be modified to:
if __name__ == '__main__':
with open('aa.txt', 'r') as f:
hosts = strip_comments(f)
for host in hosts:
print('\'%s\'' % host)
A more compact version of a filtering expression can also look like this:
for line in (l for l in open(filename) if not l.startswith('#')):
# do something with line
(l for ... ) is called "generator expression" which acts here as a wrapping iterator that will filter out all unneeded lines from file while iterating over it. Don't confuse it with the same thing in square brakets [l for ... ] which is a "list comprehension" that will first read all the lines from the file into memory and only then will start iterating over it.
Sometimes you might want to have it less one-liney and more readable:
lines = open(filename)
lines = (l for l in lines if ... )
# more filters and mappings you might want
for line in lines:
# do something with line
All the filters will be executed on the fly in one iteration.
Use regex re.compile("^(?:\s+)*#|(?:\s+)") to skip the new lines and comments.
I tend to use
for line in lines:
if '#' not in line:
#do something
This will ignore the whole line, though the answer which includes rpartition has my upvote as it can include any information from before the #
a good thing to get rid of coments that works for both inline and on a line
def clear_coments(f):
new_text = ''
for line in f.readlines():
if "#" in line: line = line.split("#")[0]
new_text += line
return new_text