I want to catch any warning that contains the string 'value'.
From this question, I see this example to catch a specific message:
warnings.filterwarnings('error', 'Unknown table .*')
The docs say about the message parameter:
message is a string containing a regular expression that the warning message must match (the match is compiled to always be case-insensitive).
I have the following code but no errors are thrown and instead I'm just getting the warnings which I cannot catch.
warnings.filterwarnings('error', message='\bvalue\b')
What am I missing? As far as I know, that regex should work for matching the 'value' string.
Python's regular expression syntax is documented here, and the first thing it says is:
Regular expressions use the backslash character ('\') to indicate special forms or to allow special characters to be used without invoking their special meaning. This collides with Python’s usage of the same character for the same purpose in string literals; for example, to match a literal backslash, one might have to write '\\' as the pattern string, because the regular expression must be \, and each backslash must be expressed as \ inside a regular Python string literal.
The solution is to use Python’s raw string notation for regular expression patterns; backslashes are not handled in any special way in a string literal prefixed with 'r'. So r"\n" is a two-character string containing '\' and 'n', while "\n" is a one-character string containing a newline. Usually patterns will be expressed in Python code using this raw string notation.
In Python, as in many languages, the string '\b' corresponds to ASCII backspace
(link).
You need to escape your backslash characters, or else use Python's special "raw/regex" prefix:
warnings.filterwarnings('error', message='\\bvalue\\b')
warnings.filterwarnings('error', message=r'\bvalue\b')
Related
This is my python code line which is giving me invalid escape sequence '/' lint issue.
pattern = 'gs:\/\/([a-z0-9-]+)\/(.+)$' # for regex matching
It is giving me out that error for all the backslash I used here .
any idea how to resolve this ?
There's two issues here:
Since this is not a raw string, the backslashes are string escapes, not regexp escapes. Since \/ is not a valid string escape sequence, you get that warning. Use a raw string so that the backslashes will be ignored by the string parser and passed to the regexp engine. See What exactly is a "raw string regex" and how can you use it?
In some languages / is part of the regular expression syntax (it's the delimiter around the regexp), so they need to be escaped. But Python doesn't use / this way, so there's no need to escape them in the first place.
Use this:
pattern = r'gs://([a-z0-9-]+)/(.+)$' # for regex matching
I would like to know the reason I get the same result when using string prefix "r" or not when looking for a period (full stop) using python regex.
After reading a number sources (Links below) a multiple times and experimenting with in code to find the same result (again see below), I am still unsure of:
What is the difference when using string prefix "r" and not using string prefix "r", when looking for a period using regex?
Which way is considered the correct way of finding a period in a string using python regex with string prefix "r" or without string prefix "r"?
re.compile("\.").sub("!", "blah.")
'blah!'
re.compile(r"\.").sub("!", "blah.")
'blah!'
re.compile(r"\.").search("blah.").group()
'.'
re.compile("\.").search("blah.").group()
'.'
Sources I have looked at:
Python docs: string literals
http://docs.python.org/2/reference/lexical_analysis.html#string-literals
Regular expression to replace "escaped" characters with their originals
Python regex - r prefix
r prefix is for raw strings
http://forums.udacity.com/questions/7000217/r-prefix-is-for-raw-strings
The raw string notation is just that, a notation to specify a string value. The notation results in different string values when it comes to backslash escapes recognized by the normal string notation. Because regular expressions also attach meaning to the backslash character, raw string notation is quite handy as it avoids having to use excessive escaping.
Quoting from the Python Regular Expression HOWTO:
The solution is to use Python’s raw string notation for regular expressions; backslashes are not handled in any special way in a string literal prefixed with 'r', so r"\n" is a two-character string containing '\' and 'n', while "\n" is a one-character string containing a newline. Regular expressions will often be written in Python code using this raw string notation.
The \. combination has no special meaning in regular python strings, so there is no difference, at all between the result of '\.' and r'\.'; you can use either:
>>> len('\.')
2
>>> len(r'\.')
2
Raw strings only make a difference when the backslash + other characters do have special meaning in regular string notation:
>>> '\b'
'\x08'
>>> r'\b'
'\\b'
>>> len('\b')
1
>>> len(r'\b')
2
The \b combination has special meaning; in a regular string it is interpreted as the backspace character. But regular expressions see \b as a word boundary anchor, so you'd have to use \\b in your Python string every time you wanted to use this in a regular expression. Using r'\b' instead makes it much easier to read and write your expressions.
The regular expression functions are passed string values; the result of Python interpreting your string literal. The functions do not know if you used raw or normal string literal syntax.
Python has this way of specifying regular expression pattern, where all special character should not be treated as special. From the docs:
So r"\n" is a two-character string containing '\' and 'n', while "\n" is a one-character string containing a newline.
Why then does this works?
import re
print re.split(r"\n", "1\n2\n3")
The first argument should be "\" and "n" and the second one should contain two newlines. But it prints:
['1', '2', '3']
The first one does contain backslash-and-n, but in regular-expression-language, backslash-and-n means newline (just like it does in Python string syntax). That is, the string r"\n" does not contain an actual newline, but it contains something that tells the regular expression engine to look for actual newlines.
If you want to search for a backslash followed by n, you need to use r"\\n".
The point of the raw strings is that they block Python's basic intepretation of string escapes, allowing you to use the backslash for its regular-expression meaning. If you don't want the regular-expression meaning, you still have to use two backslashes, as in my example above. But without raw strings it would be even worse: if you wanted to search for literal backslash-n without a raw string, you'd have to use "\\\\n". If the raw string blocked interpretation of the regular expression special characters (so that plain "\n" really meant backslash-n), you wouldn't have any way of using the regular expression syntax at all.
I am seeing the following phenomenon, couldn't seem to figure it out, and didn't find anything with some search through archives:
if I type in:
>>> if re.search(r'\n',r'this\nis\nit'):<br>
... print 'found it!'<br>
... else:<br>
... print "didn't find it"<br>
...
I will get:
didn't find it!
However, if I type in:
>>> if re.search(r'\\n',r'this\nis\nit'):<br>
... print 'found it!'<br>
... else:<br>
... print "didn't find it"<br>
...
Then I will get:
found it!
(The first one only has one backslash on the r'\n' whereas the second one has two backslashes in a row on the r'\\n' ... even this interpreter is removing one of them.)
I can guess what is going on, but I don't understand the official mechanism as to why this is happening: in the first case, I need to escape two things: both the regular expression and the special strings. "Raw" lets me escape the special strings, but not the regular expression.
But there will never be a regular expression in the second string, since it is the string being matched. So there is only a need to escape once.
However, something doesn't seem consistent to me: how am I supposed to ensure that the characters REALLY ARE taken literally in the first case? Can I type rr'' ? Or do I have to ensure that I escape things twice?
On a similar vein, how do I ensure that a variable is taken literally (or that it is NOT taken literally)? E.g., what if I had a variable tmp = 'this\nis\nmy\nhome', and I really wanted to find the literal combination of a slash and an 'n', instead of a newline?
Thanks!Mike
re.search(r'\n', r'this\nis\nit')
As you said, "there will never be a regular expression in the second string." So we need to look at these strings differently: the first string is a regex, the second just a string. Usually your second string will not be raw, so any backslashes are Python-escapes, not regex-escapes.
So the first string consists of a literal "\" and an "n". This is interpreted by the regex parser as a newline (docs: "Most of the standard escapes supported by Python string literals are also accepted by the regular expression parser"). So your regex will be searching for a newline character.
Your second string consists of the string "this" followed by a literal "\" and an "n". So this string does not contain an actual newline character. Your regex will not match.
As for your second regex:
re.search(r'\\n', r'this\nis\nit')
This version matches because your regex contains three characters: a literal "\", another literal "\" and an "n". The regex parser interprets the two slashes as a single "\" character, followed by an "n". So your regex will be searching for a "\" followed by an "n", which is found within the string. But that isn't very helpful, since it has nothing to do with newlines.
Most likely what you want is to drop the r from the second string, thus treating it as a normal Python string.
re.search(r'\n', 'this\nis\nit')
In this case, your regex (as before) is searching for a newline character. And, it finds it, because the second string contains the word "this" followed by a newline.
Escaping special sequences in string literals is one thing, escaping regular expression special characters is another. The row string modifier only effects the former.
Technically, re.search accepts two strings and passes the first to the regex builder with re.compile. The compiled regex object is used to search patterns inside simple strings. The second string is never compiled and thus it is not subject to regex special character rules.
If the regex builder receives a \n after the string literal is processed, it converts this sequence to a newline character. You also have to escape it if you need the match the sequence instead.
All rationale behind this is that regular expressions are not part of the language syntax. They are rather handled within the standard library inside the re module with common building blocks of the language.
The re.compile function uses special characters and escaping rules compatible with most commonly used regex implementations. However, the Python interpreter is not aware of the whole regular expression concept and it does not know whether a string literal will be compiled into a regex object or not. As a result, Python can't provide any kind syntax simplification such as the ones you suggested.
Regexes have their own meaning for literal backslashes, as character classes like \d. If you actually want a literal backslash character, you will in fact need to double-escape it. It's really not supposed to be parallel since you're comparing a regex to a string.
Raw strings are just a convenience, and it would be way overkill to have double-raw strings.
Why in python I can't use:
r"c:\"
When a string must contain the same quote character with which it starts, escaping that character is the only available workaround -- so the design alternative was either to make raw-string literals unable to contain their leading quote character, or keep the "backlash escapes" convention, even in string literals, just for quote characters.
Since raw-string literals were designed for handy representation of regular expression patterns (not for DOS / Windows paths!-), and in RE patterns a trailing backslash is never necessary, the design decision was easy (based on the real use case for raw-string literals).
Use "c:/" or "c:\\". Raw string literals are for escaping escape-sequences, not for including literal backslashes, though they do work that way, except in this exact case.
Its a known case I think, better use "c:\\" for that case.
From the documentation:
... a raw string cannot end in a single backslash (since the backslash would escape the following quote character).
.
Even with raw strings, \" causes the " not to be interpreted as the end of the string (though the backslash gets into your string), so r"foo\"bar" would be a legal string. This is convenient enough when writing regex but not great for writing paths.
This is not a big deal as most of the time you should be using os.path and other modules to deal with your paths.
found in Design and History FAQ http://docs.python.org/faq/design.html#why-can-t-raw-strings-r-strings-end-with-a-backslash
Raw strings were designed to ease
creating input for processors (chiefly
regular expression engines) that want
to do their own backslash escape
processing. Such processors consider
an unmatched trailing backslash to be
an error anyway, so raw strings
disallow that. In return, they allow
you to pass on the string quote
character by escaping it with a
backslash. These rules work well when
r-strings are used for their intended
purpose.