Why do 3 backslashes equal 4 in a Python string? - python

Could you tell me why '?\\\?'=='?\\\\?' gives True? That drives me crazy and I can't find a reasonable answer...
>>> list('?\\\?')
['?', '\\', '\\', '?']
>>> list('?\\\\?')
['?', '\\', '\\', '?']

Basically, because python is slightly lenient in backslash processing. Quoting from https://docs.python.org/2.0/ref/strings.html :
Unlike Standard C, all unrecognized escape sequences are left in the string unchanged, i.e., the backslash is left in the string.
(Emphasis in the original)
Therefore, in python, it isn't that three backslashes are equal to four, it's that when you follow backslash with a character like ?, the two together come through as two characters, because \? is not a recognized escape sequence.

This is because backslash acts as an escape character for the character(s) immediately following it, if the combination represents a valid escape sequence. The dozen or so escape sequences are listed here. They include the obvious ones such as newline \n, horizontal tab \t, carriage return \r and more obscure ones such as named unicode characters using \N{...}, e.g. \N{WAVY DASH} which represents unicode character \u3030. The key point though is that if the escape sequence is not known, the character sequence is left in the string as is.
Part of the problem might also be that the Python interpreter output is misleading you. This is because the backslashes are escaped when displayed. However, if you print those strings, you will see the extra backslashes disappear.
>>> '?\\\?'
'?\\\\?'
>>> print('?\\\?')
?\\?
>>> '?\\\?' == '?\\?' # I don't know why you think this is True???
False
>>> '?\\\?' == r'?\\?' # but if you use a raw string for '?\\?'
True
>>> '?\\\\?' == '?\\\?' # this is the same string... see below
True
For your specific examples, in the first case '?\\\?', the first \ escapes the second backslash leaving a single backslash, but the third backslash remains as a backslash because \? is not a valid escape sequence. Hence the resulting string is ?\\?.
For the second case '?\\\\?', the first backslash escapes the second, and the third backslash escapes the fourth which results in the string ?\\?.
So that's why three backslashes is the same as four:
>>> '?\\\?' == '?\\\\?'
True
If you want to create a string with 3 backslashes you can escape each backslash:
>>> '?\\\\\\?'
'?\\\\\\?'
>>> print('?\\\\\\?')
?\\\?
or you might find "raw" strings more understandable:
>>> r'?\\\?'
'?\\\\\\?'
>>> print(r'?\\\?')
?\\\?
This turns of escape sequence processing for the string literal. See String Literals for more details.

Because \x in a character string, when x is not one of the special backslashable characters like n, r, t, 0, etc, evaluates to a string with a backslash and then an x.
>>> '\?'
'\\?'

From the python lexical analysis page under string literals at:
https://docs.python.org/2/reference/lexical_analysis.html
There is a table that lists all the recognized escape sequences.
\\ is an escape sequence that is === \
\? is not an escape sequence and is === \?
so '\\\\' is '\\' followed by '\\' which is '\\' (two escaped \)
and '\\\' is '\\' followed by '\' which is also '\\' (one escaped \ and one raw \)
also, it should be noted that python does not distinguish between single and double quotes surrounding a string literal, unlike some other languages.
So 'String' and "String" are the exact same thing in python, they do not affect the interpretation of escape sequences.

mhawke's answer pretty much covers it, I just want to restate it in a more concise form and with minimal examples that illustrate this behaviour.
I guess one thing to add is that escape processing moves from left to right, so that \n first finds the backslash and then looks for a character to escape, then finds n and escapes it; \\n finds first backslash, finds second and escapes it, then finds n and sees it as a literal n; \? finds backslash and looks for a char to escape, finds ? which cannot be escaped, and so treats \ as a literal backslash.
As mhawke noted, the key here is that interactive interpreter escapes the backslash when displaying a string. I'm guessing the reason for that is to ensure that text strings copied from interpreter into code editor are valid python strings. However, in this case this allowance for convenience causes confusion.
>>> print('\?') # \? is not a valid escape code so backslash is left as-is
\?
>>> print('\\?') # \\ is a valid escape code, resulting in a single backslash
'\?'
>>> '\?' # same as first example except that interactive interpreter escapes the backslash
\\?
>>> '\\?' # same as second example, backslash is again escaped
\\?

Related

Replacing a string by a string with backslashes

I am creating a program that automatically generates my reports in LaTeX, where I have to escape special LaTeX characters. Basically, whenever I read $ or _ or %, etc, I have to replace it by \$, \_ and \%, respectively.
I naively tried to do mystring.replace('$','\$'), yet it adds a double backslash, as shown below:
my_text_to_parse = "$x^2+2\cdot x + 2 = 0$"
my_text_to_parse.replace('$','\$')
#=> "\\$x^2+2\cdot x + 2 = 0\\$"
Is there any way to avoid doubling escape characters?
You're seeing the double backslash because you're getting the representation of the string, not the output. In the representation, it prints an backslash because \ is a protected character and therefore must be escaped. This is because it is used in special characters (e.g. \t, \n) and usage might be confused.. When the string is actually printed or saved, those double backslashes should be printed properly as a single backslash.
For example, compare
print('\')
# SyntaxError: EOL while scanning string literal
to
print('\\')
# \
In the first string, the second quotation mark is being escaped by the backslash. This shows why you generally can't use raw backslashes in strings. In the second string, the second backslash is being escaped by the first. The two backslashes get interpreted as a single one.
print(repr('\\'))
# '\\'
But the representation of the second string still shows both backslashes. This behavior is the same as other special characters such as \n, where it can be a bit easier to see the issue. Just as \n is the special character that means line break, \\ is the special character that means single backslash.
print('hi\nmom')
# hi
# mom
print(repr('hi\nmom'))
# 'hi\nmom'
To actually answer your question, the way you're doing it should work properly, but you probably don't want to do it quite that way. This is because creating a string with '\$' doesn't make this escaping issue clear. It seems like it is a special character \$ in the same way that \n is a special character, but because there is no character defined like that, the python interpreter is smart enough to replace the single backslash with a double backslash. But you generally don't want to rely on that behavior.
A better way to do it is to explicitly escape the backslash with another one or to use a raw string, where no escaping is allowed. All of these will give the same result.
s = '$x^2+2\\cdot x + 2 = 0$'
print(s.replace('$', '\$')) # Technically works, but not as clear
# \$x^2+2\cdot x + 2 = 0\$
print(s.replace('$', '\\$')) # Escaping the backslash
# \$x^2+2\cdot x + 2 = 0\$
print(s.replace('$', r'\$')) # Using a raw string
# \$x^2+2\cdot x + 2 = 0\$
print re.sub(r"\$","\$",x)
You can try re.sub.It will give the expected result.

What does the "r" in pythons re.compile(r' pattern flags') mean?

I am reading through http://docs.python.org/2/library/re.html. According to this the "r" in pythons re.compile(r' pattern flags') refers the raw string notation :
The solution is to use Python’s raw string notation for regular
expression patterns; backslashes are not handled in any special way in
a string literal prefixed with 'r'. So r"\n" is a two-character string
containing '\' and 'n', while "\n" is a one-character string
containing a newline. Usually patterns will be expressed in Python
code using this raw string notation.
Would it be fair to say then that:
re.compile(r pattern) means that "pattern" is a regex while, re.compile(pattern) means that "pattern" is an exact match?
As #PauloBu stated, the r string prefix is not specifically related to regex's, but to strings generally in Python.
Normal strings use the backslash character as an escape character for special characters (like newlines):
>>> print('this is \n a test')
this is
a test
The r prefix tells the interpreter not to do this:
>>> print(r'this is \n a test')
this is \n a test
>>>
This is important in regular expressions, as you need the backslash to make it to the re module intact - in particular, \b matches empty string specifically at the start and end of a word. re expects the string \b, however normal string interpretation '\b' is converted to the ASCII backspace character, so you need to either explicitly escape the backslash ('\\b'), or tell python it is a raw string (r'\b').
>>> import re
>>> re.findall('\b', 'test') # the backslash gets consumed by the python string interpreter
[]
>>> re.findall('\\b', 'test') # backslash is explicitly escaped and is passed through to re module
['', '']
>>> re.findall(r'\b', 'test') # often this syntax is easier
['', '']
No, as the documentation pasted in explains the r prefix to a string indicates that the string is a raw string.
Because of the collisions between Python escaping of characters and regex escaping, both of which use the back-slash \ character, raw strings provide a way to indicate to python that you want an unescaped string.
Examine the following:
>>> "\n"
'\n'
>>> r"\n"
'\\n'
>>> print "\n"
>>> print r"\n"
\n
Prefixing with an r merely indicates to the string that backslashes \ should be treated literally and not as escape characters for python.
This is helpful, when for example you are searching on a word boundry. The regex for this is \b, however to capture this in a Python string, I'd need to use "\\b" as the pattern. Instead, I can use the raw string: r"\b" to pattern match on.
This becomes especially handy when trying to find a literal backslash in regex. To match a backslash in regex I need to use the pattern \\, to escape this in python means I need to escape each slash and the pattern becomes "\\\\", or the much simpler r"\\".
As you can guess in longer and more complex regexes, the extra slashes can get confusing, so raw strings are generally considered the way to go.
No. Not everything in regex syntax needs to be preceded by \, so ., *, +, etc still have special meaning in a pattern
The r'' is often used as a convenience for regex that do need a lot of \ as it prevents the clutter of doubling up the \

How can I print a single backslash?

When I write print('\') or print("\") or print("'\'"), Python doesn't print the backslash \ symbol. Instead it errors for the first two and prints '' for the third. What should I do to print a backslash?
This question is about producing a string that has a single backslash in it. This is particularly tricky because it cannot be done with raw strings. For the related question about why such a string is represented with two backslashes, see Why do backslashes appear twice?. For including literal backslashes in other strings, see using backslash in python (not to escape).
You need to escape your backslash by preceding it with, yes, another backslash:
print("\\")
And for versions prior to Python 3:
print "\\"
The \ character is called an escape character, which interprets the character following it differently. For example, n by itself is simply a letter, but when you precede it with a backslash, it becomes \n, which is the newline character.
As you can probably guess, \ also needs to be escaped so it doesn't function like an escape character. You have to... escape the escape, essentially.
See the Python 3 documentation for string literals.
A hacky way of printing a backslash that doesn't involve escaping is to pass its character code to chr:
>>> print(chr(92))
\
print(fr"\{''}")
or how about this
print(r"\ "[0])
For completeness: A backslash can also be escaped as a hex sequence: "\x5c"; or a short Unicode sequence: "\u005c"; or a long Unicode sequence: "\U0000005c". All of these will produce a string with a single backslash, which Python will happily report back to you in its canonical representation - '\\'.

Replace '\' character in python

I don't know why i can't find it, but i wanted to replace the special character '\' in python.
I have a String within i have '\' characters but i confident find the solution, to replace it with '-'.
This is what happening while i am trying to replace,
>>> x = 'hello\world'
>>> x
'hello\\world'
>>> x.replace('\', '-')
File "<stdin>", line 1
x.replace('\', '-')
SyntaxError: EOL while scanning string literal
EDIT:
Do try this it in the eclipse IDLE
x = 'hello\world'
print x
x.replace('\\', '-')
print x
Output:
hello\world
hello\world
You need to escape it with another backslash:
x.replace('\\', '-')
Backslashes are special, in that they are used to introduce non-printing characters like newlines into a string.
It's also how you add a ' character to a '-quoted string, which is what Python thinks you were trying to do. It sees \' and interprets as a literal quote within the string, rather than letting the ' end the string. Then it gets to the end of the string and finds EOL ("end of line") before the end of the string.
To introduce a real backslash, you need to double it. You can see that Python itself did this when printing the representation of your initial string here:
>>> x
'hello\\world'
Note the double backslash.
You ought to use a double backslash when specifying your string in the first place. The reason that doesn't need it is that \w is not a special character, so it gets interpreted as a literal backslash and a w. Had you said 'Hello\now' you would have a string with a newline in it.
You could have also marked the string as a "raw" string by prepending it with r as in r'hello\world'. This marks the string as not being eligible for any substitutions of special characters.
According to docs:
The backslash (\) character is used to escape characters that
otherwise have a special meaning, such as newline, backslash itself,
or the quote character.
You need to escape backslash with another backslash:
x.replace('\\', '-')
This \' is interpreted as a special character. Escape it:
x.replace('\\', '-')
in python your string
x = 'hello\world'
is replaced as x = "hello\world"
so to achieve u have to write
x.replace('\\','-')

How to escape “\” characters in python

i am very new to regular expression and trying get "\" character using python
normally i can escape "\" like this
print ("\\");
print ("i am \\nit");
output
\
i am \nit
but when i use the same in regX it didn't work as i thought
print (re.findall(r'\\',"i am \\nit"));
and return me output
['\\']
can someone please explain why
EDIT: The problem is actually how print works with lists & strings. It prints the representation of the string, not the string itself, the representation of a string containing just a backslash is '\\'. So findall is actually finding the single backslash correctly, but print isn't printing it as you'd expect. Try:
>>> print(re.findall(r'\\',"i am \\nit")[0])
\
(The following is my original answer, it can be ignored (it's entirely irrelevant), I'd misinterpreted the question initially. But it seems to have been upvoted a bit, so I'll leave it here.)
The r prefix on a string means the string is in "raw" mode, that is, \ are not treated as special characters (it doesn't have anything to do with "regex").
However, r'\' doesn't work, as you can't end a raw string with a backslash, it's stated in the docs:
Even in a raw string, string quotes can be escaped with a backslash, but the backslash remains in the string; for example, r"\"" is a valid string literal consisting of two characters: a backslash and a double quote; r"\" is not a valid string literal (even a raw string cannot end in an odd number of backslashes). Specifically, a raw string cannot end in a single backslash (since the backslash would escape the following quote character).
But you actually can use a non-raw string to get a single backslash: "\\".
can someone please explain why
Because re.findall found one match, and the match text consisted of a backslash. It gave you a list with one element, which is a string, which has one character, which is a backslash.
That is written ['\\'] because '\\' is how you write "a string with one backslash" - just like you had to do when you wrote the example code print "\\".
Note that you're using two different kinds of string literal here -- there's the regular string "a string" and the raw string r"a raw string". Regular string literals observe backslash escaping, so to actually put a backslash in the string, you need to escape it too. Raw string literals treat backslashes like any other character, so you're more limited in which characters you can actually put in the string (no specials that need an escape code) but it's easier to enter things like regular expressions, because you don't need to double up backslashes if you need to add a backslash to have meaning inside the string, not just when creating the string.
It is unnecessary to escape backslashes in raw strings, unless the backslash immediately precedes the closing quote.

Categories