I am parsing json file that has the following data subset.
"title": "Revert \"testcase for check\""
In my python script I do the following:
with open('%s/staging_area/pr_info.json' % cwd) as data_file:
pr_info = json.load(data_file)
pr_title=pr_info["title"]
pr_title will contain the following information after getting the title from json object.
Revert "testcase for check"
It seems that escape characters \ are not part of the string assignment. Is there any way to retain the entire string including escape characters? Thank you so much!
In case you really need to maintain the escape characters, you will have to escape the escape characters right after reading the file and before parsing the JSON.
with open('%s/staging_area/pr_info.json' % cwd) as data_file:
raw_data_file = data_file.read().replace("\\", "\\\\\\")
pr_info = json.JSONDecoder().decode(raw_data_file)
Then pr_title should still have the original escaped characters.
What is happening is:
Replace each single backslash for three backslashes: original escape character (\) + an escaped escape character (\\).
raw_data_file is now a string instead of a file pointer, so we cannot use json.load(). But the method decode from json.JSONDecoder admits a string input.
The decoder will parse the JSON string and remove the escaped escape character, while maintaining the original one from your file.
If you really need it, you should escape it again with json and remove first and last quote:
pr_title = json.dumps(pr_title)[1:-1]
but escape characters is for escaping, raw value of string is still Revert "testcase for check". So escaping function will depend on where you data is applied (DB, HTML, XML, etc).
To explain [1:-1], the dumps escapes raw string to be JSON-valid which adds \ and surrounds the string with quotation marks ". You have to remove these quotes from resulting string. Since Python could work with string same as list you can get all letters from second to penultimate with [1:-1] which literally removes the first and last quotes:
print(pr_title)
>>> "Revert \"testcase for check\""
print(pr_title[1:-1])
>>> Revert \"testcase for check\"
If your goal is to print pr_title, then you can probably use json.dumps() to print the original text.
>>> import json
>>> j = '{"name": "\"Bob\""}'
>>> print(j)
{"name": ""Bob""}
>>> json.dumps(j)
'"{\\"name\\": \\"\\"Bob\\"\\"}"'
Related
When I write print('\') or print("\") or print("'\'"), Python doesn't print the backslash \ symbol. Instead it errors for the first two and prints '' for the third. What should I do to print a backslash?
This question is about producing a string that has a single backslash in it. This is particularly tricky because it cannot be done with raw strings. For the related question about why such a string is represented with two backslashes, see Why do backslashes appear twice?. For including literal backslashes in other strings, see using backslash in python (not to escape).
You need to escape your backslash by preceding it with, yes, another backslash:
print("\\")
And for versions prior to Python 3:
print "\\"
The \ character is called an escape character, which interprets the character following it differently. For example, n by itself is simply a letter, but when you precede it with a backslash, it becomes \n, which is the newline character.
As you can probably guess, \ also needs to be escaped so it doesn't function like an escape character. You have to... escape the escape, essentially.
See the Python 3 documentation for string literals.
A hacky way of printing a backslash that doesn't involve escaping is to pass its character code to chr:
>>> print(chr(92))
\
print(fr"\{''}")
or how about this
print(r"\ "[0])
For completeness: A backslash can also be escaped as a hex sequence: "\x5c"; or a short Unicode sequence: "\u005c"; or a long Unicode sequence: "\U0000005c". All of these will produce a string with a single backslash, which Python will happily report back to you in its canonical representation - '\\'.
This question already has answers here:
How should I write a Windows path in a Python string literal?
(5 answers)
Closed 3 years ago.
I'm trying to read a CSV file into Python (Spyder), but I keep getting an error. My code:
import csv
data = open("C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener")
data = csv.reader(data)
print(data)
I get the following error:
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes
in position 2-3: truncated \UXXXXXXXX escape
I have tried to replace the \ with \\ or with / and I've tried to put an r before "C.., but all these things didn't work.
This error occurs, because you are using a normal string as a path. You can use one of the three following solutions to fix your problem:
1: Just put r before your normal string. It converts a normal string to a raw string:
pandas.read_csv(r"C:\Users\DeePak\Desktop\myac.csv")
2:
pandas.read_csv("C:/Users/DeePak/Desktop/myac.csv")
3:
pandas.read_csv("C:\\Users\\DeePak\\Desktop\\myac.csv")
The first backslash in your string is being interpreted as a special character. In fact, because it's followed by a "U", it's being interpreted as the start of a Unicode code point.
To fix this, you need to escape the backslashes in the string. The direct way to do this is by doubling the backslashes:
data = open("C:\\Users\\miche\\Documents\\school\\jaar2\\MIK\\2.6\\vektis_agb_zorgverlener")
If you don't want to escape backslashes in a string, and you don't have any need for escape codes or quotation marks in the string, you can instead use a "raw" string, using "r" just before it, like so:
data = open(r"C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener")
You can just put r in front of the string with your actual path, which denotes a raw string. For example:
data = open(r"C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener")
Consider it as a raw string. Just as a simple answer, add r before your Windows path.
import csv
data = open(r"C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener")
data = csv.reader(data)
print(data)
Try writing the file path as "C:\\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener" i.e with double backslash after the drive as opposed to "C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener"
Add r before your string. It converts a normal string to a raw string.
As per String literals:
String literals can be enclosed within single quotes (i.e. '...') or double quotes (i.e. "..."). They can also be enclosed in matching groups of three single or double quotes (these are generally referred to as triple-quoted strings).
The backslash character (i.e. \) is used to escape characters which otherwise will have a special meaning, such as newline, backslash itself, or the quote character. String literals may optionally be prefixed with a letter r or R. Such strings are called raw strings and use different rules for backslash escape sequences.
In triple-quoted strings, unescaped newlines and quotes are allowed, except that the three unescaped quotes in a row terminate the string.
Unless an r or R prefix is present, escape sequences in strings are interpreted according to rules similar to those used by Standard C.
So ideally you need to replace the line:
data = open("C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener")
To any one of the following characters:
Using raw prefix and single quotes (i.e. '...'):
data = open(r'C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener')
Using double quotes (i.e. "...") and escaping backslash character (i.e. \):
data = open("C:\\Users\\miche\\Documents\\school\\jaar2\\MIK\\2.6\\vektis_agb_zorgverlener")
Using double quotes (i.e. "...") and forwardslash character (i.e. /):
data = open("C:/Users/miche/Documents/school/jaar2/MIK/2.6/vektis_agb_zorgverlener")
Just putting an r in front works well.
eg:
white = pd.read_csv(r"C:\Users\hydro\a.csv")
It worked for me by neutralizing the '' by f = open('F:\\file.csv')
The double \ should work for Windows, but you still need to take care of the folders you mention in your path. All of them (except the filename) must exist. Otherwise you will get an error.
I am supposed to decode the string below in a script I have made (it is a task from a webpage). In order to ensure that the decoded word will be correct, I can not change the string in any way. Since the quote marks affects the string, parts like q90:;AI is not a string, which results in a syntax error.
q0Ø:;AI"E47FRBQNBG4WNB8B4LQN8ERKC88U8GEN?T6LaNBG4GØ""N6K086HB"Ø8CRHW"+LS79Ø""N29QCLN5WNEBS8GENBG4FØ47a
Is there a way I can decode the encrypted message without changing it? As of now I am just getting syntax error when I define the string in a variable.
You can surround the string with single quotes, since double quotes are used in the string already:
>>> print 'q0Ø:;AI"E47FRBQNBG4WNB8B4LQN8ERKC88U8GEN?T6LaNBG4GØ""N6K086HB"Ø8CRHW"+LS79Ø""N29QCLN5WNEBS8GENBG4FØ47a'
q0Ã:;AI"E47FRBQNBG4WNB8B4LQN8ERKC88U8GEN?T6LaNBG4GÃ""N6K086HB"Ã8CRHW"+LS79Ã""N29QCLN5WNEBS8GENBG4FÃ47a
>>>
The following works as expected:
>>> print re.sub('(\w)"(\W)', r"\1''\2", 'The "raw string literal" is a special case of a "string literal".')
The "raw string literal'' is a special case of a "string literal''.
Since I wanted to use single quotes in the replacement expression (is that the correct terminology?), I quoted it using double quotes.
But then for my edification I tried using single quotes in the replacement expression and can't understand the results:
>>> print re.sub('(\w)"(\W)', r'\1\'\'\2', 'The "raw string literal" is a special case of a "string literal".')
The "raw string literal\'\' is a special case of a "string literal\'\'.
Shouldn't the two forms produce exactly the same output?
So, my questions are:
How do I escape a single quote in a single-quoted raw string?
How do I escape a double quote in a double-quoted raw string?
Why is it that in the first parameter to re.sub() I didn't have to use raw string, but in the second parameter I have to. Both seem like string representations of regexes to this Python noob.
If it makes a difference, am using Python 2.7.5 on Mac OS X (10.9, Mavericks).
No, they should not. A raw string literal does let you escape quotes, but the backslashes will be included:
>>> r"\'"
"\\'"
where Python echoes the resulting string as a string literal with the backslash escaped.
This is explicitly documented behaviour of the raw string literal syntax:
When an 'r' or 'R' prefix is present, a character following a backslash is included in the string without change, and all backslashes are left in the string. For example, the string literal r"\n" consists of two characters: a backslash and a lowercase 'n'. String quotes can be escaped with a backslash, but the backslash remains in the string; for example, r"\"" is a valid string literal consisting of two characters: a backslash and a double quote; r"\" is not a valid string literal (even a raw string cannot end in an odd number of backslashes).
If you didn't use a raw string literal for the second parameter, Python would interpret the \digit combination as octal byte values:
>>> '\0'
'\x00'
You can construct the same string without raw string literals with doubling the backslash:
>>> '\\1\'\'\\2'
"\\1''\\2"
To answer the questions of the OP:
How do I escape a single quote in a single-quoted raw string?
That is not possible, except if you have the special case where the single quote is preceded by a backslash (as Martijn pointed out).
How do I escape a double quote in a double-quoted raw string?
See above.
Why is it that in the first parameter to re.sub() I didn't have to use raw string, but in the second parameter I have to. Both seem like string representations of regexes to this Python noob.
Completing Martijn's answer (which only covered the second parameter): The backslashes in the first parameter are attempted to be interpreted as escape characters together with their following characters, because the string is not raw. However, because the following characters do not happen to form valid escape sequences together with a backslash, the backslash is interpreted as a character:
>>> '(\w)"(\W)'
'(\\w)"(\\W)'
>>> '(\t)"(\W)'
'(\t)"(\\W)'
This question already has answers here:
Why can't Python's raw string literals end with a single backslash?
(14 answers)
Closed 6 years ago.
I am confused here, even though raw strings convert every \ to \\ but when this \ appears in the end it raises error.
>>> r'so\m\e \te\xt'
'so\\m\\e \\te\\xt'
>>> r'so\m\e \te\xt\'
SyntaxError: EOL while scanning string literal
Update:
This is now covered in Python FAQs as well: Why can’t raw strings (r-strings) end with a backslash?
You still need \ to escape ' or " in raw strings, since otherwise the python interpreter doesn't know where the string stops. In your example, you're escaping the closing '.
Otherwise:
r'it wouldn\'t be possible to store this string'
r'since it'd produce a syntax error without the escape'
Look at the syntax highlighting to see what I mean.
Raw strings can't end in single backslashes because of how the parser works (there is no actual escaping going on, though). The workaround is to add the backslash as a non-raw string literal afterwards:
>>> print(r'foo\')
File "<stdin>", line 1
print(r'foo\')
^
SyntaxError: EOL while scanning string literal
>>> print(r'foo''\\')
foo\
Not pretty, but it works. You can add plus to make it clearer what is happening, but it's not necessary:
>>> print(r'foo' + '\\')
foo\
Python strings are processed in two steps:
First the tokenizer looks for the closing quote. It recognizes backslashes when it does this, but doesn't interpret them - it just looks for a sequence of string elements followed by the closing quote mark, where "string elements" are either (a character that's not a backslash, closing quote or a newline - except newlines are allowed in triple-quotes), or (a backslash, followed by any single character).
Then the contents of the string are interpreted (backslash escapes are processed) depending on what kind of string it is. The r flag before a string literal only affects this step.
Quote from https://docs.python.org/3.4/reference/lexical_analysis.html#literals:
Even in a raw literal, quotes can be escaped with a backslash, but the
backslash remains in the result; for example, r"\"" is a valid string
literal consisting of two characters: a backslash and a double quote;
r"\" is not a valid string literal (even a raw string cannot end in an
odd number of backslashes). Specifically, a raw literal cannot end in
a single backslash (since the backslash would escape the following
quote character). Note also that a single backslash followed by a
newline is interpreted as those two characters as part of the literal,
not as a line continuation.
So in raw string, backslash are not treated specially, except when preceding " or '. Therefore, r'\' or r"\" is not a valid string cause right quote is escaped thus making the string literal invalid. In such case, there's no difference whether r exists, i.e. r'\' is equivalent to '\' and r"\" is equivalent to "\".