We Keep Coding

What will be the regex to find till the last "]" If any other square brackets come it will consider inside [duplicate]

My Python program receives JSON data, and I need to get bits of information out of it. How can I parse the data and use the result? I think I need to use json.loads for this task, but I can't understand how to do it.
For example, suppose that I have jsonStr = '{"one" : "1", "two" : "2", "three" : "3"}'. Given this JSON, and an input of "two", how can I get the corresponding data, "2"?
Beware that .load is for files; .loads is for strings. See also: Reading JSON from a file.
Occasionally, a JSON document is intended to represent tabular data. If you have something like this and are trying to use it with Pandas, see Python - How to convert JSON File to Dataframe.
Some data superficially looks like JSON, but is not JSON.
For example, sometimes the data comes from applying repr to native Python data structures. The result may use quotes differently, use title-cased True and False rather than JSON-mandated true and false, etc. For such data, see Convert a String representation of a Dictionary to a dictionary or How to convert string representation of list to a list.
Another common variant format puts separate valid JSON-formatted data on each line of the input. (Proper JSON cannot be parsed line by line, because it uses balanced brackets that can be many lines apart.) This format is called JSONL. See Loading JSONL file as JSON objects.
Sometimes JSON data from a web source is padded with some extra text. In some contexts, this works around security restrictions in browsers. This is called JSONP and is described at What is JSONP, and why was it created?. In other contexts, the extra text implements a security measure, as described at Why does Google prepend while(1); to their JSON responses?. Either way, handling this in Python is straightforward: simply identify and remove the extra text, and proceed as before.

Very simple:
import json
data = json.loads('{"one" : "1", "two" : "2", "three" : "3"}')
print(data['two']) # or `print data['two']` in Python 2

Sometimes your json is not a string. For example if you are getting a json from a url like this:
j = urllib2.urlopen('http://site.com/data.json')
you will need to use json.load, not json.loads:
j_obj = json.load(j)
(it is easy to forget: the 's' is for 'string')

For URL or file, use json.load(). For string with .json content, use json.loads().
#! /usr/bin/python
import json
# from pprint import pprint
json_file = 'my_cube.json'
cube = '1'
with open(json_file) as json_data:
data = json.load(json_data)
# pprint(data)
print "Dimension: ", data['cubes'][cube]['dim']
print "Measures: ", data['cubes'][cube]['meas']

Following is simple example that may help you:
json_string = """
{
"pk": 1,
"fa": "cc.ee",
"fb": {
"fc": "",
"fd_id": "12345"
}
}"""
import json
data = json.loads(json_string)
if data["fa"] == "cc.ee":
data["fb"]["new_key"] = "cc.ee was present!"
print json.dumps(data)
The output for the above code will be:
{"pk": 1, "fb": {"new_key": "cc.ee was present!", "fd_id": "12345",
"fc": ""}, "fa": "cc.ee"}
Note that you can set the ident argument of dump to print it like so (for example,when using print json.dumps(data , indent=4)):
{
"pk": 1,
"fb": {
"new_key": "cc.ee was present!",
"fd_id": "12345",
"fc": ""
},
"fa": "cc.ee"
}

Parsing the data
Using the standard library json module
For string data, use json.loads:
import json
text = '{"one" : "1", "two" : "2", "three" : "3"}'
parsed = json.loads(example)
For data that comes from a file, or other file-like object, use json.load:
import io, json
# create an in-memory file-like object for demonstration purposes.
text = '{"one" : "1", "two" : "2", "three" : "3"}'
stream = io.StringIO(text)
parsed = json.load(stream) # load, not loads
It's easy to remember the distinction: the trailing s of loads stands for "string". (This is, admittedly, probably not in keeping with standard modern naming practice.)
Note that json.load does not accept a file path:
>>> json.load('example.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
Both of these functions provide the same set of additional options for customizing the parsing process. Since 3.6, the options are keyword-only.
For string data, it is also possible to use the JSONDecoder class provided by the library, like so:
import json
text = '{"one" : "1", "two" : "2", "three" : "3"}'
decoder = json.JSONDecoder()
parsed = decoder.decode(text)
The same keyword parameters are available, but now they are passed to the constructor of the JSONDecoder, not the .decode method. The main advantage of the class is that it also provides a .raw_decode method, which will ignore extra data after the end of the JSON:
import json
text_with_junk = '{"one" : "1", "two" : "2", "three" : "3"} ignore this'
decoder = json.JSONDecoder()
# `amount` will count how many characters were parsed.
parsed, amount = decoder.raw_decode(text_with_junk)
Using requests or other implicit support
When data is retrieved from the Internet using the popular third-party requests library, it is not necessary to extract .text (or create any kind of file-like object) from the Response object and parse it separately. Instead, the Response object directly provides a .json method which will do this parsing:
import requests
response = requests.get('https://www.example.com')
parsed = response.json()
This method accepts the same keyword parameters as the standard library json functionality.
Using the results
Parsing by any of the above methods will result, by default, in a perfectly ordinary Python data structure, composed of the perfectly ordinary built-in types dict, list, str, int, float, bool (JSON true and false become Python constants True and False) and NoneType (JSON null becomes the Python constant None).
Working with this result, therefore, works the same way as if the same data had been obtained using any other technique.
Thus, to continue the example from the question:
>>> parsed
{'one': '1', 'two': '2', 'three': '3'}
>>> parsed['two']
'2'
I emphasize this because many people seem to expect that there is something special about the result; there is not. It's just a nested data structure, though dealing with nesting is sometimes difficult to understand.
Consider, for example, a parsed result like result = {'a': [{'b': 'c'}, {'d': 'e'}]}. To get 'e' requires following the appropriate steps one at a time: looking up the a key in the dict gives a list [{'b': 'c'}, {'d': 'e'}]; the second element of that list (index 1) is {'d': 'e'}; and looking up the 'd' key in there gives the 'e' value. Thus, the corresponding code is result['a'][1]['d']: each indexing step is applied in order.
See also How can I extract a single value from a nested data structure (such as from parsing JSON)?.
Sometimes people want to apply more complex selection criteria, iterate over nested lists, filter or transform the data, etc. These are more complex topics that will be dealt with elsewhere.
Common sources of confusion
JSON lookalikes
Before attempting to parse JSON data, it is important to ensure that the data actually is JSON. Check the JSON format specification to verify what is expected. Key points:
The document represents one value (normally a JSON "object", which corresponds to a Python dict, but every other type represented by JSON is permissible). In particular, it does not have a separate entry on each line - that's JSONL.
The data is human-readable after using a standard text encoding (normally UTF-8). Almost all of the text is contained within double quotes, and uses escape sequences where appropriate.
Dealing with embedded data
Consider an example file that contains:
{"one": "{\"two\": \"three\", \"backslash\": \"\\\\\"}"}
The backslashes here are for JSON's escape mechanism.
When parsed with one of the above approaches, we get a result like:
>>> example = input()
{"one": "{\"two\": \"three\", \"backslash\": \"\\\\\"}"}
>>> parsed = json.loads(example)
>>> parsed
{'one': '{"two": "three", "backslash": "\\\\"}'}
Notice that parsed['one'] is a str, not a dict. As it happens, though, that string itself represents "embedded" JSON data.
To replace the embedded data with its parsed result, simply access the data, use the same parsing technique, and proceed from there (e.g. by updating the original result in place):
>>> parsed['one'] = json.loads(parsed['one'])
>>> parsed
{'one': {'two': 'three', 'backslash': '\\'}}
Note that the '\\' part here is the representation of a string containing one actual backslash, not two. This is following the usual Python rules for string escapes, which brings us to...
JSON escaping vs. Python string literal escaping
Sometimes people get confused when trying to test code that involves parsing JSON, and supply input as an incorrect string literal in the Python source code. This especially happens when trying to test code that needs to work with embedded JSON.
The issue is that the JSON format and the string literal format each have separate policies for escaping data. Python will process escapes in the string literal in order to create the string, which then still needs to contain escape sequences used by the JSON format.
In the above example, I used input at the interpreter prompt to show the example data, in order to avoid confusion with escaping. Here is one analogous example using a string literal in the source:
>>> json.loads('{"one": "{\\"two\\": \\"three\\", \\"backslash\\": \\"\\\\\\\\\\"}"}')
{'one': '{"two": "three", "backslash": "\\\\"}'}
To use a double-quoted string literal instead, double-quotes in the string literal also need to be escaped. Thus:
>>> json.loads('{\"one\": \"{\\\"two\\\": \\\"three\\\", \\\"backslash\\\": \\\"\\\\\\\\\\\"}\"}')
{'one': '{"two": "three", "backslash": "\\\\"}'}
Each sequence of \\\" in the input becomes \" in the actual JSON data, which becomes " (embedded within a string) when parsed by the JSON parser. Similarly, \\\\\\\\\\\" (five pairs of backslashes, then an escaped quote) becomes \\\\\" (five backslashes and a quote; equivalently, two pairs of backslashes, then an escaped quote) in the actual JSON data, which becomes \\" (two backslashes and a quote) when parsed by the JSON parser, which becomes \\\\" (two escaped backslashes and a quote) in the string representation of the parsed result (since now, the quote does not need escaping, as Python can use single quotes for the string; but the backslashes still do).
Simple customization
Aside from the strict option, the keyword options available for json.load and json.loads should be callbacks. The parser will call them, passing in portions of the data, and use whatever is returned to create the overall result.
The "parse" hooks are fairly self-explanatory. For example, we can specify to convert floating-point values to decimal.Decimal instances instead of using the native Python float:
>>> import decimal
>>> json.loads('123.4', parse_float=decimal.Decimal)
Decimal('123.4')
or use floats for every value, even if they could be converted to integer instead:
>>> json.loads('123', parse_int=float)
123.0
or refuse to convert JSON's representations of special floating-point values:
>>> def reject_special_floats(value):
... raise ValueError
...
>>> json.loads('Infinity')
inf
>>> json.loads('Infinity', parse_constant=reject_special_floats)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 370, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
File "<stdin>", line 2, in reject_special_floats
ValueError
Customization example using object_hook and object_pairs_hook
object_hook and object_pairs_hook can be used to control what the parser does when given a JSON object, rather than creating a Python dict.
A supplied object_pairs_hook will be called with one argument, which is a list of the key-value pairs that would otherwise be used for the dict. It should return the desired dict or other result:
>>> def process_object_pairs(items):
... return {k: f'processed {v}' for k, v in items}
...
>>> json.loads('{"one": 1, "two": 2}', object_pairs_hook=process_object_pairs)
{'one': 'processed 1', 'two': 'processed 2'}
A supplied object_hook will instead be called with the dict that would otherwise be created, and the result will substitute:
>>> def make_items_list(obj):
... return list(obj.items())
...
>>> json.loads('{"one": 1, "two": 2}', object_hook=make_items_list)
[('one', 1), ('two', 2)]
If both are supplied, the object_hook will be ignored and only the object_items_hook will be used.
Text encoding issues and bytes/unicode confusion
JSON is fundamentally a text format. Input data should be converted from raw bytes to text first, using an appropriate encoding, before the file is parsed.
In 3.x, loading from a bytes object is supported, and will implicitly use UTF-8 encoding:
>>> json.loads('"text"')
'text'
>>> json.loads(b'"text"')
'text'
>>> json.loads('"\xff"') # Unicode code point 255
'ÿ'
>>> json.loads(b'"\xff"') # Not valid UTF-8 encoded data!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 343, in loads
s = s.decode(detect_encoding(s), 'surrogatepass')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 1: invalid start byte
UTF-8 is generally considered the default for JSON. While the original specification, ECMA-404 does not mandate an encoding (it only describes "JSON text", rather than JSON files or documents), RFC 8259 demands:
JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].
In such a "closed ecosystem" (i.e. for local documents that are encoded differently and will not be shared publicly), explicitly apply the appropriate encoding first:
>>> json.loads(b'"\xff"'.decode('iso-8859-1'))
'ÿ'
Similarly, JSON files should be opened in text mode, not binary mode. If the file uses a different encoding, simply specify that when opening it:
with open('example.json', encoding='iso-8859-1') as f:
print(json.load(f))
In 2.x, strings and byte-sequences were not properly distinguished, which resulted in a lot of problems and confusion particularly when working with JSON.
Actively maintained 2.x codebases (please note that 2.x itself has not been maintained since Jan 1, 2020) should consistently use unicode values to represent text and str values to represent raw data (str is an alias for bytes in 2.x), and accept that the repr of unicode values will have a u prefix (after all, the code should be concerned with what the value actually is, not what it looks like at the REPL).
Historical note: simplejson
simplejson is simply the standard library json module, but maintained and developed externally. It was originally created before JSON support was added to the Python standard library. In 2.6, the simplejson project was incorporated into the standard library as json. Current development maintains compatibility back to 2.5, although there is also an unmaintained, legacy branch that should support as far back as 2.2.
The standard library generally uses quite old versions of the package; for example, my 3.8.10 installation reports
>>> json.__version__
'2.0.9'
whereas the most recent release (as of this writing) is 3.18.1. (The tagged releases in the Github repository only go as far back as 3.8.2; the 2.0.9 release dates to 2009.
I have as yet been unable to find comprehensive documentation of which simplejson versions correspond to which Python releases.

TypeError: string indices must be integers, not 'str' , Python [duplicate]

My Python program receives JSON data, and I need to get bits of information out of it. How can I parse the data and use the result? I think I need to use json.loads for this task, but I can't understand how to do it.
For example, suppose that I have jsonStr = '{"one" : "1", "two" : "2", "three" : "3"}'. Given this JSON, and an input of "two", how can I get the corresponding data, "2"?
Beware that .load is for files; .loads is for strings. See also: Reading JSON from a file.
Occasionally, a JSON document is intended to represent tabular data. If you have something like this and are trying to use it with Pandas, see Python - How to convert JSON File to Dataframe.
Some data superficially looks like JSON, but is not JSON.
For example, sometimes the data comes from applying repr to native Python data structures. The result may use quotes differently, use title-cased True and False rather than JSON-mandated true and false, etc. For such data, see Convert a String representation of a Dictionary to a dictionary or How to convert string representation of list to a list.
Another common variant format puts separate valid JSON-formatted data on each line of the input. (Proper JSON cannot be parsed line by line, because it uses balanced brackets that can be many lines apart.) This format is called JSONL. See Loading JSONL file as JSON objects.
Sometimes JSON data from a web source is padded with some extra text. In some contexts, this works around security restrictions in browsers. This is called JSONP and is described at What is JSONP, and why was it created?. In other contexts, the extra text implements a security measure, as described at Why does Google prepend while(1); to their JSON responses?. Either way, handling this in Python is straightforward: simply identify and remove the extra text, and proceed as before.

Very simple:
import json
data = json.loads('{"one" : "1", "two" : "2", "three" : "3"}')
print(data['two']) # or `print data['two']` in Python 2

Sometimes your json is not a string. For example if you are getting a json from a url like this:
j = urllib2.urlopen('http://site.com/data.json')
you will need to use json.load, not json.loads:
j_obj = json.load(j)
(it is easy to forget: the 's' is for 'string')

For URL or file, use json.load(). For string with .json content, use json.loads().
#! /usr/bin/python
import json
# from pprint import pprint
json_file = 'my_cube.json'
cube = '1'
with open(json_file) as json_data:
data = json.load(json_data)
# pprint(data)
print "Dimension: ", data['cubes'][cube]['dim']
print "Measures: ", data['cubes'][cube]['meas']

Following is simple example that may help you:
json_string = """
{
"pk": 1,
"fa": "cc.ee",
"fb": {
"fc": "",
"fd_id": "12345"
}
}"""
import json
data = json.loads(json_string)
if data["fa"] == "cc.ee":
data["fb"]["new_key"] = "cc.ee was present!"
print json.dumps(data)
The output for the above code will be:
{"pk": 1, "fb": {"new_key": "cc.ee was present!", "fd_id": "12345",
"fc": ""}, "fa": "cc.ee"}
Note that you can set the ident argument of dump to print it like so (for example,when using print json.dumps(data , indent=4)):
{
"pk": 1,
"fb": {
"new_key": "cc.ee was present!",
"fd_id": "12345",
"fc": ""
},
"fa": "cc.ee"
}

Parsing the data
Using the standard library json module
For string data, use json.loads:
import json
text = '{"one" : "1", "two" : "2", "three" : "3"}'
parsed = json.loads(example)
For data that comes from a file, or other file-like object, use json.load:
import io, json
# create an in-memory file-like object for demonstration purposes.
text = '{"one" : "1", "two" : "2", "three" : "3"}'
stream = io.StringIO(text)
parsed = json.load(stream) # load, not loads
It's easy to remember the distinction: the trailing s of loads stands for "string". (This is, admittedly, probably not in keeping with standard modern naming practice.)
Note that json.load does not accept a file path:
>>> json.load('example.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
Both of these functions provide the same set of additional options for customizing the parsing process. Since 3.6, the options are keyword-only.
For string data, it is also possible to use the JSONDecoder class provided by the library, like so:
import json
text = '{"one" : "1", "two" : "2", "three" : "3"}'
decoder = json.JSONDecoder()
parsed = decoder.decode(text)
The same keyword parameters are available, but now they are passed to the constructor of the JSONDecoder, not the .decode method. The main advantage of the class is that it also provides a .raw_decode method, which will ignore extra data after the end of the JSON:
import json
text_with_junk = '{"one" : "1", "two" : "2", "three" : "3"} ignore this'
decoder = json.JSONDecoder()
# `amount` will count how many characters were parsed.
parsed, amount = decoder.raw_decode(text_with_junk)
Using requests or other implicit support
When data is retrieved from the Internet using the popular third-party requests library, it is not necessary to extract .text (or create any kind of file-like object) from the Response object and parse it separately. Instead, the Response object directly provides a .json method which will do this parsing:
import requests
response = requests.get('https://www.example.com')
parsed = response.json()
This method accepts the same keyword parameters as the standard library json functionality.
Using the results
Parsing by any of the above methods will result, by default, in a perfectly ordinary Python data structure, composed of the perfectly ordinary built-in types dict, list, str, int, float, bool (JSON true and false become Python constants True and False) and NoneType (JSON null becomes the Python constant None).
Working with this result, therefore, works the same way as if the same data had been obtained using any other technique.
Thus, to continue the example from the question:
>>> parsed
{'one': '1', 'two': '2', 'three': '3'}
>>> parsed['two']
'2'
I emphasize this because many people seem to expect that there is something special about the result; there is not. It's just a nested data structure, though dealing with nesting is sometimes difficult to understand.
Consider, for example, a parsed result like result = {'a': [{'b': 'c'}, {'d': 'e'}]}. To get 'e' requires following the appropriate steps one at a time: looking up the a key in the dict gives a list [{'b': 'c'}, {'d': 'e'}]; the second element of that list (index 1) is {'d': 'e'}; and looking up the 'd' key in there gives the 'e' value. Thus, the corresponding code is result['a'][1]['d']: each indexing step is applied in order.
See also How can I extract a single value from a nested data structure (such as from parsing JSON)?.
Sometimes people want to apply more complex selection criteria, iterate over nested lists, filter or transform the data, etc. These are more complex topics that will be dealt with elsewhere.
Common sources of confusion
JSON lookalikes
Before attempting to parse JSON data, it is important to ensure that the data actually is JSON. Check the JSON format specification to verify what is expected. Key points:
The document represents one value (normally a JSON "object", which corresponds to a Python dict, but every other type represented by JSON is permissible). In particular, it does not have a separate entry on each line - that's JSONL.
The data is human-readable after using a standard text encoding (normally UTF-8). Almost all of the text is contained within double quotes, and uses escape sequences where appropriate.
Dealing with embedded data
Consider an example file that contains:
{"one": "{\"two\": \"three\", \"backslash\": \"\\\\\"}"}
The backslashes here are for JSON's escape mechanism.
When parsed with one of the above approaches, we get a result like:
>>> example = input()
{"one": "{\"two\": \"three\", \"backslash\": \"\\\\\"}"}
>>> parsed = json.loads(example)
>>> parsed
{'one': '{"two": "three", "backslash": "\\\\"}'}
Notice that parsed['one'] is a str, not a dict. As it happens, though, that string itself represents "embedded" JSON data.
To replace the embedded data with its parsed result, simply access the data, use the same parsing technique, and proceed from there (e.g. by updating the original result in place):
>>> parsed['one'] = json.loads(parsed['one'])
>>> parsed
{'one': {'two': 'three', 'backslash': '\\'}}
Note that the '\\' part here is the representation of a string containing one actual backslash, not two. This is following the usual Python rules for string escapes, which brings us to...
JSON escaping vs. Python string literal escaping
Sometimes people get confused when trying to test code that involves parsing JSON, and supply input as an incorrect string literal in the Python source code. This especially happens when trying to test code that needs to work with embedded JSON.
The issue is that the JSON format and the string literal format each have separate policies for escaping data. Python will process escapes in the string literal in order to create the string, which then still needs to contain escape sequences used by the JSON format.
In the above example, I used input at the interpreter prompt to show the example data, in order to avoid confusion with escaping. Here is one analogous example using a string literal in the source:
>>> json.loads('{"one": "{\\"two\\": \\"three\\", \\"backslash\\": \\"\\\\\\\\\\"}"}')
{'one': '{"two": "three", "backslash": "\\\\"}'}
To use a double-quoted string literal instead, double-quotes in the string literal also need to be escaped. Thus:
>>> json.loads('{\"one\": \"{\\\"two\\\": \\\"three\\\", \\\"backslash\\\": \\\"\\\\\\\\\\\"}\"}')
{'one': '{"two": "three", "backslash": "\\\\"}'}
Each sequence of \\\" in the input becomes \" in the actual JSON data, which becomes " (embedded within a string) when parsed by the JSON parser. Similarly, \\\\\\\\\\\" (five pairs of backslashes, then an escaped quote) becomes \\\\\" (five backslashes and a quote; equivalently, two pairs of backslashes, then an escaped quote) in the actual JSON data, which becomes \\" (two backslashes and a quote) when parsed by the JSON parser, which becomes \\\\" (two escaped backslashes and a quote) in the string representation of the parsed result (since now, the quote does not need escaping, as Python can use single quotes for the string; but the backslashes still do).
Simple customization
Aside from the strict option, the keyword options available for json.load and json.loads should be callbacks. The parser will call them, passing in portions of the data, and use whatever is returned to create the overall result.
The "parse" hooks are fairly self-explanatory. For example, we can specify to convert floating-point values to decimal.Decimal instances instead of using the native Python float:
>>> import decimal
>>> json.loads('123.4', parse_float=decimal.Decimal)
Decimal('123.4')
or use floats for every value, even if they could be converted to integer instead:
>>> json.loads('123', parse_int=float)
123.0
or refuse to convert JSON's representations of special floating-point values:
>>> def reject_special_floats(value):
... raise ValueError
...
>>> json.loads('Infinity')
inf
>>> json.loads('Infinity', parse_constant=reject_special_floats)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 370, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
File "<stdin>", line 2, in reject_special_floats
ValueError
Customization example using object_hook and object_pairs_hook
object_hook and object_pairs_hook can be used to control what the parser does when given a JSON object, rather than creating a Python dict.
A supplied object_pairs_hook will be called with one argument, which is a list of the key-value pairs that would otherwise be used for the dict. It should return the desired dict or other result:
>>> def process_object_pairs(items):
... return {k: f'processed {v}' for k, v in items}
...
>>> json.loads('{"one": 1, "two": 2}', object_pairs_hook=process_object_pairs)
{'one': 'processed 1', 'two': 'processed 2'}
A supplied object_hook will instead be called with the dict that would otherwise be created, and the result will substitute:
>>> def make_items_list(obj):
... return list(obj.items())
...
>>> json.loads('{"one": 1, "two": 2}', object_hook=make_items_list)
[('one', 1), ('two', 2)]
If both are supplied, the object_hook will be ignored and only the object_items_hook will be used.
Text encoding issues and bytes/unicode confusion
JSON is fundamentally a text format. Input data should be converted from raw bytes to text first, using an appropriate encoding, before the file is parsed.
In 3.x, loading from a bytes object is supported, and will implicitly use UTF-8 encoding:
>>> json.loads('"text"')
'text'
>>> json.loads(b'"text"')
'text'
>>> json.loads('"\xff"') # Unicode code point 255
'ÿ'
>>> json.loads(b'"\xff"') # Not valid UTF-8 encoded data!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/json/__init__.py", line 343, in loads
s = s.decode(detect_encoding(s), 'surrogatepass')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 1: invalid start byte
UTF-8 is generally considered the default for JSON. While the original specification, ECMA-404 does not mandate an encoding (it only describes "JSON text", rather than JSON files or documents), RFC 8259 demands:
JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].
In such a "closed ecosystem" (i.e. for local documents that are encoded differently and will not be shared publicly), explicitly apply the appropriate encoding first:
>>> json.loads(b'"\xff"'.decode('iso-8859-1'))
'ÿ'
Similarly, JSON files should be opened in text mode, not binary mode. If the file uses a different encoding, simply specify that when opening it:
with open('example.json', encoding='iso-8859-1') as f:
print(json.load(f))
In 2.x, strings and byte-sequences were not properly distinguished, which resulted in a lot of problems and confusion particularly when working with JSON.
Actively maintained 2.x codebases (please note that 2.x itself has not been maintained since Jan 1, 2020) should consistently use unicode values to represent text and str values to represent raw data (str is an alias for bytes in 2.x), and accept that the repr of unicode values will have a u prefix (after all, the code should be concerned with what the value actually is, not what it looks like at the REPL).
Historical note: simplejson
simplejson is simply the standard library json module, but maintained and developed externally. It was originally created before JSON support was added to the Python standard library. In 2.6, the simplejson project was incorporated into the standard library as json. Current development maintains compatibility back to 2.5, although there is also an unmaintained, legacy branch that should support as far back as 2.2.
The standard library generally uses quite old versions of the package; for example, my 3.8.10 installation reports
>>> json.__version__
'2.0.9'
whereas the most recent release (as of this writing) is 3.18.1. (The tagged releases in the Github repository only go as far back as 3.8.2; the 2.0.9 release dates to 2009.
I have as yet been unable to find comprehensive documentation of which simplejson versions correspond to which Python releases.

Python - Issues with Unicode String from API Call

I'm using Python to call an API that returns the last name of some soccer players. One of the players has a "ć" in his name.
When I call the endpoint, the name prints out with the unicode attached to it:
>>> last_name = (json.dumps(response["response"][2]["player"]["lastname"]))
>>> print(last_name)
"Mitrovi\u0107"
>>> print(type(last_name))
<class 'str'>
If I were to take copy and paste that output and put it in a variable on its own like so:
>>> print("Mitrovi\u0107")
Mitrović
>>> print(type("Mitrovi\u0107"))
<class 'str'>
Then it prints just fine?
What is wrong with the API endpoint call and the string that comes from it?

Well, you serialise the string with json.dumps() before printing it, that's why you get a different output.
Compare the following:
>>> print("Mitrović")
Mitrović
and
>>> print(json.dumps("Mitrović"))
"Mitrovi\u0107"
The second command adds double quotes to the output and escapes non-ASCII chars, because that's how strings are encoded in JSON. So it's possible that response["response"][2]["player"]["lastname"] contains exactly what you want, but maybe you fooled yourself by wrapping it in json.dumps() before printing.
Note: don't confuse Python string literals and JSON serialisation of strings. They share some common features, but they aren't the same (eg. JSON strings can't be single-quoted), and they serve a different purpose (the first are for writing strings in source code, the second are for encoding data for sending it accross the network).
Another note: You can avoid most of the escaping with ensure_ascii=False in the json.dumps() call:
>>> print(json.dumps("Mitrović", ensure_ascii=False))
"Mitrović"

Count the number of characters in your string & I'll bet you'll notice that the result of json is 13 characters:
"M-i-t-r-o-v-i-\-u-0-1-0-7", or "Mitrovi\\u0107"
When you copy "Mitrovi\u0107" you're coping 8 characters and the '\u0107' is a single unicode character.
That would suggest the endpoint is not sending properly json-escaped unicode, or somewhere in your doc you're reading it as ascii first. Carefully look at exactly what you're receiving.

Python removing nested unicode 'u' sign from string

I have a unicode object which should represent a json but it contains the unicode u in it as part of the string value e.g. u'{u\'name\':u\'my_name\'}'
My goal is to be able to load this into a json object. Just using json.loads fails. I know this happens because of the u inside the string which are not part of an acceptable json format.
I, then, tired sanitizing the string using replace("u\'", "'"), encode('ascii', 'ignore') and other methods without success.
What finally worked was using ast.literal_eval but I'm worried about using it. I found a few sources online claiming its safe. But, I also found other sources claiming it's bad practice and one should avoid it.
Are there other methods I'm missing?

The unicode string is the result of unicode being called on a dictionary.
>>> d = {u'name': u'myname'}
>>> u = unicode(d)
>>> u
u"{u'name': u'myname'}"
If you control the code that's doing this, the best fix is to change it to call json.dumps instead.
>>> json.dumps(d)
'{"name": "myname"}'
If you don't control the creation of this object, you'll need to use ast.literal_eval to create the dictionary, as the unicode string is not valid json.
>>> json.loads(u)
Traceback (most recent call last):
...
ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
>>> ast.literal_eval(u)
{u'name': u'myname'}
The docs confirm that ast.literal_eval is safe:
can be used for safely evaluating strings containing Python values from untrusted sources
You could use eval instead, but as you don't control the creation of the object you cannot be certain that it has not been crafted by a malicious user, to cause damage to your system.

How do I get rid of the "u" from a decoded JSON object?

I have a dictionary of dictionaries in Python:
d = {"a11y_firesafety.html":{"lang:hi": {"div1": "http://a11y.in/a11y/idea/a11y_firesafety.html:hi"}, "lang:kn": {"div1": "http://a11y.in/a11ypi/idea/a11y_firesafety.html:kn}}}
I have this in a JSON file and I encoded it using json.dumps(). Now when I decode it using json.loads() in Python I get a result like this:
temp = {u'a11y_firesafety.html': {u'lang:hi': {u'div1': u'http://a11y.in/a11ypi/idea/a11y_firesafety.html:hi'}, u'lang:kn': {u'div1': u'http://a11y.in/a11ypi/idea/a11y_firesafety.html:kn'}}}
My problem is with the "u" which signifies the Unicode encoding in front of every item in my temp (dictionary of dictionaries). How to get rid of that "u"?

Why do you care about the 'u' characters? They're just a visual indicator; unless you're actually using the result of str(temp) in your code, they have no effect on your code. For example:
>>> test = u"abcd"
>>> test == "abcd"
True
If they do matter for some reason, and you don't care about consequences like not being able to use this code in an international setting, then you could pass in a custom object_hook (see the json docs here) to produce dictionaries with string contents rather than unicode.

You could also use this:
import fileinput
fout = open("out.txt", 'a')
for i in fileinput.input("in.txt"):
str = i.replace("u\"","\"").replace("u\'","\'")
print >> fout,str
The typical json responses from standard websites have these two encoding representations - u' and u"
This snippet gets rid of both of them. It may not be required as this encoding doesn't hinder any logical processing, as mentioned by previous commenter

There is no "unicode" encoding, since unicode is a different data type and I don't really see any reason unicode would be a problem, since you may always convert it to string doing e.g. foo.encode('utf-8').
However, if you really want to have string objects upfront you should probably create your own decoder class and use it while decoding JSON.