Extract specific value from JSON with Python - python
I am trying to get all the websites from this json file
Unfortunately, when I use this code:
import requests
response = requests.get("https://github.com/solana-labs/token-list/blob/main/src/tokens/solana.tokenlist.json")
output = response.json()
# Extract specific node content.
print(output['website'])
I get following error:
Traceback (most recent call last):
File "/Users/dusandev/Desktop/StreamFlowWebTests/extract.py", line 5, in <module>
output = response.json()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py",
line 346, in loads
return _default_decoder.decode(s)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py",
line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py",
line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)
Any help is appreciated. Thank you in advance
Use raw data to get raw json and then iterate over 'tokens' attr of the response object:
import requests
response = requests.get(
"https://raw.githubusercontent.com/solana-labs/token-list/main/src/tokens/solana.tokenlist.json")
output = response.json()
for i in output['tokens']:
if i.get('extensions'):
print(i.get('extensions').get('website'))
The file https://github.com/solana-labs/token-list/blob/main/src/tokens/solana.tokenlist.json is not a json. Use https://raw.githubusercontent.com/solana-labs/token-list/main/src/tokens/solana.tokenlist.json instead.
If you visit the url https://github.com/solana-labs/token-list/blob/main/src/tokens/solana.tokenlist.json in a browser, you'll get a fully rendered web page. In order to get just JSON you need to use the "view raw" link. That winds up being
https://raw.githubusercontent.com/solana-labs/token-list/main/src/tokens/solana.tokenlist.json
You will then have several thousand elements in the array attached to the "tokens" key in the response dictionary. To get the website element you'll need to iterate through the list and look at the "extensions"
>>> output["tokens"][0]
{'chainId': 101, 'address': 'CbNYA9n3927uXUukee2Hf4tm3xxkffJPPZvGazc2EAH1', 'symbol': 'agEUR', 'name': 'agEUR (Wormhole)', 'decimals': 8, 'logoURI': 'https://raw.githubusercontent.com/solana-labs/token-list/main/assets/mainnet/CbNYA9n3927uXUukee2Hf4tm3xxkffJPPZvGazc2EAH1/logo.png', 'tags': ['ethereum', 'wrapped', 'wormhole'], 'extensions': {'address': '0x1a7e4e63778B4f12a199C062f3eFdD288afCBce8', 'assetContract': 'https://etherscan.io/address/0x1a7e4e63778B4f12a199C062f3eFdD288afCBce8', 'bridgeContract': 'https://etherscan.io/address/0x3ee18B2214AFF97000D974cf647E7C347E8fa585', 'coingeckoId': 'ageur', 'description': 'Angle is the first decentralized, capital efficient and over-collateralized stablecoin protocol', 'discord': 'https://discord.gg/z3kCpTaKMh', 'twitter': 'https://twitter.com/AngleProtocol', 'website': 'https://www.angle.money'}}
>>> output["tokens"][0]["extensions"]["website"]
'https://www.angle.money'
This error usually means that the output can not be parsed as a json.
you have 2 options:
use "https://raw.githubusercontent.com/solana-labs/token-list/main/src/tokens/solana.tokenlist.json" instead-
import requests
response = requests.get("https://raw.githubusercontent.com/solana-labs/token-list/main/src/tokens/solana.tokenlist.json")
output = response.json()
first_website = output["tokens"][0]["extensions"]["website"]
#all websites:
for token in output['tokens']:
if extensions := token.get('extensions'): print(extensions.get('website'))
#output:
'https://www.angle.money'
you can parse it using BeautifulSoup - https://www.dataquest.io/blog/web-scraping-python-using-beautiful-soup/
Related
How can I fix this problem serializing JSON?
I want to post input data with python to a JSON document database. But this error appears. I think it's a problem of serializing but don't think how to fix it:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\33769\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "C:\Users\33769\AppData\Local\Programs\Python\Python310\lib\json\encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Users\33769\AppData\Local\Programs\Python\Python310\lib\json\encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "C:\Users\33769\AppData\Local\Programs\Python\Python310\lib\json\encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type set is not JSON serializable
>>> res = requests.post(url, headers=headers, data=data)
>>> print(res.text)
{"error":"bad_request","reason":"invalid UTF-8 JSON"}
Here is the code I used:
>>> import requests
>>> import json
>>> url = 'http://admin:pass#localhost:5984/db_reviewin'
>>> data = {'key':'value'}
>>> headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
>>> a = input("ton_nom:")
ton_nom:ayoub
>>> b = input("ton_age")
ton_age16
>>> c = input("gender:")
gender:M
>>> e_mail = input("ton e_mail:")
ton e_mail:ayoub_semsar#yahoo.com
>>> d = input("country:")
country:france
>>> data = {"full_name":{a}, "age":{b}, "gender":{c}, "e_mail":{e_mail}, "country":{d}}
>>> res = requests.post(url, headers=headers, data=json.dumps(data))
You can't use a set, use a list and make it like this
data = {"full_name":[a], "age":[b], "gender":[c], "e_mail":[e_mail], "country":[d]}
You are trying to insert a string instead of object (or vice versa).
For example;
This is a valid JSON:
{"full_name": "ayoub"}
Or this is another valid JSON:
{"full_name": {"name": "ayoub"}}
However, this is the JSON that returns from your code (let's include just the first column):
{"full_name": {"ayoub"}}
You need to remove curly brackets from inside your dictionary or you should convert them to a JSON list which can contain multiple string inside it:
data = {"full_name": a, "age": b, "gender": c, "e_mail": e_mail, "country": d}
data = {"full_name":[a], "age":[b], "gender":[c], "e_mail":[e_mail], "country":[d]}
JSONDecodeError Issue? python
Can someone help me fix this, I don't know why I am getting this error.
I am trying to use a python program someone made, I tried to mess around with it but I could not figure out the issue.
Error:
PS D:\Python> python .\quizlet.py
Traceback (most recent call last):
File "D:\Python\quizlet.py", line 69, in <module>
q = QuizletParser(website)
File "D:\Python\quizlet.py", line 17, in QuizletParser
data = json.loads(BeautifulSoup(session.get(link).content, features="lxml").find_all('script')[-6].string[44:-152])
File "C:\Users\john\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\john\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 14 (char 13)
I am trying to use the code I found here from a bit ago: https://github.com/daijro/python-quizlet
Source:
from requests_html import HTMLSession
from box import Box
import box
import json
from bs4 import BeautifulSoup
from difflib import SequenceMatcher
def FindFlashcard(flashcards: box.box_list.BoxList, match: str):
similar = lambda a, b: SequenceMatcher(None, a, b).ratio()
data = max(list(zip([similar(match, x.term) for x in flashcards], [x for x in range(len(flashcards))])))
flashcard = flashcards[data[1]]
flashcard.update({'similarity': data[0]})
return flashcard
def QuizletParser(link: str):
session = HTMLSession()
data = json.loads(BeautifulSoup(session.get(link).content, features="lxml").find_all('script')[-6].string[44:-152])
flashcards = []
for i in list(data['termIdToTermsMap'].values()):
i = {
'index': i['rank'],
'id': i['id'],
'term': i['word'],
'definition': i['definition'],
'setId': i['setId'],
'image': i['_imageUrl'],
'termTts': 'https://quizlet.com'+i['_wordTtsUrl'],
'termTtsSlow': 'https://quizlet.com'+i['_wordSlowTtsUrl'],
'definitionTts': 'https://quizlet.com'+i['_definitionTtsUrl'],
'definitionTtsSlow': 'https://quizlet.com'+i['_definitionSlowTtsUrl'],
'lastModified': i['lastModified'],
}
flashcards.append(i)
output = {
'title': data['set']['title'],
'flashcards': flashcards,
'author': {
'name': data['creator']['username'],
'id': data['creator']['id'],
'timestamp': data['creator']['timestamp'],
'lastModified': data['creator']['lastModified'],
'image': data['creator']['_imageUrl'],
'timezone': data['creator']['timeZone'],
'isAdmin': data['creator']['isAdmin'],
},
'id': data['set']['id'],
'link': data['set']['_webUrl'],
'thumbnail': data['set']['_thumbnailUrl'],
'timestamp': data['set']['timestamp'],
'lastModified': data['set']['lastModified'],
'publishedTimestamp': data['set']['publishedTimestamp'],
'authorsId': data['set']['creatorId'],
'termLanguage': data['set']['wordLang'],
'definitionLanguage': data['set']['defLang'],
'description': data['set']['description'],
'numTerms': data['set']['numTerms'],
'hasImages': data['set']['hasImages'],
'hasUploadedImage': data['hasUploadedImage'],
'hasDiagrams': data['set']['hasDiagrams'],
'hasImages': data['set']['hasImages'],
}
return Box(output)
website = 'https://quizlet.com/475389316/python-web-scraping-flash-cards/'
text = 'Two popular parsers'
q = QuizletParser(website)
flashcard = FindFlashcard(q.flashcards, match=text) # finds the flashcard most similar to the input
print(flashcard.term + " " + flashcard.definition) # calculates how similar the identified flashcard is to the input
Hard to give a solution without looking at the data.
A few tips for debugging JSON errors:
Check the input data to the JSONDecoder. You might be adding a comma to the last key-value pair of the input dictionary (which is very common).
Check the data type. If your input data came from an external source check the data first.
I would suggest doing a print of this and pasting it here if possible.
input_data = BeautifulSoup(session.get(link).content, features="lxml").find_all('script')[-6].string[44:-152]
print(input_data)
can't pickle dictionary in django
I have a simple dictionary that i am trying to save to cache and looks like it django is trying to pickle:
podcasts = []
for i in items:
s = re.sub('[\s+]', '', str(i))
s2 = re.findall(r'<link/>(.*?)<itunes',s)[0]
item_obj = {}
item_obj['title'] = title
item_obj['url'] = s2
item_obj['created_at'] = created_at
item_obj['duration'] = duration
podcasts.append(item_obj)
This has a very simple format that outputs:
[{'title': "Podcast1", 'url': 'https://example.com\\n', 'created_at': 'Thu, 28 Dec 2017', 'duration': '00:30:34'}]
I am running this from a custom management command like this:
python3 manage.py podcast_job
I attempt to save to cache:
podcasts = get_podcasts()
print(podcasts)
cache.set('podcasts', podcasts)
I get the error:
File "podcast_job.py", line 13, in handle
cache.set('podcasts', podcasts)
File "python3.6/site-packages/django_redis/cache.py", line 33, in _decorator
return method(self, *args, **kwargs)
File "python3.6/site-packages/django_redis/cache.py", line 68, in set
return self.client.set(*args, **kwargs)
File "python3.6/site-packages/django_redis/client/default.py", line 109, in set
nvalue = self.encode(value)
File "python3.6/site-packages/django_redis/client/default.py", line 329, in encode
value = self._serializer.dumps(value)
File "python3.6/site-packages/django_redis/serializers/pickle.py", line 33, in dumps
return pickle.dumps(value, self._pickle_version)
RecursionError: maximum recursion depth exceeded while calling a Python object
If I try to save with a string I get no error and it saves fine:
cache.set('podcasts', str(podcasts))
How can I save the list of dictionaries and not get the error above?
If you are using datetime objects for created_at and duration, make sure you render them to strings.
Pickle does not deal well with functions. Check out this answer for some insight: https://stackoverflow.com/a/1253813/4225229 You could serialize the result of the function (try json.dumps()) and cache that.
I converted the dictionary with json as Jacob suggested like this:
cache.set('podcasts', json.dumps(podcasts))
JSON Parsing of ADSB Data
Coding this using Wing IDE. When running my code, it opens up decoder.py and provides the following error: File "C:\Python27\Lib\json\decoder.py", line 367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 341 - line 29 column 1 (char 340 - 15199)
I've looked around google and SO, but haven't been able to match a root cause to my specific example. Any ideas?
The end goal is to parse ADS-B data from adsbexchange.
For the record, I also replaced the final print line with something more simple like print parsed_json but the results were the same.
import json
json_string = """{"Id":3429058,"Rcvr":11182,"HasSig":true,"Sig":6,"Icao":"3452C2","Bad":false,"FSeen":"\/Date(1512567030924)\/","CMsgs":2,"AltT":0,"Tisb":false,"TrkH":false,"Sqk":"","VsiT":0,"WTC":0,"Species":0,"EngType":0,"EngMount":0,"Mil":false,"Cou":"Spain","HasPic":false,"Interested":false,"FlightsCount":0,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1},
{"Id":4509293,"Rcvr":13019,"HasSig":false,"Icao":"44CE6D","Bad":false,"Reg":"OO-SSM","FSeen":"\/Date(1512567029549)\/","TSecs":1,"CMsgs":1,"Alt":23700,"GAlt":23700,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A319","Mdl":"Airbus A319 112","Man":"Airbus","CNum":"1388","Op":"Brussels Airlines","OpIcao":"BEL","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Belgium","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2000"},
{"Id":4960709,"Rcvr":11071,"HasSig":true,"Sig":15,"Icao":"4BB1C5","Bad":false,"Reg":"TC-LNE","FSeen":"\/Date(1512567029189)\/","TSecs":1,"CMsgs":4,"Alt":41000,"GAlt":41000,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A333","Mdl":"Airbus A330 303","Man":"Airbus","CNum":"1706","Op":"Turkish Airlines","OpIcao":"THY","Sqk":"","VsiT":0,"WTC":3,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Turkey","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2015"},
{"Id":11260420,"Rcvr":11012,"HasSig":true,"Sig":30,"Icao":"ABD204","Bad":false,"Reg":"N8606C","FSeen":"\/Date(1512567029096)\/","TSecs":2,"CMsgs":1,"AltT":0,"Tisb":false,"TrkH":false,"Type":"B738","Mdl":" BOEING 737-8H4","Man":"Boeing","CNum":"35964","Op":"Southwest Airlines","OpIcao":"SWA","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"United States","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1},
"""
parsed_json = json.loads(json_string)
print(parsed_json['Type'])
Your json string is invalid.
import json
json_string = """[{"Id":3429058,"Rcvr":11182,"HasSig":true,"Sig":6,"Icao":"3452C2","Bad":false,"FSeen":"\/Date(1512567030924)\/","CMsgs":2,"AltT":0,"Tisb":false,"TrkH":false,"Sqk":"","VsiT":0,"WTC":0,"Species":0,"EngType":0,"EngMount":0,"Mil":false,"Cou":"Spain","HasPic":false,"Interested":false,"FlightsCount":0,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1},
{"Id":4509293,"Rcvr":13019,"HasSig":false,"Icao":"44CE6D","Bad":false,"Reg":"OO-SSM","FSeen":"\/Date(1512567029549)\/","TSecs":1,"CMsgs":1,"Alt":23700,"GAlt":23700,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A319","Mdl":"Airbus A319 112","Man":"Airbus","CNum":"1388","Op":"Brussels Airlines","OpIcao":"BEL","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Belgium","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2000"},
{"Id":4960709,"Rcvr":11071,"HasSig":true,"Sig":15,"Icao":"4BB1C5","Bad":false,"Reg":"TC-LNE","FSeen":"\/Date(1512567029189)\/","TSecs":1,"CMsgs":4,"Alt":41000,"GAlt":41000,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A333","Mdl":"Airbus A330 303","Man":"Airbus","CNum":"1706","Op":"Turkish Airlines","OpIcao":"THY","Sqk":"","VsiT":0,"WTC":3,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Turkey","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2015"},
{"Id":11260420,"Rcvr":11012,"HasSig":true,"Sig":30,"Icao":"ABD204","Bad":false,"Reg":"N8606C","FSeen":"\/Date(1512567029096)\/","TSecs":2,"CMsgs":1,"AltT":0,"Tisb":false,"TrkH":false,"Type":"B738","Mdl":" BOEING 737-8H4","Man":"Boeing","CNum":"35964","Op":"Southwest Airlines","OpIcao":"SWA","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"United States","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1}]
"""
parsed_json = json.loads(json_string)
for i in parse_json:
if "Type" in i:
print (i['Type'])
Your json string is wrong. It contains an array of objects which should be within the square [] brackets.
Change your json string to
json_string = """[{"Id":3429058,"Rcvr":11182,"HasSig":true,"Sig":6,"Icao":"3452C2","Bad":false,"FSeen":"\/Date(1512567030924)\/","CMsgs":2,"AltT":0,"Tisb":false,"TrkH":false,"Sqk":"","VsiT":0,"WTC":0,"Species":0,"EngType":0,"EngMount":0,"Mil":false,"Cou":"Spain","HasPic":false,"Interested":false,"FlightsCount":0,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1},
{"Id":4509293,"Rcvr":13019,"HasSig":false,"Icao":"44CE6D","Bad":false,"Reg":"OO-SSM","FSeen":"\/Date(1512567029549)\/","TSecs":1,"CMsgs":1,"Alt":23700,"GAlt":23700,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A319","Mdl":"Airbus A319 112","Man":"Airbus","CNum":"1388","Op":"Brussels Airlines","OpIcao":"BEL","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Belgium","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2000"},
{"Id":4960709,"Rcvr":11071,"HasSig":true,"Sig":15,"Icao":"4BB1C5","Bad":false,"Reg":"TC-LNE","FSeen":"\/Date(1512567029189)\/","TSecs":1,"CMsgs":4,"Alt":41000,"GAlt":41000,"AltT":0,"Tisb":false,"TrkH":false,"Type":"A333","Mdl":"Airbus A330 303","Man":"Airbus","CNum":"1706","Op":"Turkish Airlines","OpIcao":"THY","Sqk":"","VsiT":0,"WTC":3,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"Turkey","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1,"Year":"2015"},
{"Id":11260420,"Rcvr":11012,"HasSig":true,"Sig":30,"Icao":"ABD204","Bad":false,"Reg":"N8606C","FSeen":"\/Date(1512567029096)\/","TSecs":2,"CMsgs":1,"AltT":0,"Tisb":false,"TrkH":false,"Type":"B738","Mdl":" BOEING 737-8H4","Man":"Boeing","CNum":"35964","Op":"Southwest Airlines","OpIcao":"SWA","Sqk":"","VsiT":0,"WTC":2,"Species":1,"Engines":"2","EngType":3,"EngMount":0,"Mil":false,"Cou":"United States","HasPic":false,"Interested":false,"FlightsCount":0,"Gnd":false,"SpdTyp":0,"CallSus":false,"TT":"a","Trt":1}]
"""
Adding Item data to a DynamoDB table using boto does not work
I have been trying to add items to a DynamoDB table using boto, but somehow it doesn't seem to work. I tried using users.Item() and users.put_item but nothing worked. Below is the script that I have in use.
import boto.dynamodb2
import boto.dynamodb2.items
import json
from boto.dynamodb2.fields import HashKey, RangeKey, GlobalAllIndex
from boto.dynamodb2.layer1 import DynamoDBConnection
from boto.dynamodb2.table import Table
from boto.dynamodb2.items import Item
from boto.dynamodb2.types import NUMBER
region = "us-east-1"
con = boto.dynamodb2.connect_to_region(region)
gettables = con.list_tables()
mytable = "my_table"
if mytable not in gettables['TableNames']:
print "The table *%s* is not in the list of tables created. A new table will be created." % req_table
Table.create(req_table,
schema = [HashKey('username'),
RangeKey('ID', data_type = NUMBER)],
throughput = {'read': 1, 'write': 1})
else:
print "The table *%s* exists." % req_table
con2table = Table(req_table,connection=con)
con2table.put_item(data={'username': 'abcd',
'ID': '001',
'logins':'10',
'timeouts':'20'
'daysabsent': '30'
})
I tried this, the table gets created and it is fine. But when I try to put in the items, I get the following error message.
Traceback (most recent call last):
File "/home/ec2-user/DynamoDB_script.py", line 29, in <module>
'daysabsent':'30'
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 821, in put_item
return item.save(overwrite=overwrite)
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/items.py", line 455, in save
returned = self.table._put_item(final_data, expects=expects)
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/table.py", line 835, in _put_item
self.connection.put_item(self.table_name, item_data, **kwargs)
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 1510, in put_item
body=json.dumps(params))
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 2842, in make_request
retry_handler=self._retry_handler)
File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 954, in _mexe
status = retry_handler(response, i, next_sleep)
File "/usr/lib/python2.7/dist-packages/boto/dynamodb2/layer1.py", line 2882, in _retry_handler
response.status, response.reason, data)
boto.dynamodb2.exceptions.ValidationException: ValidationException: 400 Bad Request
{u'message': u'One or more parameter values were invalid: Type mismatch for key version expected: N actual: S', u'__type': u'com.amazon.coral.validate#ValidationException'}
Thank you.
From the error message you are getting, it sounds like you are trying to send string values for an attribute that is defined as numeric in DynamoDB. The specific issue looks to be related to your Range Key ID which is defined as a numeric value N but you are sending it a string value '001'.
Looks like of of the values you are trying to load has empty value. I got the same error when I was trying to load this. I got exception when partner_name property was a empty string. try: item_old = self.table.get_item(hash_key=term) except BotoClientError as ex: # if partner alias does not exist then create a new entry! if ex.message == "Key does not exist.": item_old = self.table.new_item(term) else: raise ex item_old['partner_code'] = partner_code item_old['partner_name'] = partner_name item_old.put()