I'm testing the following function:
def getDataMapOfFirstLine(line):
datamap = {}
for item in line:
hierarchy = item.split('^')
partialmap = datamap
i=0
for node in hierarchy:
partialmap = partialmap.setdefault(node, i)
i += 1
return datamap
It should create a dictionary out of the first line of a csv-file, that looks like this:
nummer;such;ans;bverb^konum;bverb^namebspr;bverb^bank^iident;
1213;HANS;Hans Dominik;111000222;Hans' account; DE2145432523534232;
1444555;DIRK;Dirk Daniel;13300002;Dirk's account; DE2134634565462352;
As you see these circumflex-signs in each semicolon-separated string are something like a join in SQL. If I execute it, I get this error:
Traceback (most recent call last):
File "./importtool.py", line 173, in <module>
main()
File "./importtool.py", line 38, in main
analyseImportFile(importfile, parser, options)
File "./importtool.py", line 119, in analyseImportFile
datamap = getDataMapOfFirstLine(line)
File "./importtool.py", line 149, in getDataMapOfFirstLine
partialmap = partialmap.setdefault(node, i)
AttributeError: 'int' object has no attribute 'setdefault'
If I replace the i in the setdefault-function by {} I get no error:
{'bverb': {'namebspr': {}, 'konum': {}, 'bank': {'iident': {}}}, 'such': {}, 'ans': {}}
This is nearly, what I want, but instead of the {} I would like to get a column-number.
I just don't get what is wrong. I tried this in interactive mode:
>>> mydict = {'foo': "Hallo", 'bar': 5}
>>> mydict.setdefault("sth", 12)
12
>>> print mydict
{'sth': 12, 'foo': 'Hallo', 'bar': 5}
As you see, this works...
I appreciate every help. Thanks in advance!
Your problem is this line:
partialmap = partialmap.setdefault(node, i)
dict.setdefault returns the thing that was set (or what was already there). In this case, it's an integer so you're setting partialmap to an int. You can probably just not grab the return value (which is what you've done in the interactive terminal BTW):
partialmap.setdefault(node, i)
Related
I was running python code below:
def dict2struct(d):
res = namedtuple("config", d.keys())(*d.values())
return res
cfg = {'fieldx': 'Allan', "fieldy": 45, 'fieldt': {'head': False, 'number': 2}}
res = dict2struct(cfg)
print(res)
res.fieldx = "Jack"
and got the error:
config(fieldx='Allan', fieldy=45, fieldt={'head': False, 'number': 2})
Traceback (most recent call last):
File "*****\Testings\test_x.py", line 97, in <module>
res.fieldx = "Jack"
AttributeError: can't set attribute
I have read the questions below and understand that it was because namedtuples are immutable and the property setter was restricted.
AttributeError: can't set attribute
AttributeError: can't set attribute in python
However, I need to convert a lot of strings from a configurational dictionary's keys to properties of a struct or a class. These strings values are not fixed as they are many.
Is there a way to get around of this AttributeError for my problem?
Steve's solution above worked for me.
pypi.org/project/attributedict
I'm getting the following error when executing the following script:
Error Type: <type 'exceptions.TypeError'>
Error Contents: 'NoneType' object is not iterable
Traceback (most recent call last):
File "addon.py", line 75, in <module>
plugin.run()
File "xbmcswift2/plugin.py", line 332, in run
items = self._dispatch(self.request.path)
File "/plugin.py", line 306, in _dispatch
listitems = view_func(**items)
File "/addon.py", line 42, in all_episodes
items = thisiscriminal.compile_playable_podcast(playable_podcast)
File "/lib/thisiscriminal.py", line 121, in compile_playable_podcast
for podcast in playable_podcast:
TypeError: 'NoneType' object is not iterable
The code in question is as follows, any advice would be greatly appreciated as I have no idea what I'm doing wrong:
def get_playable_podcast(soup):
"""
#param: parsed html page
"""
r = urllib.urlopen('https://thisiscriminal.com/wp-json/criminal/v1/episodes?posts=10000&page=1')
data = json.loads(r.read().decode('utf-8'))
for post in data['posts']:
print post['title']
print post['episodeNumber']
print post['audioSource']
print post['image']['medium']
subjects = []
item = {
'title': post['title'],
'audioSource': post['audioSource'],
'episodeNumber': post['episodeNumber'],
'medium': post['image']['medium']
}
subjects.append(item)
print subjects
def compile_playable_podcast(playable_podcast):
"""
#para: list containing dict of key/values pairs for playable podcasts
"""
items = []
for podcast in playable_podcast:
items.append({
post['title']: podcast['title']['episodeNumber'],
post['audioSource']: podcast['audioSource'],
post['image']['medium']: podcast['medium'],
'is_playable': True,})
return items
I assume your script does something alike to the following,
podcast = get_playable_podcast(soup)
compiled = compile_playable_podcast(podcast)
The problem is that get_playable_podcast has no return statement. In such a case, Python defaults to returning None - which you then pass into compile_playable_podcast. Since None is not iterable, compile_playable_podcast rightfully raises a TypeError.
Now, the solution is of course to return the podcast list you're building in get_playable_podcast, like so,
def get_playable_podcast(soup):
"""
#param: parsed html page
"""
r = urllib.urlopen('https://thisiscriminal.com/wp-json/criminal/v1/episodes?posts=10000&page=1')
data = json.loads(r.read().decode('utf-8'))
subjects = []
for post in data['posts']:
print post['title']
print post['episodeNumber']
print post['audioSource']
print post['image']['medium']
item = {
'title': post['title'],
'audioSource': post['audioSource'],
'episodeNumber': post['episodeNumber'],
'medium': post['image']['medium']
}
subjects.append(item)
print subjects
return subjects
Beside this, it may be worthwhile to carefully check your script for unused parameters and/or duplicate code.
I'm trying to add the JSON output below into a dictionary, to be saved into a SQL database.
{'Parkirisca': [
{
'ID_Parkirisca': 2,
'zasedenost': {
'Cas': '2016-10-08 13:17:00',
'Cas_timestamp': 1475925420,
'ID_ParkiriscaNC': 9,
'P_kratkotrajniki': 350
}
}
]}
I am currently using the following code to add the value to a dictionary:
import scraperwiki
import json
import requests
import datetime
import time
from pprint import pprint
html = requests.get("http://opendata.si/promet/parkirisca/lpt/")
data = json.loads(html.text)
for carpark in data['Parkirisca']:
zas = carpark['zasedenost']
free_spaces = zas.get('P_kratkotrajniki')
last_updated = zas.get('Cas_timestamp')
parking_type = carpark.get('ID_Parkirisca')
if parking_type == "Avtomatizirano":
is_automatic = "Yes"
else:
is_automatic = "No"
scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')
savetodb = {
'scraped': scraped,
'id': carpark.get("ID_Parkirisca"),
'total_spaces': carpark.get("St_mest"),
'free_spaces': free_spaces,
'last_updated': last_updated,
'is_automatic': is_automatic,
'lon': carpark.get("KoordinataX_wgs"),
'lat': carpark.get("KoordinataY_wgs")
}
unique_keys = ['id']
pprint savetodb
However when I run this, it gets stuck at for zas in carpark["zasedenost"] and outputs the following error:
Traceback (most recent call last):
File "./code/scraper", line 17, in <module>
for zas in carpark["zasedenost"]:
KeyError: 'zasedenost'
I've been led to believe that zas is in fact now a string, rather than a dictionary, but I'm new to Python and JSON, so don't know what to search for to get a solution. I've also searched here on Stack Overflow for KeyErrror when key exist questions, but they didn't help, and I believe that this might be due to the fact that's a sub for loop.
Update: Now, when I swapped the double quotes for single quotes, I get the following error:
Traceback (most recent call last):
File "./code/scraper", line 17, in <module>
free_spaces = zas.get('P_kratkotrajniki')
AttributeError: 'unicode' object has no attribute 'get'
I fixed up your code:
Added required imports.
Fixed the pprint savetodb line which isn't valid Python.
Didn't try to iterate over carpark['zasedenost'].
I then added another pprint statement in the for loop to see what's in carpark when the KeyError occurs. From there, the error is clear. (Not all the elements in the array in your JSON contain the 'zasedenost' key.)
Here's the code I used:
import datetime
import json
from pprint import pprint
import time
import requests
html = requests.get("http://opendata.si/promet/parkirisca/lpt/")
data = json.loads(html.text)
for carpark in data['Parkirisca']:
pprint(carpark)
zas = carpark['zasedenost']
free_spaces = zas.get('P_kratkotrajniki')
last_updated = zas.get('Cas_timestamp')
parking_type = carpark.get('ID_Parkirisca')
if parking_type == "Avtomatizirano":
is_automatic = "Yes"
else:
is_automatic = "No"
scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')
savetodb = {
'scraped': scraped,
'id': carpark.get("ID_Parkirisca"),
'total_spaces': carpark.get("St_mest"),
'free_spaces': free_spaces,
'last_updated': last_updated,
'is_automatic': is_automatic,
'lon': carpark.get("KoordinataX_wgs"),
'lat': carpark.get("KoordinataY_wgs")
}
unique_keys = ['id']
pprint(savetodb)
And here's the output on the iteration where the KeyError occurs:
{u'A_St_Mest': None,
u'Cena_dan_Eur': None,
u'Cena_mesecna_Eur': None,
u'Cena_splosno': None,
u'Cena_ura_Eur': None,
u'ID_Parkirisca': 7,
u'ID_ParkiriscaNC': 72,
u'Ime': u'P+R Studenec',
u'Invalidi_St_mest': 9,
u'KoordinataX': 466947,
u'KoordinataX_wgs': 14.567929171694901,
u'KoordinataY': 101247,
u'KoordinataY_wgs': 46.05457609543313,
u'Opis': u'2,40 \u20ac /dan',
u'St_mest': 187,
u'Tip_parkirisca': None,
u'U_delovnik': u'24 ur (ponedeljek - petek)',
u'U_sobota': None,
u'U_splosno': None,
u'Upravljalec': u'JP LPT d.o.o.'}
Traceback (most recent call last):
File "test.py", line 14, in <module>
zas = carpark['zasedenost']
KeyError: 'zasedenost'
As you can see, the error is quite accurate. There's no key 'zasedenost' in the dictionary. If you look through your JSON, you'll see that's true for a number of the elements in that array.
I'd suggest a fix, but I don't know what you want to do in the case where this dictionary key is absent. Perhaps you want something like this:
zas = carpark.get('zasedenost')
if zas is not None:
free_spaces = zas.get('P_kratkotrajniki')
last_updated = zas.get('Cas_timestamp')
else:
free_spaces = None
last_updated = None
Let's make it a bit easier.
Code:
level3 = {'a':'aa'}
level2 = {'b':level3, 'd':level3}
level1 = {'j':level2, 'k':level2}
def print_rec(node = None):
if node is None:
node = level1
if node == 'aa':
return
for key, successor in node.items():
print(key,":",node.get(key))
print_rec(successor)
print_rec()
Outputs:
k : {'d': {'a': 'aa'}, 'b': {'a': 'aa'}}
d : {'a': 'aa'}
a : aa
b : None
Traceback (most recent call last):
File "test.py", line 13, in <module>
print_rec(level1)
File "test.py", line 11, in print_rec
print_rec(node)
File "test.py", line 11, in print_rec
print_rec(node)
File "test.py", line 8, in print_rec
for key in node:
TypeError: 'NoneType' object is not iterable
I think node = node.get(key) will be executed only if the key is in node. So why the new node will get a NoneType? Anyone can help?
In the for loop, it seems like you're using the name node for two different things:
for key in node:
print(key,":",node.get(key))
node = node.get(key)
print_rec(node)
When in the first iteration, you change the value of node. In the second iteration, when you do node.get(key), you're using the new node, but you want to be using the original node.
This should help:
for key in node:
print(key,":",node.get(key))
successor = node.get(key)
print_rec(successor)
It can be written even more concisely like this:
for key, successor in node.items():
print(key,":",successor)
print_rec(successor)
How to validate words divided by a comma by FormEncode ?
Something like this:
"foo1, foo2, foo3" -> ["foo1", "foo2", "foo3"]
You'll probably need a custom validator. Here's a quick example:
import formencode
class CommaSepList(formencode.validators.FancyValidator):
def _to_python(self, value, state):
return value.split(",")
def validate_python(self, value, state):
for elem in value:
if elem == "":
raise formencode.Invalid("an element of the list is empty", value, state)
>>> CommaSepList.to_python("1,2,3")
['1', '2', '3']
>>> CommaSepList.to_python("1,,")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.5/site-packages/FormEncode-1.2.3dev-py2.5.egg/formencode/api.py", line 416, in to_python
vp(value, state)
File "myValidator.py", line 17, in validate_python
raise formencode.Invalid("an element of the list is empty", value, state)
Of course, you'll want to add validation specific to your use case.
Assuming each word is separated by a comma and a space (', '):
>>> x = "foo1, bar2, foo3"
>>> x.split(', ')
['foo1', 'bar2', 'foo3']
And then pass that list on to FormEncode and have it do whatever you need it to do.