This is similar to, Python creating dynamic global variable from list, but I'm still confused.
I get lots of flo data in a semi proprietary format. I've already used Python to strip the data to my needs and save the data into a json file called badactor.json and are saved in the following format:
[saddr as a integer, daddr as a integer, port, date as Julian, time as decimal number]
An arbitrary example [1053464536, 1232644361, 2222, 2014260, 15009]
I want to go through my weekly/monthly flo logs and save everything by Julian date. To start I want to go through the logs and create a list that is named according to the Julian date it happened, i.e, 2014260 and then save it to the same name 2014260.json. I have the following, but it is giving me an error:
#!/usr/bin/python
import sys
import json
import time
from datetime import datetime
import calendar
#these are varibles I've had to use throughout, kinda a boiler plate for now
x=0
templist2 = []
templist3 = []
templist4 = []
templist5 = []
bad = {}
#this is my list of "bad actors", list is in the following format
#[saddr as a integer, daddr as a integer, port, date as Julian, time as decimal number]
#or an arbitrary example [1053464536, 1232644361, 2222, 2014260, 15009]
badactor = 'badactor.json'
with open(badactor, 'r') as f1:
badact = json.load(f1)
f1.close()
for i in badact:
print i[3] #troubleshooting to verify my value is being read in
tmp = str(i[3])
print tmp#again just troubleshooting
tl=[i[0],i[4],i[1],i[2]]
bad[tmp]=bad[tmp]+tl
print bad[tmp]
Trying to create the variable is giving me the following error:
Traceback (most recent call last):
File "savetofiles.py", line 39, in <module>
bad[tmp]=bad[tmp]+tl
KeyError: '2014260'
By the time your code is executed, there is no key "2014260" in the "bad" dict.
Your problem is here:
bad[tmp]=bad[tmp]+tl
You're saying "add t1 to something that doesn't exist."
Instead, you seem to want to do:
bad[tmp]=tl
I suggest you initialize bad to be an empty collections.defaultdict instead of just regular built-in dict. i.e.
import collections
...
bad = collections.defaultdict(list)
That way, initial empty list values will be created for you automatically the first time a date key is encountered and the error you're getting from the bad[tmp]=bad[tmp]+tl statement will go away since it will effectively become bad[tmp]=list()+tl — where the list() call just creates and returns an empty list — the first time a particular date is encountered.
It's also not clear whether you really need the tmp = str(i[3]) conversion because values of any non-mutable type are valid dictionary (or defaultdict) keys, not just strings — assuming i[3] isn't a string already. Regardless, subsequent code would be more readable if you named the result something else, like julian_date = i[3] (or julian_date = str(i[3]) if the conversion really is required).
Related
I'm working on a human stats database for a simulation game and cannot figure out a certain function. The data about each person is stored as a string in a humandict list. Every in-game year the ageup() func should be called, and change each strings data value.
This is the string data format that i use ( the list consists of these values which store data about every human ) :
##ID, age, strengths, smarts
I call the .split() method in order to divide those different numbers in a string to a list, and apply int() to each list item in order to use them in math. The ageup() function should access every humandict item and change the age value by 1. This is the code I currently use ( which doesn't work as intended ):
for unit in humandict:
human = unit.split()
age = int(human[1])
age += 1
replace = (str(human[0])+" "+str(age)+" "+str(human[2])+" "+str(human[3]))
humandict[int(human[0])] = replace
print(humandict)
The code successfully runs once, and the next time the function is called I then get the following error:
File "main.py", line 15, in ageup
human = unit.split()
AttributeError: 'NoneType' object has no attribute 'split'
I simply don't understand where the problem is arising, it can due to wrong ID assignment or something else. But I know for sure that using dictionary here is a better and efficient way to handle data.
So here is how you can implement the same stuff with dictionary:
human_list_of_dict = [{'ID':<int>, 'age':<int>, 'strengths':<str>, 'smarts':<str>}]
above written is a list of dictionary to store data right now it has only 1 dictionary in it but there can be as much as you need. then you simple call it just like a list with few changes.
for unit in human_list_of_dict:
unit['age'] = unit['age']+1
By this way you can save you hassle of converting string to list and vice-versa. Also code is efficient this way(since there is less data manipulation).
I am new to dealing with json files and I am hoping for some help.
Here is a part of the json file (since it would be way too much for me to post it all) that I am dealing with
[{"id":804,"name":{"english":"Naganadel","japanese":"\u30a2\u30fc\u30b4\u30e8\u30f3"},"type":["Poison","Dragon"],"base":{"HP":73,"Attack":73,"Defense":73,"Sp. Attack":127,"Sp. Defense":73,"Speed":121}},{"id":805,"name":{"english":"Stakataka","japanese":"\u30c4\u30f3\u30c7\u30c4\u30f3\u30c7"},"type":["Rock","Steel"],"base":{"HP":61,"Attack":131,"Defense":211,"Sp. Attack":53,"Sp. Defense":101,"Speed":13}},{"id":806,"name":{"english":"Blacephalon","japanese":"\u30ba\u30ac\u30c9\u30fc\u30f3"},"type":["Fire","Ghost"],"base":{"HP":53,"Attack":127,"Defense":53,"Sp. Attack":151,"Sp. Defense":79,"Speed":107}},{"id":807,"name":{"english":"Zeraora","japanese":"\u30bc\u30e9\u30aa\u30e9"},"type":["Electric"],"base":{"HP":88,"Attack":112,"Defense":75,"Sp. Attack":102,"Sp. Defense":80,"Speed":143}},{"id":808,"name":{"english":"Meltan","japanese":"\u30e1\u30eb\u30bf\u30f3"},"type":["Steel"],"base":{"HP":46,"Attack":65,"Defense":65,"Sp. Attack":55,"Sp. Defense":35,"Speed":34}},{"id":809,"name":{"english":"Melmetal","japanese":"\u30e1\u30eb\u30e1\u30bf\u30eb"},"type":["Steel"],"base":{"HP":135,"Attack":143,"Defense":143,"Sp. Attack":80,"Sp. Defense":65,"Speed":34}}]
I am attempting to take the id, name, type, base, hp, attack, defense, and speed of each pokemon. I attached what I currently have which include my attempting to take the id and print it.
When I run this file I get list indices must be integers or slices, not str.
import json
def main():
f = open('pokedex.json')
data = json.load(f)
f.close()
#print data
id_poke = data['_embedded']['id_poke']
id_info = []
for i in id_poke:
id_poke.append(i['id'])
if __name__ == '__main__':
main()
Take a look at the json sample you included in your question: It starts with a [, meaning it is a list, not a dictionary. When you assign this object to the variable data and then try to index into this list with the (string) key _embedded, you get the error you saw.
I don't know how you expected this to work since your json file has neither _embedded nor id_poke as keys, but to get you started, here's how to print out the numeric id and English name of each object; you can take it from there.
for poke in data: # magic iteration over a list: data[0], data[1] etc.
print(poke["id"], poke["name"]["english"])
Declare
id_poke = data['_embedded']['id_poke']
As str()
I'm currently writing code that's supposed to read a file that has the dates and magnitudes of the major earthquakes in recent years and return a dictionary where the keys are the dates the earthquakes took place, and then the values are the magnitudes of the earthquakes that happened on that date.
My code currently looks like this:
def magnitudedictionary():
earth = open("earthquakes.txt", "r")
magdict = {}
for line in earth:
alist = line.split()
magnitude= float(alist[0])
date = alist[1]
if date in magdict:
magdict[date].append(magnitude)
else:
magdict[date] = magnitude
earth.close()
return magdict
But whenever I try to run the code, I always get a Traceback that says:
Traceback (most recent call last):
File "/Users/MargaretJagger/PycharmProjects/Homework 6/Q2.py", line 18, in <module> magnitudedictionary()
File "/Users/MargaretJagger/PycharmProjects/Homework 6/Q2.py", line 10, in magnitudedictionary
magdict[date].append(magnitude)
AttributeError: 'float' object has no attribute 'append'
Process finished with exit code 1
I'm not quite sure what the issue is exactly, but I know that it has something to do with the float and the dictionary values not matching up.
You probably want a defaultDict for this. Then you can avoid the test and just push into the values.
Here's a simple mockup:
from collections import defaultdict
earth = '''7.6 20190801
8.2 20180201
7.1 20190801
6.5 20190801
4.2 20180201'''
magdict = defaultdict(list) # values will default to new lists
for line in earth.split('\n'):
alist = line.split(' ')
magnitude= float(alist[0])
date = alist[1]
magdict[date].append(magnitude) #magdict[date] will default to a list if the key doesn't already exist
print(magdict['20190801'])
>>> [7.6, 7.1, 6.5]
the values are the magnitudes of the earthquakes that happened on that date.
Since you are talking of “magnitudes”, plural, I assume that you want to be able to store multiple values per date. That means that you should also make sure that your dictionary values are actual lists that store multiple values, instead of just a single value.
Compare the following example dictionaries:
{
"2019-04-17": 2.1,
"2019-04-18": 3.5
}
{
"2019-04-17": [1.7, 2.5],
"2019-04-18": [3.2]
}
The first dictionary only maps the date to a single float. So for every date key, you get a single value. The second dictionary maps to a list of floats. Such a list can only contain a single value, or many (it could also contain none).
When you look at your code that sets the values in the dictionary, you can see that you actually built this with multiple values in mind:
if date in magdict:
magdict[date].append(magnitude)
else:
magdict[date] = magnitude
When there’s already the date in the dictionary, then you want to append it. Otherwise you set the date/value pair directly (which adds the key). It’s just that the way you do it, you are setting a single float value (i.e. the first dictionary type above) instead of a list of floats.
So what you need to do instead is create a list of floats here:
if date in magdict:
magdict[date].append(magnitude)
else:
magdict[date] = [magnitude]
The [magnitude] create a one-element list with magnitude as the first value. Since the value in your dictionary is now a list, calls to append() will succeed and correctly add another value to the list.
The error is in the else clause.
It should be magdict[date] = [magnitude] and not magdict[date] = magnitude.
The python dictionary has a very nice method, setdefault, that should help here:
def magnitudedictionary():
earth = open("earthquakes.txt", "r")
magdict = {}
for line in earth:
alist = line.split()
magnitude= float(alist[0])
date = alist[1]
magdict.setdefault(date, []).append(magnitude)
earth.close()
return magdict
Here is a small bit of documentation on the method in question: https://www.tutorialspoint.com/python/dictionary_setdefault.htm
I'm extremely new to python and was having some trouble with removing duplicate values from an attribute of a class (I think this is the correct terminology).
Specifically I want to remove every value that is the same year. I should note that I'm printing only the first four value and searching for the first four values. The data within the attribute is actually in Yearmonthday format (example: 19070101 is the year 1907 on the first on january).
Anyways, here is my code:
import csv
import os
class Datatype:
'Data from the weather station'
def __init__ (self, inputline):
[ self.DATE,
self.PRCP] = inputline.split(',')
filename ='LAWe.txt'
LAWd = open(filename, 'r')
LAWefile = LAWd.read()
LAWd.close()
'Recognize the line endings for MS-DOS, UNIX, and Mac and apply the .split() method to the string wholeFile'
if '\r\n' in LAWefile:
filedat = LAWefile.split('\r\n') # the split method, applied to a string, produces a list
elif '\r' in LAWefile:
filedat = LAWefile.split('\r')
else:
filedat = LAWefile.split('\n')
collection = dict()
date= dict()
for thisline in filedat:
thispcp = Datatype(thisline) # here is where the Datatype object is created (running the __init__ function)
collection[thispcp.DATE] = thispcp # the dictionary will be keyed by the ID attribute
for thisID in collection.keys():
studyPRP = collection[thisID]
if studyPRP.DATE.isdigit():
list(studyPRP.DATE)
if len(date[studyPRP.DATE][0:4]):
pass #if year is seen once, then skip and go to next value in attribute
else:
print studyPRP.DATE[0:4] #print value in this case the year)
date[studyPRP.DATE]=studyPRP.DATE[0:4]
I get a this error:
Traceback (most recent call last):
File "project.py", line 61, in
if len(date[studyPRP.DATE][0:4]):
KeyError: '19770509'
A key error (which means a value isn't in a list? but it is for my data) can be fixed by using a set function (or so I've read), but I have 30,000 pieces of information I'm dealing with and it seems like you have to manually type in that info so that's not an option for me.
Any help at all would be appreciated
Sorry if this is confusing or nonsensical as I'm extremely new to python.
Replace this
if len(date[studyPRP.DATE][0:4])
by this
if len(date[studyPRP.DATE[0:4]]):
Explanation :
In the first line you are selecting the whole date as the key KeyError: '19770509' in the 4 first entry of date
In the correction you send the the first 4 character of the date(the year) in the dictionary
Don't know what exactly you want here. I'll reply based on I can help you on what.
Your error is because you are accessing your year in data before you are adding it.
Also, what you are adding to your collection is like
{
<object>.DATE: <object>
}
I don't know what you need here. Your lower for loop can be written as under:
for thisID in collection:
if thisID.isdigit():
if thisID[0:4] in date and len(date[thisID[0:4]]):
#if year is seen once, then skip and go to next
# value in attribute
pass
else:
print thisID[0:4] #print value in this case the year)
date[thisID[0:4]]=thisID[0:4]
Note your studyPRP.DATE is same as thisID.
I am reading the data from a text file and would like order the data by gpa. I wrote the following code
import string
def main() :
my_list = []
# open the input file for reading
filename = raw_input("Enter name of the grade file: ")
infile = open(filename, 'r')
print infile
# process subsequent lines of the file
for line in infile :
name, hours, qpoints = line.split('\t')
gpa = float(qpoints)/float(hours)
my_tuple = (name, gpa)
print name, gpa
my_list += my_tuple
data = sorted(my_list, key = lambda x: gpa)
if __name__ == '__main__' :
main()
I get an error in main() I find that the 'line.split('\t') does not give the output that I expect. I am not sure what I am doing wrong. I thought, that I would get the first part working before I tried to add the camparison function cmpGpa Thank you for your help.
You have forgotten the parantheses after the function calls in new_id = (st.getName, st.getGpa). Change that to new_id = (st.getName(), st.getGpa()), and I think you'll be better off.
The code concerning st = makeStudent(line) does not seem to be problematic - I would guess that there are some problems with your input file. Try to print each line and see if the output is as expected. If this seems ok, try to print the variables after splitting the line, and see if it's like you want. You'll probably find a tiny bug somewhere.
A couple of other things that you may consider:
Using key to sort your data is easier and faster (see note 8) than cmp, and is the preferred way of sorting lists since Python 2.4.
In your case, sorted(your_list, key=lambda x: x.getGpa()) will sort a list of Student instances based on their gpa.
If the gpa value is static (hours and qpoints don't change), you should consider calulating the gpa in __init__. Why would you want to recalculate it each time you access it? If hours or qpoints may change after initialization, you may be better of making some set_variable methods that updated the gpa after setting the new value.
Get rid of your get_variable_name methods in your Student class unless you have a good reason to implement them. As the code stands now, I don't see the need for them, simply access the variables directly.