My goal: Sort a list of Products (dict) first by Price, then by Name.
My problem: Str values with numbers in them aren't sorted properly (AKA "Human sorting" or "Natural Sorting").
I found this function from a similar question:
Python sorting list of dictionaries by multiple keys
def multikeysort(items, columns):
from operator import itemgetter
comparers = [((itemgetter(col[1:].strip()), -1) if col.startswith('-') else
(itemgetter(col.strip()), 1)) for col in columns]
def comparer(left, right):
for fn, mult in comparers:
result = cmp(fn(left), fn(right))
if result:
return mult * result
else:
return 0
return sorted(items, cmp=comparer)
The problem is that my Prices are str type, like this:
products = [
{'name': 'Product 200', 'price': '3000.00'},
{'name': 'Product 4', 'price': '100.10'},
{'name': 'Product 15', 'price': '20.00'},
{'name': 'Product 1', 'price': '5.05'},
{'name': 'Product 2', 'price': '4.99'},
]
So they're getting sorted alphabetically, like this:
'100.10'
'20.10'
'3000.00'
'4.99'
'5.05'
Similarly, when I sort by name, I get this:
'Product 1'
'Product 15'
'Product 2'
'Product 200'
'Product 4'
The names should be listed in "human" order (1,2,15 instead of 1,15,2). Is it possible to fix this? I'm pretty new to python, so maybe I'm missing something vital. Thanks.
EDIT
More Info: I'm sending the list of products to a Django template, which requires the numbers to be properly formatted. If I float the prices and then un-float them, I have to iterate through the list of products twice, which seems like overkill.
Your sort function is overkill. Try this simple approach:
from pprint import pprint
products = [
{'name': 'Product 200', 'price': '3000.00'},
{'name': 'Product 4', 'price': '100.10'},
{'name': 'Product 15', 'price': '20.00'},
{'name': 'Product 1', 'price': '5.05'},
{'name': 'Product 2', 'price': '4.99'},
]
sorted_products = sorted(products, key=lambda x: (float(x['price']), x['name']))
pprint(sorted_products)
Result:
[{'name': 'Product 2', 'price': '4.99'},
{'name': 'Product 1', 'price': '5.05'},
{'name': 'Product 15', 'price': '20.00'},
{'name': 'Product 4', 'price': '100.10'},
{'name': 'Product 200', 'price': '3000.00'}]
The essence of my solution is to have the key function return a tuple of the sort conditions. Tuples always compare lexicographically, so the first item is the primary sort, the second is the secondary sort, and so on.
I think your best bet is to parse the prices as floats (so you can sort them):
float("1.00")
# output: 1.0
Then output them with two decimal places:
"{:.2f}".format(1.0)
# output: "1.00"
Try typecasting them to floats in the question and when you need to print 2 decimal places, you can easily format the output like so:
float_num = float("110.10")
print "{0:.2f}".format(float_num) # prints 110.10
To break ties should there be any sorting the strings using the integer value from the product, you can return a tuple:
products = [
{'name': 'Product 200', 'price': '2.99'},
{'name': 'Product 4', 'price': '4.99'},
{'name': 'Product 15', 'price': '4.99'},
{'name': 'Product 1', 'price': '9.99'},
{'name': 'Product 2', 'price': '4.99'},
]
def key(x):
p, i = x["name"].rsplit(None, 1)
return float(x["price"]), p, int(i)
sorted_products = sorted(products, key=key)
Which would give you:
[{'name': 'Product 200', 'price': '2.99'},
{'name': 'Product 2', 'price': '4.99'},
{'name': 'Product 4', 'price': '4.99'},
{'name': 'Product 15', 'price': '4.99'},
{'name': 'Product 1', 'price': '9.99'}]
As opposed to:
[{'name': 'Product 200', 'price': '2.99'},
{'name': 'Product 15', 'price': '4.99'},
{'name': 'Product 2', 'price': '4.99'},
{'name': 'Product 4', 'price': '4.99'},
{'name': 'Product 1', 'price': '9.99'}]
using just float(x['price']), x['name']
Related
I need to parse json from a partial string I get back from a web service. I have the following snippet of code which is working fine but is extremely ugly. Is there a better or cleaner way to do this?
x = '"1":{"name":"item one","code":"1"},"2":{"name":"item two","code":"2"},"3":{"name":"item three","code":"3"}'
split = x.split('},')
index = 0
for s in split:
split[index] = '{' + s + '}}'
index += 1
joined = ','.join(split)
joined = '[' + joined[:-1] + ']'
j = json.loads(joined)
print(j)
Here is the result:
[{'1': {'name': 'item one', 'code': '1'}},
{'2': {'name': 'item two', 'code': '2'}},
{'3': {'name': 'item three', 'code': '3'}}]
You can use the following snippet:
>>> [dict([t]) for t in json.loads(f"{{{x}}}").items()]
[{'1': {'name': 'item one', 'code': '1'}},
{'2': {'name': 'item two', 'code': '2'}},
{'3': {'name': 'item three', 'code': '3'}}]
You can fix the inconsistency by hand (add the missing braces) and use json module to parse:
data = json.loads('{' + x + '}')
Then you can convert the parsed data to the desired representation:
[{item[0]: item[1]} for item in data.items()]
#[{'1': {'name': 'item one', 'code': '1'}},
# {'2': {'name': 'item two', 'code': '2'}},
# {'3': {'name': 'item three', 'code': '3'}}]
Otherwise, you will end up implementing your own JSON parser, which is not trivial.
I have the following code:
myd = mergedCollection.find(myquery).sort("Price")
print("MY D: "+str(myd))
shoes = myd[0]["theAssociatedShoes"]
print("Shoes: "+ str(shoes))
With the output:
MY D: <pymongo.cursor.Cursor object at 0x05859538>
Shoes: [{'Title': 'Nike Cosmic Unity "Amalgam"', 'Price': 160, 'Currency': 'USD', 'Picture': 'https://static.nike.com/a/images/t_default/3bca4f51-f2e4-4948-a665-27e03eea4ddd/cosmic-unity-amalgam-basketball-shoe-nDHKr4.png', 'Link': 'nike.com/t/cosmic-unity-amalgam-basketball-shoe-nDHKr4/DA6725-500', 'Brand': 'nike'}, {'Title': 'Ultraboost 21 Shoes', 'Price': 180, 'Currency': ' USD', 'Picture': 'https://assets.adidas.com/images/w_280,h_280,f_auto,q_auto:sensitive/3728ddf5b7dc4a2ca3e3ac7d0106c5a1_9366/ultraboost-21-shoes.jpg', 'Link': 'adidas.com/us/ultraboost-21-shoes/FY0350.html', 'Brand': 'adidas'}, {'Title': 'Fresh', 'Price': 129, 'Currency': ' USD', 'Picture': 'https://nb.scene7.com/is/image/NB/m880f11_nb_02_i?$pdpflexf2$&wid=440&hei=440', 'Link': 'newbalance.com/pd/fresh-foam-880v11/M880V11-33418.html', 'Brand': 'newbalance'}, {'Title': 'Jordan Delta Breathe', 'Price': 130, 'Currency': 'USD', 'Picture': 'https://static.nike.com/a/images/t_default/b54eef6b-6dd5-4c07-9b09-901ab9d7b01a/jordan-delta-breathe-mens-shoe-2ggX3h.png', 'Link': 'nike.com/t/jordan-delta-breathe-mens-shoe-2ggX3h/CW0783-901', 'Brand': 'jordan'},...]
...
How come the Shoes are not sorted by price here? I have also tried using this code:
myd = mergedCollection.find(myquery)[0]["theAssociatedShoes"].sort("Price")
But that throws a syntax error. I've also tried this solution to no avail.
myd = mergedCollection.find(myquery).sort("theAssociatedShoes.Price", -1)
Looking at your code, the data you are interested in is all in one record. MongoDB's sort only works across multiple records. To get the results in price order, use python's sorted built-in function and a lambda expression.
myd = collection.find()
for record in sorted(myd[0]["theAssociatedShoes"], key=lambda price: price['Price']):
print(record)
I have a list of dictionaries from which I need to extract information, and then print it out in a special format.
dict_list = [
{'description' : 'item 1',
'amount' : int_amount1
},
{'description' : 'item 2',
'amount' : int_amount2
},
{'description' : 'item 3',
'amount' : int_amount3
]
I need to access the values in these dictionaries and print them out in the following way:
'item 1' int_amount1
'item 2' int_amount2
'item 3' int_amount3
If possible I would also need to print each key value at 20 characters max.
Thus far I have tried for loops and list comprehensions but the farthest I've gotten is just printing out all the values in a one by one list.
Any help would be appreciated.
You just need to iterate over the list and do your printing for each item
dict_list = [
{'description': 'item 1', 'amount': 'int_amount1'},
{'description': 'item 2', 'amount': 'int_amount2'},
{'description': 'item 3', 'amount': 'int_amount3'}
]
for item in dict_list:
print(f"{item['description']:20s}{item['amount']:20s}")
Giving
item 1 int_amount1
item 2 int_amount2
item 3 int_amount3
dict_list = [
{'description': 'item 1', 'amount': 'int_amount1'},
{'description': 'item 2', 'amount': 'int_amount2'},
{'description': 'item 3', 'amount': 'int_amount3'}
]
spaces = 20 * " "
for item in dict_list:
print(f"{item['description']}{spaces}{item['amount']}")
should work fine because it gives you:
item 1 int_amount1
item 2 int_amount2
item 3 int_amount3
I have an SQL database that I need to fetch and convert to JSON. I am thinking that the first step to do that is to fetch the data from the database and load it as a dataframe, then convert the dataframe into JSON object.
Let's say I have the following dataframe.
df_school = pd.DataFrame({'id':[1,2,3,4], 'school_code': ['ABC', 'IJK', 'QRS', 'XYZ'], 'name': ['School A','School B', 'School C', 'School D'], 'type':['private', 'public', 'public', 'private']})
print(df_school)
I want to convert it to JSON with the following code.
import collections
object_list =[]
for idx, row in df_school.iterrows():
d = collections.OrderedDict()
d['id'] = row['id']
d['school_code'] = row['school_code']
d['name'] = row['name']
d['type'] = row['type']
object_list.append(d)
j = json.dumps(object_list)
object_list = 'school_objects.js'
f = open(object_list, 'w')
print(j)
But the result is string. It only looks like a JSON, but when I try to access the item inside the so-called JSON, like j[0] it prints [, not an item inside the JSON.
I also tried another approach, by converting the result from SQL directly to JSON.
query = "Select * from school;"
df_school = pd.read_sql_query(query, connection)
json_school = df_school.head(10).to_json(orient='records')
But I also still got string.
How do I convert it to real JSON object?
Given the provided df_school variable, we can just do j=df_school.to_json(orient='records') to turn it into a JSON formatted string.
Once we have j storing the JSON formatted string, if we want to do something with it, we first have to load the JSON into Python again using json.loads(j).
So if we do:
j = df_school.to_json(orient='records')
# parse j into Python
loaded_json = json.loads(j)
print(loaded_json[0])
# print outputs: {'id': 1, 'name': 'School A', 'school_code': 'ABC', 'type': 'private'}
Hope this helps!
import pandas as pd
import json
df_school = pd.DataFrame({'id':[1,2,3,4], 'school_code': ['ABC', 'IJK', 'QRS', 'XYZ'], 'name': ['School A','School B', 'School C', 'School D'], 'type':['private', 'public', 'public', 'private']})
str_school = df_school.to_json(orient='records')
json_school = json.loads(str_school)
json_school[0]
{'id': 1, 'school_code': 'ABC', 'name': 'School A', 'type': 'private'}
JSON is a string encoding of objects.
Once you use json.dumps() or similar, you'll get a string.
Try the below code, Hope this will help :
data = [{columns:df_school.iloc[i][columns] for columns in list(df_school.columns) } for i in range(df_school.shape[0]) ]
print(data)
print("***********************")
print(type(data[0]))
Ouput will be :
[{'id': 1, 'school_code': 'ABC', 'name': 'School A', 'type': 'private'},
{'id': 2, 'school_code': 'IJK', 'name': 'School B', 'type': 'public'},
{'id': 3, 'school_code': 'QRS', 'name': 'School C', 'type': 'public'},
{'id': 4, 'school_code': 'XYZ', 'name': 'School D', 'type': 'private'}]
*************************
<class 'dict'>
data={k:list(v.values()) for k,v in df_school.to_dict().items()}
{
'id': [1, 2, 3, 4],
'school_code': ['ABC', 'IJK', 'QRS', 'XYZ'],
'name': ['School A', 'School B', 'School C', 'School D'],
'type': ['private', 'public', 'public', 'private']
}
I have this:
items = {{'project':'Project 1','description':'Task description','time':1222222},
{'project':'Project 2','description':'Task description 2','time':1224322},
{'project':'Project 1','description':'Task description 3','time':13222152}}
And I need something like this:
resultitems = {
'project':'Project 1','pritems':{
{'description':'Task description','time':1222222},
{'description':'Task description 3','time':13222152}},
'project':'Project 2',pritems':{
{'description':'Task description 2','time':1224322}},
}
of simply the name of each project as a key
I've tried this approach:
resultitems = {}
resultitems['Project 2'] = {}
resultitems['Project 2'].update(..)
update does not work, since it replaces the previous value
in php, it was easy,
$resultitems['Project 2'][] = array(...)
but don't find the way to do this in Python
result_items = {
'house project': [{'task': 'cleaning', 'hours': 20}, {'task': 'painting', 'hours: 30', etc.],
'garden project': [{'task': 'mowing the lawn', 'hours': 1, etc.
etc.
}
Your variable 'items' is not correct. If it is a list of dictionaries, it should be:
items = [{...}, {...}, {...}]
Please write the source of the data, from where do you get the data. This will determine the way you will fill in the desired dictionary. If you already have the data as in 'items' (i.e. a list of dictionaries), then here is how to converted it:
items = [{'project':'Project 1','description':'Task description','time':1222222},
{'project':'Project 2','description':'Task description 2','time':1224322},
{'project':'Project 1','description':'Task description 3','time':13222152}]
dct = {}
for e in items :
if e['project'] not in dct :
dct[e['project']] = []
dct[e['project']].append(dict([(k, v) for k,v in e.items() if k != 'project']))
print dct
and output is:
{'Project 2': [{'description': 'Task description 2', 'time': 1224322}], 'Project 1': [{'description': 'Task description', 'time': 1222222}, {'description': 'Task description 3', 'time': 13222152}]}
Finally, I used this:
newdata = {}
for data in result['data']:
try:
newdata[data['project']].append({"description":data['description'],"start":data['start'],"time":data['dur']})
except:
newdata[data['project']] = []
newdata[data['project']].append({"description":data['description'],"start":data['start'],"time":data['dur']})
print newdata
And the result has been like this, and this is what I needed:
{
u'Project 1': [
{'start': u'2015-07-09T18:09:41-03:00', 'description': u'Task 1 name', 'time': 1432000},
{'start': u'2015-07-09T17:42:36-03:00', 'description': u'Task 2 name', 'time': 618000}
],
u'Project 2': [
{'start': u'2015-07-09T20:14:16-03:00', 'description': u'Other Task Name', 'time': 4424000}
],
u'Project 3': [
{'start': u'2015-07-09T22:29:51-03:00', 'description': u'another task name for pr3', 'time': 3697000},
{'start': u'2015-07-09T19:38:02-03:00', 'description': u'something more to do', 'time': 59000},
{'start': u'2015-07-09T19:11:49-03:00', 'description': u'Base tests', 'time': 0},
{'start': u'2015-07-09T19:11:29-03:00', 'description': u'Domain', 'time': 0}
],
u'Project something': [
{'start': u'2015-07-09T19:39:30-03:00', 'description': u'Study more', 'time': 2069000},
{'start': u'2015-07-09T15:46:39-03:00', 'description': u'Study more (2)', 'time': 3800000},
{'start': u'2015-07-09T11:46:00-03:00', 'description': u'check forms', 'time': 660000}
]
}
by the way, I was no asking about the structure itself.. instead what I needed was someway to program a "something like this" structure.