Related
I have a dataframe like below:
df = pd.DataFrame({
'Aapl': [12, 5, 8],
'Fs': [18, 12, 8],
'Bmw': [6, 18, 12],
'Year': ['2020', '2025', '2030']
})
I want a dictionary like:
d={'2020':[12,18,16],
'2025':[5,12,18],
'2030':[8,8,12]
}
I am not able to develop the whole logic:
lst = [list(item.values()) for item in df.to_dict().values()]
dic={}
for items in lst:
for i in items[-1]:
dic[i]=#2000 will hold all the 0th values of other lists and so on
Is there any easier way using pandas ?
Convert Year to index, transpose and then in dict comprehension create lists:
d = {k: list(v) for k, v in df.set_index('Year').T.items()}
print (d)
{'2020': [12, 18, 6], '2025': [5, 12, 18], '2030': [8, 8, 12]}
Or use DataFrame.agg:
d = df.set_index('Year').agg(list, axis=1).to_dict()
print (d)
{'2020': [12, 18, 6], '2025': [5, 12, 18], '2030': [8, 8, 12]}
Try this:
import pandas as pd
data = {'Name': ['Ankit', 'Amit',
'Aishwarya', 'Priyanka'],
'Age': [21, 19, 20, 18],
'Stream': ['Math', 'Commerce',
'Arts', 'Biology'],
'Percentage': [88, 92, 95, 70]}
# Convert the dictionary into DataFrame
df = pd.DataFrame(data, columns=['Name', 'Age',
'Stream', 'Percentage'])
Sometimes DataFrame columns is an array of arrays:
df['value'].values
array([[1.51808096e+11],
[1.49119648e+11],
...
[1.18009284e+11],
[1.44851665e+11]])
And sometimes a regular array:
df['value'].values
array([1.51808096e+11,
1.49119648e+11,
...
1.18009284e+11,
1.44851665e+11])
DataFrame created with a csv will sometimes give one format, and sometimes the other. This causes issues. Using df['value'].values = df['value'].values.flatten() does not work.
cols = df.columns
df = pd.DataFrame(df.values.flatten().reshape(-1, df.shape[1]), columns=cols)
I was having a heck of a time recreating your data to have an array of arrays print out from my column. But maybe using reshape and then grabbing the first index will work for you. Like:
# With just an array
arr = np.array([[1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5]])
arr.reshape(1,-1)[0]
Output:
array([1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5])
Or with a dataframe example:
lst = [['tom', 'reacher', 25], ['krish', 'pete', 30],
['nick', 'wilson', 26], ['juli', 'williams', 22]]
df = pd.DataFrame(lst, columns =['FName', 'LName', 'Age'])
df.values.reshape(1,-1)[0]
Output:
array(['tom', 'reacher', 25, 'krish', 'pete', 30, 'nick', 'wilson', 26,
'juli', 'williams', 22], dtype=object)
If this doesn't work, could you add a minimal working example to your question to recreate your dataframe?
list comprehension would do the job
array = [v for e in [x for x in df['values']] for v in e]
I have a dictionary of values in tuple form, how to get the values in list form.
I want to get values from the tuples and create new lists and create another 3 lists with squares from them.
dictionary={1:(1,2,3),2:(3,4,5),3:(6,7,8),4:(9,10,11),5:(12,13,14)}
s=list(d.values())
d=[item for t in s for item in t]
print(d)
I used list comprehension i got this output:
[1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
Using list comprehension
Expected_output:
[1,3,6,9,12],
[2,4,7,10,13],
[3,5,8,11,14],
squares**2 output above three list :
[1,9,36,81,144],
[4,16,49,100,169],
[9,25,64,121,196]
Provided with a Dictionary
First take a empty list and assign it to a variable “l”
Using list comprehension separate the values and store that in a variable
Iterate the values and append the empty list “l”
Now iterate the “l” using index values i[o], i[1], i[2] and store in various variables respectively
Using map function square the variables and store the values and print them using the list of variables
x = {
1:(1,2,3),
2:(4,5,6),
3:(7,8,9),
4:(10,11,12),
5:(13,14,15)
}
l = []
y = [i for i in x.values()]
for i in y:
l.append(i)
print(l)
m = [i[0] for i in l]
n = [i[1] for i in l]
o = [i[2] for i in l]
m1 = map(lambda i:i**2, m)
n1 = map(lambda i:i**2, n)
o1 = map(lambda i:i**2, o)
print(m)
print(list(m1))
print(n)
print(list(n1))
print(o)
print(list(o1))
you can use zip to collect the index elements of each list together, then use list comprehension to square them
dictionary={1:(1,2,3),2:(3,4,5),3:(6,7,8),4:(9,10,11),5:(12,13,14)}
list_vals = list(zip(*dictionary.values()))
squares = [[num ** 2 for num in nums] for nums in list_vals]
print(list_vals)
print(squares)
OUTPUT
[(1, 3, 6, 9, 12), (2, 4, 7, 10, 13), (3, 5, 8, 11, 14)]
[[1, 9, 36, 81, 144], [4, 16, 49, 100, 169], [9, 25, 64, 121, 196]]
Thanks to comments from #roganjosh highlighting that the dict will only be assured to be ordered if the pythong version is 3.6 or higher. If your python version is less than that you would first need to sort the values by order of the keys. Below is an example.
dictionary={2:(3,4,5),3:(6,7,8),4:(9,10,11),5:(12,13,14),1:(1,2,3)}
ordered_key_val = sorted(dictionary.items(), key=lambda items: items[0])
list_vals = list(zip(*[val for key, val in ordered_key_val]))
squares = [[num ** 2 for num in nums] for nums in list_vals]
print(list_vals)
print(squares)
You can use numpy to transpose the entire list once the values of the dictionary are obtained. You can use the below program
import numpy as np
dictionary={1:(1,2,3),2:(3,4,5),3:(6,7,8),4:(9,10,11),5:(12,13,14)}
list_out= []
for i in dictionary.keys():
list_out.append(dictionary[i])
tran_list = np.transpose(list_out)
out_list = tran_list*tran_list
Output of this is:
>>> out_list
array([[ 1, 9, 36, 81, 144],
[ 4, 16, 49, 100, 169],
[ 9, 25, 64, 121, 196]])
This is an array output! Anyway if you want it only in the list, ofcourse , you can play with it!
You can do this way:
>>> temp = list(zip(*dictionary.values()))
>>> [list(i) for i in temp]
[[1, 3, 6, 9, 12], [2, 4, 7, 10, 13], [3, 5, 8, 11, 14]]
>>> [[i**2 for i in elem] for elem in temp]
[[1, 9, 36, 81, 144], [4, 16, 49, 100, 169], [9, 25, 64, 121, 196]]
I have one dictionary here
d={1:(1,2,3),2:(4,5,6),3:(7,8,9),4:(10,11,12),5:(13,14,15)}
first I want to get values in tuple in three lists then I used list comprehension here The below code gives the tuple values in three lists
myList1 = [d [i][0] for i in (d.keys()) ]
print(myList1)
myList2 = [d [i][1] for i in (d.keys()) ]
print(myList2)
myList3 = [d [i][2] for i in (d.keys()) ]
print(myList3)
Here all the tuple values converted into list form
[1, 4, 7, 10, 13]
[2, 5, 8, 11, 14]
[3, 6, 9, 12, 15]
Now I want to squares the elements in three lists here I Used lambda expression the below code squares the elements in the lists
a1= list(map(lambda x: x**2 ,myList1))
print(a1)
a2= list(map(lambda x: x**2 ,myList2))
print(a2)
a3= list(map(lambda x: x**2 ,myList3))
print(a3)
The output is:
[1, 16, 49, 100, 169]
[4, 25, 64, 121, 196]
[9, 36, 81, 144, 225]
I have a OP : {'2017-05-06': [3, 7, 8],'2017-05-07': [3, 9, 10],'2017-05-08': [4]}
from the OP I just want another OP :
{'2017-05-06': [15, 11, 10],'2017-05-07': [19, 13, 12],'2017-05-08': [4]}
which means:
Ncleand is 2017-05-06
element total is 18 so '2017-05-06': [3 -18, 7-18, 8-18] = '2017-05-06': [15, 11, 10]
likewise all elements data.
So final output is {'2017-05-06': [15, 11, 10],'2017-05-07': [19, 13, 12],'2017-05-08': [4]}
How to do this?
Note : I am using python 3.6.2 and pandas 0.22.0
code so far :
import pandas as pd
dfs = pd.read_excel('ff2.xlsx', sheet_name=None)
dfs1 = {i:x.groupby(pd.to_datetime(x['date']).dt.strftime('%Y-%m-%d'))['duration'].sum() for i, x in dfs.items()}
d = pd.concat(dfs1).groupby(level=1).apply(list).to_dict()
actuald = pd.concat(dfs1).div(80).astype(int)
sum1 = actuald.groupby(level=1).transform('sum')
m = actuald.groupby(level=1).transform('size') > 1
cleand = sum1.sub(actuald).where(m, actuald).groupby(level=1).apply(list).to_dict()
print (cleand)
From the cleand I want to do this?
In a compact (but somehow inefficient) way:
>>> op = {'2017-05-06': [3, 7, 8],'2017-05-07': [3, 9, 10],'2017-05-08': [4]}
>>> { x:[sum(y)-i for i in y] if len(y)>1 else y for x,y in op.items() }
#output:
{'2017-05-06': [15, 11, 10], '2017-05-07': [19, 13, 12], '2017-05-08': [4]}
def get_list_manipulation(list_):
subtraction = list_
if len(list_) != 1:
total = sum(list_)
subtraction = [total-val for val in list_]
return subtraction
for key, values in data.items():
data[key] = get_list_manipulation(values)
>>>{'2017-05-06': [15, 11, 10], '2017-05-07': [19, 13, 12], '2017-05-08': [4]}
I am a new python user, and I need help about combining list elements under a condition.
I have a list like this:
x = [['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
I would like to combine list elements which start with the same letter in a list by summing up the other elements. for example, I'd like to obtain this list for x:
x = [['a', 30, 120], ['b', 10, 20]]
How can I achieve this ?
A one-liner using itertools.groupby():
In [45]: lis=[['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
In [46]: lis.sort(key=itemgetter(0)) #sort the list first
In [47]: lis
Out[47]: [['a', 10, 20], ['a', 20, 100], ['b', 10, 20]]
In [49]: [[k]+map(sum,zip(*[x[1:] for x in g])) for k,g in groupby(lis,key=itemgetter(0))]
Out[49]: [['a', 30, 120], ['b', 10, 20]]
A simple solution:
In [23]: lis=[['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
In [24]: ans=[]
In [25]: lis.sort(key=itemgetter(0)) #sort the list according to the first elem
In [26]: lis
Out[26]: [['a', 10, 20], ['a', 20, 100], ['b', 10, 20]]
In [27]: for x in lis:
if ans:
if x[0]==ans[-1][0]: #if the value of the first elem of last element in ans is same as x[0]
ans[-1][1]+=x[1]
ans[-1][2]+=x[2]
else:
ans.append(x)
else:ans.append(x)
....:
In [28]: ans
Out[28]: [['a', 30, 120], ['b', 10, 20]]
Without sorting the list using defaultdict():
In [69]: dic=defaultdict(list)
In [70]: for x in lis:
dic[x[0]].append(x[1:])
....:
In [71]: dic
Out[71]: defaultdict(<type 'list'>, {'a': [[10, 20], [20, 100]], 'b': [[10, 20]]})
In [72]: [[k]+map(sum,zip(*i)) for k,i in dic.items()]
Out[72]: [['a', 30, 120], ['b', 10, 20]]
Another approach using dict and map:
>>> x = [['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
>>> d = {}
>>> from operator import add
>>> for k, v1, v2 in x:
d[k] = map(add, d.get(k, [0, 0]), [v1, v2])
>>> d
{'a': [30, 120], 'b': [10, 20]}
I'm going to use the answer code for a huge data which include over millons elements. I'd like the reduce the list elements this way.
In such a case you probably don't want to be sorting the data or building a fully copy as you're iterating over it.
The following solution does neither. It can also handle sublists of any length (as long as all lengths are the same):
def add(d, l):
k = l[0] # extract the key
p = d.get(k, None) # see if we already have a partial sum for this key
if p:
d[k] = [x+y for x,y in zip(p, l[1:])] # add to the previous sum
else:
d[k] = l[1:] # create a new sum
return d
x = [['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
result = [[k] + v for k,v in reduce(add, x, {}).items()]
print(result)
Alternatively,
import collections, operator
x = [['a', 10, 20], ['b', 10, 20], ['a', 20, 100]]
d = collections.defaultdict(lambda:[0] * (len(x[0]) - 1))
for el in x:
d[el[0]] = map(operator.add, d[el[0]], el[1:])
result = [[k] + v for k,v in d.items()]
print(result)
This works exactly the same as the first version, but uses defaultdict and explicit iteration.