We Keep Coding

How to create a dataframe from a dictionary such that each entry is a list?

How do I create a dataframe from a dictionary where each entry is a list? For example, the script below
for k,v in pose.items():
print(k,v)
prints
p1_center_head [-0.91411722 -1.11261988 1.70305252]
p1_right_hand [-0.6696707 -1.45284259 0.85303462]
p1_front_chest [-0.90311265 -1.15465713 1.09575438]
p1_right_shoulder [-0.88741797 -1.37508869 1.49578547]
p1_right_elbow [-0.8592056 -1.48356283 1.14748859]
p1_right_head [-0.92714155 -1.27181339 1.6833576 ]
p1_left_shoulder [-1.14574909 -1.15120935 1.48121977]
p1_left_head [-1.06484675 -1.14306569 1.65506554]
p2_center_head [1.0070374 1.08997941 1.69709778]
p1_right_wrist [-0.72482365 -1.47362924 0.92447436]
p2_right_shoulder [0.95422077 1.31905603 1.48511732]
p2_right_hand [0.69995475 1.25936687 0.90006113]
p2_right_wrist [0.75100577 1.30776393 0.94339645]
p2_left_shoulder [1.24170136 1.11059046 1.47705841]
p2_back_chest [1.19639254 1.35239506 0.93713373]
p1_back_chest [-1.06148314 -1.38686275 1.03079486]
p2_front_chest [0.9896763 1.14398277 0.99924839]
p2_right_elbow [0.88454074 1.43418157 1.13609421]
p2_right_head [1.01738358 1.24647975 1.68129683]
p2_left_head [1.15873718 1.12672687 1.66546488]
When I create a dataframe and save it to a csv file however, each entry contains one float instead of a list of floats.
pose = pd.DataFrame.from_dict(pose)
pose.to_csv("pose.csv"),index=False)
I need each entry to be a list e.g. pose["p1_center_head"] = [-0.91411722 -1.11261988 1.70305252]

See if this is what you want:
df = pd.DataFrame.from_dict(pose, orient="index")
df['list_item'] = df.apply(list, axis=1)
df.drop(columns=[0, 1, 2], inplace=True)
print(df)
Output:
list_item
p1_center_head [-0.91411722, -1.11261988, 1.70305252]
p1_right_hand [-0.6696707, -1.45284259, 0.85303462]
p1_front_chest [-0.90311265, -1.15465713, 1.09575438]
p1_right_shoulder [-0.88741797, -1.37508869, 1.49578547]
p1_right_elbow [-0.8592056, -1.48356283, 1.14748859]
p1_right_head [-0.92714155, -1.27181339, 1.6833576]
p1_left_shoulder [-1.14574909, -1.15120935, 1.48121977]
p1_left_head [-1.06484675, -1.14306569, 1.65506554]
p2_center_head [1.0070374, 1.08997941, 1.69709778]
p1_right_wrist [-0.72482365, -1.47362924, 0.92447436]
p2_right_shoulder [0.95422077, 1.31905603, 1.48511732]
p2_right_hand [0.69995475, 1.25936687, 0.90006113]
p2_right_wrist [0.75100577, 1.30776393, 0.94339645]
p2_left_shoulder [1.24170136, 1.11059046, 1.47705841]
p2_back_chest [1.19639254, 1.35239506, 0.93713373]
p1_back_chest [-1.06148314, -1.38686275, 1.03079486]
p2_front_chest [0.9896763, 1.14398277, 0.99924839]
p2_right_elbow [0.88454074, 1.43418157, 1.13609421]
p2_right_head [1.01738358, 1.24647975, 1.68129683]
p2_left_head [1.15873718, 1.12672687, 1.66546488]
Then you can further query for an entry like this:
df.loc['p1_right_head']
Output:
list_item [-0.92714155, -1.27181339, 1.6833576]
Name: p1_right_head, dtype: object

List object return two separate List on append()

I have a function named "search_suggestion" that takes search parameter and pass into MySQL then a result is appended into an empty list "Suggestion" inside a function below
def search_suggestion(self,search,limit=25):
"""This method takes the parameter search return the search suggestion of employees in database"""
cursor = None
suggestions = []
try:
cursor = kasaa()
cursor.execute(
'''
SELECT ospos_people.first_name,ospos_people.last_name
FROM ospos_employees
INNER JOIN ospos_people ON ospos_employees.person_id = ospos_people.person_id
WHERE ospos_employees.deleted = 0 AND ospos_people.first_name LIKE %s OR ospos_people.last_name LIKE %s
OR ospos_people.phone_number LIKE %s OR ospos_people.email LIKE %s
ORDER BY ospos_people.first_name ASC LIMIT %s
''',(search,search,search,search,limit)
)
row = cursor.fetchall()
for ro in row:
suggestions.append(ro["first_name"]+ " " + ro["last_name"])
print(suggestions)
except Exception as e:
print(e)
finally:
cursor.close()
what am expecting is a list like ['alkhadil Issa', 'john Magufuli'] a one single list
instead am getting two list.
[alkhadil Issa']
['alkhadil Issa' 'john Magufuli']
I have try to check if len(suggestions) < 1: before append ro["first_name"] but am not getting what i want. What is the most efficient way of doing this, any patient you can afford on my learning journey i would appreciate

I replicated your problem by manually creating an output similar to what cursor.fetchall() returns according to you.
>>> dic1 = {'first_name': 'Abdallah', 'last_name': 'Abdillah'}
>>> dic2 = {'first_name': 'Joseph', 'last_name': 'Magufuli'}
>>> row = [dic1, dic2]
>>> row
[{'first_name': 'Abdallah', 'last_name': 'Abdillah'}, {'first_name': 'Joseph', 'last_name': 'Magufuli'}]
Assuming cursor.fetchall() returns something similar to the list above your code should work fine:
>>> suggestions = []
>>> for r in row:
... suggestions.append(r['first_name'] + " " + r['last_name'])
... print(suggestions)
...
['Abdallah Abdillah']
['Abdallah Abdillah', 'Joseph Magufuli']
If that is not the case, then your problem is your cursor.fetchall() result.
Edit:
I just realized your problem is getting 2 lists. Please be aware that your print statement is inside the for loop, so each time a value is added to the list, the resulting list is printed. If you only want to print the list in the end, just add the print statement after the loop ends:
So, instead of:
>>> for dic in row:
... suggestions.append(dic['first_name'] + " " + dic['last_name'])
... print(suggestions)
...
['Abdallah Abdillah']
['Abdallah Abdillah', 'Joseph Magufuli']
Place the print outside of the loop:
>>> for r in row:
... suggestions.append(r['first_name'] + " " + r['last_name'])
...
>>> print(suggestions)
['Abdallah Abdillah', 'Joseph Magufuli']

From set to set of set?

I have set of elements, I want to convert this set to a list of list of 5 elements.
i.e.
I want set below
symbol_list = set([u'DIVISLAB', u'TITAN', u'JINDALSTEL', u'ENDURANCE', u'PGHH', u'GMRINFRA', u'UNIONBANK', u'RAMCOCEM', u'GAIL', u'ICICIGI', u'L&TFH', u'HINDUNILVR', u'SBIN', u'PRESTIGE', u'BERGEPAINT', u'LT', u'HINDPETRO', u'RELIANCE', u'GODREJCP', u'GRAPHITE', u'RELINFRA', u'NBCC', u'MCDOWELL-N', u'SYNGENE', u'IOC', u'PETRONET', u'SUNPHARMA', u'GRASIM', u'FEDERALBNK', u'GRUH', u'CANBK', u'BBTC', u'FCONSUMER', u'MFSL', u'MRF', u'TATACHEM', u'IDFCFIRSTB', u'FRETAIL', u'OIL', u'DBL', u'PFIZER', u'BANKINDIA', u'CHOLAFIN', u'MARUTI', u'HDFC', u'EXIDEIND', u'VOLTAS', u'PAGEIND', u'RELCAPITAL', u'HDFCAMC', u'INDHOTEL', u'INDIGO', u'BHARATFORG', u'BPCL', u'MOTHERSUMI', u'COLPAL', u'LTTS', u'BAJAJHLDNG', u'GICRE', u'KOTAKBANK', u'ABCAPITAL', u'CADILAHC', u'PIDILITIND', u'APOLLOTYRE', u'AUBANK', u'TCS', u'NATCOPHARM', u'AMARAJABAT', u'EICHERMOT', u'QUESS', u'SBILIFE', u'HCLTECH', u'SHREECEM', u'UPL', u'ESCORTS', u'DLF', u'BRITANNIA', u'MPHASIS', u'LUPIN', u'ONGC', u'GSPL', u'TATAGLOBAL', u'DISHTV', u'NIACL', u'NMDC', u'VARROC', u'SUNTV', u'IGL', u'GLENMARK', u'WIPRO', u'MARICO', u'COROMANDEL', u'TORNTPHARM', u'ASHOKLEY', u'MRPL', u'OBEROIRLTY', u'BIOCON', u'HINDALCO', u'SAIL', u'MGL', u'ICICIBANK', u'NTPC', u'BAJFINANCE', u'ACC', u'CONCOR', u'IDEA', u'RBLBANK', u'PEL', u'MUTHOOTFIN', u'M&MFIN', u'JUBILANT', u'OFSS', u'EDELWEISS', u'HEXAWARE', u'BEL', u'ADANIPORTS', u'DRREDDY', u'CROMPTON', u'ASIANPAINT', u'JSWSTEEL', u'AJANTPHARM', u'AXISBANK', u'SPARC', u'APOLLOHOSP', u'RECLTD', u'GODREJAGRO', u'JSWENERGY', u'ADANIPOWER', u'SRF', u'BANKBARODA', u'IDBI', u'HEG', u'ENGINERSIN', u'TATAMTRDVR', u'LTI', u'IBVENTURES', u'NHPC', u'BATAINDIA', u'HEROMOTOCO', u'ZEEL', u'AUROPHARMA', u'HDFCBANK', u'NAUKRI', u'ULTRACEMCO', u'ITC', u'HUDCO', u'TORNTPOWER', u'INFY', u'MINDTREE', u'IBULHSGFIN', u'BHARTIARTL', u'TATASTEEL', u'GODREJIND', u'AMBUJACEM', u'M&M', u'POWERGRID', u'HDFCLIFE', u'MANAPPURAM', u'DHFL', u'RPOWER', u'BALKRISIND', u'ABFRL', u'PNBHOUSING', u'HINDZINC', u'STRTECH', u'RAJESHEXPO', u'TATAMOTORS', u'TATAPOWER', u'DMART', u'CIPLA', u'HAVELLS', u'COALINDIA', u'LICHSGFIN', u'JUBLFOOD', u'BAJAJ-AUTO', u'DABUR', u'CUMMINSIND', u'NATIONALUM', u'INFRATEL', u'ABB', u'VEDL', u'BHEL', u'UBL', u'BOSCHLTD', u'BAJAJFINSV', u'TECHM', u'INDIANB', u'CASTROLIND', u'PIIND', u'PFC', u'PNB', u'BANDHANBNK', u'YESBANK', u'ALKEM', u'INDUSINDBK', u'SIEMENS', u'TVSMOTOR', u'GSKCONS', u'SRTRANSFIN', u'ICICIPRULI', u'VGUARD'])
to be of form
convertedset = ([[u'DIVISLAB', u'TITAN', u'JINDALSTEL', u'ENDURANCE', u'PGHH'], [u'GMRINFRA', u'UNIONBANK', u'RAMCOCEM', u'GAIL', u'ICICIGI'],[u'L&TFH', u'HINDUNILVR', u'SBIN'...]])

Try this :
symbol_list = [u'DIVISLAB', u'TITAN', u'JINDALSTEL', u'ENDURANCE', u'PGHH', u'GMRINFRA', u'UNIONBANK', u'RAMCOCEM', u'GAIL', u'ICICIGI', u'L&TFH', u'HINDUNILVR', u'SBIN', u'PRESTIGE', u'BERGEPAINT', u'LT', u'HINDPETRO', u'RELIANCE', u'GODREJCP', u'GRAPHITE', u'RELINFRA', u'NBCC', u'MCDOWELL-N', u'SYNGENE', u'IOC', u'PETRONET', u'SUNPHARMA', u'GRASIM', u'FEDERALBNK', u'GRUH', u'CANBK', u'BBTC', u'FCONSUMER', u'MFSL', u'MRF', u'TATACHEM', u'IDFCFIRSTB', u'FRETAIL', u'OIL', u'DBL', u'PFIZER', u'BANKINDIA', u'CHOLAFIN', u'MARUTI', u'HDFC', u'EXIDEIND', u'VOLTAS', u'PAGEIND', u'RELCAPITAL', u'HDFCAMC', u'INDHOTEL', u'INDIGO', u'BHARATFORG', u'BPCL', u'MOTHERSUMI', u'COLPAL', u'LTTS', u'BAJAJHLDNG', u'GICRE', u'KOTAKBANK', u'ABCAPITAL', u'CADILAHC', u'PIDILITIND', u'APOLLOTYRE', u'AUBANK', u'TCS', u'NATCOPHARM', u'AMARAJABAT', u'EICHERMOT', u'QUESS', u'SBILIFE', u'HCLTECH', u'SHREECEM', u'UPL', u'ESCORTS', u'DLF', u'BRITANNIA', u'MPHASIS', u'LUPIN', u'ONGC', u'GSPL', u'TATAGLOBAL', u'DISHTV', u'NIACL', u'NMDC', u'VARROC', u'SUNTV', u'IGL', u'GLENMARK', u'WIPRO', u'MARICO', u'COROMANDEL', u'TORNTPHARM', u'ASHOKLEY', u'MRPL', u'OBEROIRLTY', u'BIOCON', u'HINDALCO', u'SAIL', u'MGL', u'ICICIBANK', u'NTPC', u'BAJFINANCE', u'ACC', u'CONCOR', u'IDEA', u'RBLBANK', u'PEL', u'MUTHOOTFIN', u'M&MFIN', u'JUBILANT', u'OFSS', u'EDELWEISS', u'HEXAWARE', u'BEL', u'ADANIPORTS', u'DRREDDY', u'CROMPTON', u'ASIANPAINT', u'JSWSTEEL', u'AJANTPHARM', u'AXISBANK', u'SPARC', u'APOLLOHOSP', u'RECLTD', u'GODREJAGRO', u'JSWENERGY', u'ADANIPOWER', u'SRF', u'BANKBARODA', u'IDBI', u'HEG', u'ENGINERSIN', u'TATAMTRDVR', u'LTI', u'IBVENTURES', u'NHPC', u'BATAINDIA', u'HEROMOTOCO', u'ZEEL', u'AUROPHARMA', u'HDFCBANK', u'NAUKRI', u'ULTRACEMCO', u'ITC', u'HUDCO', u'TORNTPOWER', u'INFY', u'MINDTREE', u'IBULHSGFIN', u'BHARTIARTL', u'TATASTEEL', u'GODREJIND', u'AMBUJACEM', u'M&M', u'POWERGRID', u'HDFCLIFE', u'MANAPPURAM', u'DHFL', u'RPOWER', u'BALKRISIND', u'ABFRL', u'PNBHOUSING', u'HINDZINC', u'STRTECH', u'RAJESHEXPO', u'TATAMOTORS', u'TATAPOWER', u'DMART', u'CIPLA', u'HAVELLS', u'COALINDIA', u'LICHSGFIN', u'JUBLFOOD', u'BAJAJ-AUTO', u'DABUR', u'CUMMINSIND', u'NATIONALUM', u'INFRATEL', u'ABB', u'VEDL', u'BHEL', u'UBL', u'BOSCHLTD', u'BAJAJFINSV', u'TECHM', u'INDIANB', u'CASTROLIND', u'PIIND', u'PFC', u'PNB', u'BANDHANBNK', u'YESBANK', u'ALKEM', u'INDUSINDBK', u'SIEMENS', u'TVSMOTOR', u'GSKCONS', u'SRTRANSFIN', u'ICICIPRULI', u'VGUARD']
convertedlist = [symbol_list[i:i+5] for i in range(0, len(symbol_list), 5)]
Output :
[['DIVISLAB', 'TITAN', 'JINDALSTEL', 'ENDURANCE', 'PGHH'],
['GMRINFRA', 'UNIONBANK', 'RAMCOCEM', 'GAIL', 'ICICIGI'],
['L&TFH', 'HINDUNILVR', 'SBIN', 'PRESTIGE', 'BERGEPAINT'],
['LT', 'HINDPETRO', 'RELIANCE', 'GODREJCP', 'GRAPHITE'], ...]
Note :
Don't convert symbol_list to a set as even if not converted to a set it would contain unique elements 'cause len(symbol_list) == len(set(symbol_list)). Both are having 201 elements.

This can be handled by the below code, without using any lib,
split_size = 5
converted_list = []
# ouput list
split_list = []
# chlid list items, part of the output list
for index, elt in enumerate(symbol_list):
split_list.append(elt)
# we append to the list till the split_size cut
# and then append this list to the output list and continue
if index and index % split_size == 0:
converted_list.append(split_list)
split_list = []

Create new dictionary with specific keys from old dictionary

I want to make a new dictionary that prints a new object containing uuid, name, website, and email address for all rows of my dict that have values for all four of these attributes.
I thought I did this for email, name, and website below in my code but I noticed sometimes name or email wont print (because they have missing values), how do I drop those? Also, uuid is outside of the nested dictionary, how do I add that in the new dictionary too?
I attached my code and an element from my code below.
new2 = {}
for i in range (0, len(json_file)):
try:
check = json_file[i]['payload']
new = {k: v for k, v in check.items() if v is not None}
new2 = {k: new[k] for k in new.keys() & {'name', 'website', 'email'}}
print(new2)
except:
continue
Dictionary sample:
{
"payload":{
"existence_full":1,
"geo_virtual":"[\"56.9459720|-2.1971226|20|within_50m|4\"]",
"latitude":"56.945972",
"locality":"Stonehaven",
"_records_touched":"{\"crawl\":8,\"lssi\":0,\"polygon_centroid\":0,\"geocoder\":0,\"user_submission\":0,\"tdc\":0,\"gov\":0}",
"address":"The Lodge, Dunottar",
"email":"dunnottarcastle#btconnect.com",
"existence_ml":0.5694238217658721,
"domain_aggregate":"",
"name":"Dunnottar Castle",
"search_tags":[
"Dunnottar Castle Aberdeenshire",
"Dunotter Castle"
],
"admin_region":"Scotland",
"existence":1,
"category_labels":[
[
"Landmarks",
"Buildings and Structures"
]
],
"post_town":"Stonehaven",
"region":"Kincardineshire",
"review_count":"719",
"geocode_level":"within_50m",
"tel":"01569 762173",
"placerank":65,
"longitude":"-2.197123",
"placerank_ml":37.27916073464469,
"fax":"01330 860325",
"category_ids_text_search":"",
"website":"http://www.dunnottarcastle.co.uk",
"status":"1",
"geocode_confidence":"20",
"postcode":"AB39 2TL",
"category_ids":[
108
],
"country":"gb",
"_geocode_quality":"4"
},
"uuid":"3867aaf3-12ab-434f-b12b-5d627b3359c3"
}

Try using the dict.get() method:
def new_dict(input_dict, keys, fallback='payload'):
ret = dict()
for key in keys:
val = input_dict.get(key) or input_dict[fallback].get(key)
if val:
ret.update({key:val})
if len(ret) == 4: # or you could do: if set(ret.keys()) == set(keys):
print(ret)
for dicto in json_file:
new_dict(dicto, ['name','website','email','uuid'])
{'name': 'Dunnottar Castle', 'website': 'http://www.dunnottarcastle.co.uk', 'email': 'dunnottarcastle#btconnect.com', 'uuid': '3867aaf3-12ab-434f-b12b-5d627b3359c3'}

Multiple split of input?

I know that you can use split() to split a user input into two, but how would you split input that consists of multiple variables ? For example:
User input:
Shawn=14:soccer#2991842
What I would like to do:
name = Shawn
age = 14
course = soccer
idnumber = 2991842
What's the best way to do such thing ?

str = 'Shawn=14:soccer#2991842'
keys = ['name', 'age', 'course', 'idnumber']
values = re.split('[=:#]', str)
print dict(zip(keys, values))
Out[114]: {'age': '14', 'course': 'soccer', 'idnumber': '2991842', 'name': 'Shawn'}

I think Regex will work best here:
>>> from re import split
>>> mystr = "Shawn=14:soccer#2991842"
>>> split("\W", mystr)
['Shawn', '14', 'soccer', '2991842']
>>> lst = split("\W", mystr)
>>> name = lst[0]
>>> name
'Shawn'
>>> age = lst[1]
>>> age
'14'
>>> course = lst[2]
>>> course
'soccer'
>>> idnumber = lst[3]
>>> idnumber
'2991842'
>>>
Also, the above is a step-by-step demonstration. You can actually just do:
name, age, course, idnumber = split("\W", mystr)

Here's how I would do it.
def splitStr(str):
temp = str.split(':')
temp_nameAge = temp[0].split('=')
temp_courseId = temp[1].split('#')
name = temp_nameAge[0]
age = int(temp_nameAge[1])
course = temp_courseId[0]
idnumber = int(temp_courseId[1])
print 'Name = %s, age = %i, course = %s, id_number = %i' % (name, age, course, idnumber)

Another thing you can do is use split like: string.split(":").
Then you can change the format to "name:age:course:number"

You could just keep splitting the splits...
text2split = "Shawn=14:soccer#2991842"
name = text2split.split('=')[0]
age = text2split.split('=')[1].split(':')[0]
course = text2split.split('=')[1].split(':')[1].split('#')[0]
idnumber = text2split.split('=')[1].split(':')[1].split('#')[1]
This isn't the most elegant way to do it, but it'll work so long as text2split always has the same delimeters.

If you are ok with storing them under dictionary keys, you could use named group references
import re
x='shawn=14:soccer#2991842'
re.match(r'(?P<name>.*?)=(?P<age>.*):(?P<course>.*?)#(?P<idnumber>.*)', x).groupdict()
{'idnumber': '2991842', 'course': 'soccer', 'age': '14', 'name': 'shawn