We Keep Coding

Filter DataTable Bokeh

I have a minimal sample below to support my question.
I have a collection of dataframes in a dictionary that I used to generate DataTable(s) and storage them in an array with the def fDP, then storage in dictionaries the panels and tabs in order to filter them later with the Select widget as in the example. It works fine still I would like to use another Select widget to filter the tabs even further, in that case by product. I tried few different approaches and none worked out, if someone could help me out please.
I added the second Select widget at the end of the code, to give a better idea what I have in mind.
d1 = {'table1' : [['Product1', 0], ['Product2',0]],
'table2': [['Product1', 1], ['Product2',1]],
'table3': [['Product1', 0], ['Product2',3]]}
dfs = {k:pd.DataFrame(v, columns=['Product', 'value']) for k, v in zip(d1.keys(), [d1[t] for t in d1.keys()])}
def fDP(df):
tables = []
for s in df.keys():
src = ColumnDataSource(df[s])
cls = [TableColumn(field=c, title=c) for c in df[s].columns]
dt = DataTable(source=src,columns=cls, width=600, height=200,editable=True)
tables.append(dt)
return tables
plist = list(dfs['table1']['Product'].unique())
tables1 = fDP(dfs)
panels1 = {t: Panel(child=p, title='') for t, p in zip(dfs.keys(), tables1)}
tabs1 = Tabs(tabs=[x for x in panels1.values()], align='start', width=10)
ls = [x for x in dfs.keys()]
sel1 = Select(title='Select Check:', align='start', value=ls[0], options=ls, width=195, margin = (15, 5, 0, 0))
colT = column([sel1, tabs1], spacing=-26)
sel1.js_on_change('value', CustomJS(args={'sel':sel1, 'tab':tabs1, 'diPanel':panels1}
,code='''
var sv = sel.value
tab.tabs = [diPanel[sv]]
'''))
show(colT)
selP = Select(title='Select Product:', align='start', value=plist[0], options=plist, width=195, margin = (15, 5, 0, 0))

The code below is an example how you could solve it (made for bokeh v2.1.1)
The basic idea is to pass the original dataframe to the callback, then filter that data based on current dropdown values and create a new data dictionary that you then assign to the table data source.
For more practical cases (more data) I would recommend using the dataframe-js JS library.
In that case you need to use Bokeh preamble or postamble template approach like explained in this post to be able to attach this library to your generated HTML. Having dataframe-js in your code allows you to use many basic Python Pandas functions transferred to JS domain and this is more than enough for what you need.
import pandas as pd
from bokeh.models import ColumnDataSource, DataTable, TableColumn, Tabs, Select, Panel, Row, Column, CustomJS
from bokeh.plotting import show
d1 = {'table1' : [['Product1', 11], ['Product2',12]],
'table2': [['Product1', 21], ['Product2',22]],
'table3': [['Product1', 31], ['Product2',32]]}
dfs = {k:pd.DataFrame(v, columns=['Product', 'value']) for k, v in zip(d1.keys(), [d1[t] for t in d1.keys()])}
def fDP(df):
tables = []
for s in df.keys():
src = ColumnDataSource(df[s])
cls = [TableColumn(field=c, title=c) for c in df[s].columns]
dt = DataTable(source=src,columns=cls, width=600, height=200,editable=True)
tables.append(dt)
return tables
plist = list(dfs['table1']['Product'].unique())
plist.insert(0, ' ')
tables1 = fDP(dfs)
panels1 = {t: Panel(child=p, title='') for t, p in zip(dfs.keys(), tables1)}
tabs1 = Tabs(tabs=[x for x in panels1.values()], align='start', width=10)
ls = [x for x in dfs.keys()]
sel1 = Select(title='Select Check:', align='start', value=ls[0], options=ls, width=195, margin = (15, 5, 0, 0))
selP = Select(title='Select Product:', align='start', value=plist[0], options=plist, width=195, margin = (15, 5, 0, 0))
colT = Column(Row(sel1, selP), tabs1, spacing=-26)
sel1.js_on_change('value', CustomJS(args={'sel':sel1, 'selP':selP, 'tab':tabs1, 'diPanel':panels1}
,code='''
var sv = sel.value
tab.tabs = [diPanel[sv]]
selP.value = ' '
'''))
ds = ColumnDataSource(d1)
selP.js_on_change('value', CustomJS(args={'selT':sel1, 'selP':selP, 'tab':tabs1, 'diPanel':panels1, 'ds': ds}
,code='''
var tb_name = selT.value
var p_name = selP.value
var c_tb_ds = diPanel[tb_name].child.source // table datasource to update
var c_tb_ds_data = ds.data[tb_name] // original data to work with
var new_data = {}
var p_index = null
var keys = Object.keys(c_tb_ds.data)
if (index > -1) {
keys.splice(index, 1); // 1 means remove one item only => index
}
for (var k in keys){
new_data[keys[k]] = []
}
for (var i in c_tb_ds_data) {
if(p_index == null) {
for (var j in keys) {
if(c_tb_ds_data[i][j] == p_name) {
if(c_tb_ds_data[i][j].includes('Product')) {
p_index = i
}
}
}
}
}
if(p_index != null) {
for (var j in keys) {
new_data[keys[j]].push(c_tb_ds_data[p_index][j])
}
}
c_tb_ds.data = new_data // update table data source'''))
show(colT)
Result:

How to remove bracket from JSON in Flask

I want to remove one fo them bracket from JSON in Flask. I want to get result from database and convert to JSON.
i=0
a = []
for food in sorted_similar_food:
if i==0:
i = i+1
else:
name = get_title_from_index(food[0])
name = str(name)
db_cursor.execute("SELECT * FROM recipe where name = " + "'" + name + "'")
r = [dict((db_cursor.description[i][0], value)
for i, value in enumerate(row)) for row in db_cursor.fetchall()]
a.append( r)
return jsonify({'cursor': a})
AND my result JSON
{ "cursor": [
[
{
"id": 3,
"image": "https://firebasestorage.googleapis.com/v0/b/cricket-17449.appspot.com/o/manti.jpg?alt=media&token=d11a65ec-6486-4b24-a54a-2840ce4fdc",
"ind": "kiyma yumurta sogan un",
"name": "manti",
"recip": "Ge"
}
],
....
]}
There are two brackets. I want to one of them. I should remove one of them.

You start with an empty list a=[]. Inside your loop, you append a list of DB results (r is another list). What you want to do is a.extend(r). That way the elements inside r are appended to a and you end up with just the a list.

What is the most efficient way to a multiple variable in dictionary in python?

this my code, i'm looking, is other way to code this in most efficient way?
i have multiple variables and inserted to the dictionary.
please feel to suggest and other options like array and etc will do.
def momentEndSpan(span_type,max_combo,length):
if "simply supported" == span_type:
q = max_combo
force = {}
RA = {"PA" : q*length/2}
RB = {"PB" : q*length/2}
RA_moment = {"MA" : 0}
R_mid_moment = {"Mmid": (q*math.pow(length,2))/8 }
RB_moment = { "MB" : 0}
force.update(RA)
force.update(RB)
force.update(RA_moment)
force.update(R_mid_moment)
force.update(RB_moment)
return force
elif "one end continuous" == span_type:
q = max_combo
x = (3/8)*length
force = {}
RA = {"Phinge" : 3*q*length/8}
RB = {"Pfixed" : 5*q*length/8}
RA_moment = {"Mhinge" : 0}
R_mid_moment = {"Mmid": (q*math.pow(length,2))*(9/128) }
RB_moment = { "MB" : -1*(q*math.pow(length,2))/8 }
force.update(RA)
force.update(RB)
force.update(RA_moment)
force.update(R_mid_moment)
force.update(RB_moment)
return force
Thank you very much

The "More Pythonic" way is to create one dictionary and update once.
q = max_combo
force = {}
if "simply supported" == span_type:
new = {"PA" : q*length/2,
"PB" : q*length/2,
"MA" : 0, "Mmid": (q*math.pow(length,2))/8,
"MB" : 0}
elif "one end continuous" == span_type:
x = (3/8)*length
new = {"Phinge" : 3*q*length/8,
"Pfixed" : 5*q*length/8,
"Mhinge" : 0,
"Mmid": (q*math.pow(length,2))*(9/128),
"MB" : -1*(q*math.pow(length,2))/8 }
force.update(new)
Also, note that if the force dictionary doesn't contain any previously defined items you can simply return the new and/or just continue to update the new in your next operations if there are any. Or just use name force instead of new.
q = max_combo
if "simply supported" == span_type:
force = {...}
elif "one end continuous" == span_type:
x = (3/8)*length
force = {...}

Adding to JSON in Python and converting to an object

I have a JSON array shown below.
[
"3D3iAR9M4HDETajfD79gs9BM8qhMSq5izX",
"35xfg4UnpEJeHDo55HNwJbr1V3G1ddCuVA"
]
I would like to add a value in the form of the string (self.tx_amount_5) so I get a JSON OBJECT something like this:
{
"3D3iAR9M4HDETajfD79gs9BM8qhMSq5izX" : 100000
"35xfg4UnpEJeHDo55HNwJbr1V3G1ddCuVA" : 100000
}
The part of code that has generated the first JSON array is:
r = requests.get('http://api.blockcypher.com/v1/btc/main/addrs/A/balance')
balance = r.json()['balance']
with open("Entries#x1.csv") as f,open("winningnumbers.csv") as nums:
nums = set(imap(str.rstrip, nums))
r = csv.reader(f)
results = defaultdict(list)
for row in r:
results[sum(n in nums for n in islice(row, 1, None))].append(row[0])
self.number_matched_0 = results[0]
self.number_matched_1 = results[1]
self.number_matched_2 = results[2]
self.number_matched_3 = results[3]
self.number_matched_4 = results[4]
self.number_matched_5 = results[5]
self.number_matched_5_json = json.dumps(self.number_matched_5, sort_keys = True, indent = 4)
print(self.number_matched_5_json)
if len(self.number_matched_3) == 0:
print('Nobody matched 3 numbers')
else:
self.tx_amount_3 = int((balance*0.001)/ len(self.number_matched_3))
if len(self.number_matched_4) == 0:
print('Nobody matched 4 numbers')
else:
self.tx_amount_4 = int((balance*0.1)/ len(self.number_matched_4))
if len(self.number_matched_5) == 0:
print('Nobody matched 3 numbers')
else:
self.tx_amount_5 = int((balance*0.4)/ len(self.number_matched_5))

If I understand correctly, you can create the dictionary like this:
import json
s="""[
"3D3iAR9M4HDETajfD79gs9BM8qhMSq5izX",
"35xfg4UnpEJeHDo55HNwJbr1V3G1ddCuVA"
]"""
d = {el: self.tx_amount_5 for el in json.loads(s)}
print(d)
which produces
{'3D3iAR9M4HDETajfD79gs9BM8qhMSq5izX': 100000,
'35xfg4UnpEJeHDo55HNwJbr1V3G1ddCuVA': 100000}

re reading a csv file in python without loading it again

I made the following code which works but I want to improve it. I don't want to re-read the file, but if I delete sales_input.seek(0) it won't iterate throw each row in sales. How can i improve this?
def computeCritics(mode, cleaned_sales_input = "data/cleaned_sales.csv"):
if mode == 1:
print "creating customer.critics.recommendations"
critics_output = open("data/customer/customer.critics.recommendations",
"wb")
ID = getCustomerSet(cleaned_sales_input)
sales_dict = pickle.load(open("data/customer/books.dict.recommendations",
"r"))
else:
print "creating books.critics.recommendations"
critics_output = open("data/books/books.critics.recommendations",
"wb")
ID = getBookSet(cleaned_sales_input)
sales_dict = pickle.load(open("data/books/users.dict.recommendations",
"r"))
critics = {}
# make critics dict and pickle it
for i in ID:
with open(cleaned_sales_input, 'rb') as sales_input:
sales = csv.reader(sales_input) # read new
for j in sales:
if mode == 1:
if int(i) == int(j[2]):
sales_dict[int(j[6])] = 1
else:
if int(i) == int(j[6]):
sales_dict[int(j[2])] = 1
critics[int(i)] = sales_dict
pickle.dump(critics, critics_output)
print "done"
cleaned_sales_input looks like
6042772,2723,3546414,9782072488887,1,9.99,314968
6042769,2723,3546414,9782072488887,1,9.99,314968
...
where number 6 is the book ID and number 0 is the customer ID
I want to get a dict wich looks like
critics = {
CustomerID1: {
BookID1: 1,
BookID2: 0,
........
BookIDX: 0
},
CustomerID2: {
BookID1: 0,
BookID2: 1,
...
}
}
or
critics = {
BookID1: {
CustomerID1: 1,
CustomerID2: 0,
........
CustomerIDX: 0
},
BookID1: {
CustomerID1: 0,
CustomerID2: 1,
...
CustomerIDX: 0
}
}
I hope this isn't to much information

Here are some suggestions:
Let's first look at this code pattern:
for i in ID:
for j in sales:
if int(i) == int(j[2])
notice that i is only being compared with j[2]. That's its only purpose in the loop. int(i) == int(j[2]) can only be True at most once for each i.
So, we can completely remove the for i in ID loop by rewriting it as
for j in sales:
key = j[2]
if key in ID:
Based on the function names getCustomerSet and getBookSet, it sounds as if
ID is a set (as opposed to a list or tuple). We want ID to be a set since
testing membership in a set is O(1) (as opposed to O(n) for a list or tuple).
Next, consider this line:
critics[int(i)] = sales_dict
There is a potential pitfall here. This line is assigning sales_dict to
critics[int(i)] for each i in ID. Each key int(i) is being mapped to the very same dict. As we loop through sales and ID, we are modifying sales_dict like this, for example:
sales_dict[int(j[6])] = 1
But this will cause all values in critics to be modified simultaneously, since all keys in critics point to the same dict, sales_dict. I doubt that is what you want.
To avoid this pitfall, we need to make copies of the sales_dict:
critics = {i:sales_dict.copy() for i in ID}
def computeCritics(mode, cleaned_sales_input="data/cleaned_sales.csv"):
if mode == 1:
filename = 'customer.critics.recommendations'
path = os.path.join("data/customer", filename)
ID = getCustomerSet(cleaned_sales_input)
sales_dict = pickle.load(
open("data/customer/books.dict.recommendations", "r"))
key_idx, other_idx = 2, 6
else:
filename = 'books.critics.recommendations'
path = os.path.join("data/books", filename)
ID = getBookSet(cleaned_sales_input)
sales_dict = pickle.load(
open("data/books/users.dict.recommendations", "r"))
key_idx, other_idx = 6, 2
print "creating {}".format(filename)
ID = {int(item) for item in ID}
critics = {i:sales_dict.copy() for i in ID}
with open(path, "wb") as critics_output:
# make critics dict and pickle it
with open(cleaned_sales_input, 'rb') as sales_input:
sales = csv.reader(sales_input) # read new
for j in sales:
key = int(j[key_idx])
if key in ID:
other_key = int(j[other_idx])
critics[key][other_key] = 1
critics[key] = sales_dict
pickle.dump(dict(critics), critics_output)
print "done"

#unutbu's answer is better but if you are stuck with this structure you can put the whole file in memory:
sales = []
with open(cleaned_sales_input, 'rb') as sales_input:
sales_reader = csv.reader(sales_input)
[sales.append(line) for line in sales_reader]
for i in ID:
for j in sales:
#do stuff