We Keep Coding

Interaction between multiple Select filters in Bokeh on DataTable

Using the bokeh package I am trying to filter a data table using two (or more) filters. As my data is quite large, I would like the filters to interact with each other so that they only show options that, when combined, will show data instead of an empty table.
The MRE below shows two filters working on a data set. If however you select 'United States' for the country, all potential years are shown, rather than just the two available for the US.
How can I make the select filters work with the others in this regard? I.e. in this MRE have year only show 2008 and 2007 when US is selected.
from bokeh.layouts import column
from bokeh.models import ColumnDataSource, CustomJS
from bokeh.models.widgets import DataTable, TableColumn, Select
from bokeh.plotting import save, output_file
from pandas_datareader import wb
df = wb.download(indicator='NY.GDP.PCAP.KD', country=['US', 'CA', 'MX'], start=2005, end=2008)
df = df[0:10]
df = df.reset_index()
source = ColumnDataSource(df)
original_source = ColumnDataSource(df)
columns = [
TableColumn(field="country", title="Country"),
TableColumn(field="year", title="Year"),
TableColumn(field="NY.GDP.PCAP.KD", title="NY.GDP.PCAP.KD"),
]
data_table = DataTable(source=source, columns=columns)
# callback code to be used by all the filter widgets
# requires (source, original_source, country_select_obj, year_select_obj, target_object)
combined_callback_code = """
var data = source.data;
var original_data = original_source.data;
var country = country_select_obj.value;
console.log("country: " + country);
var year = year_select_obj.value;
console.log("year: " + year);
for (var key in original_data) {
data[key] = [];
for (var i = 0; i < original_data['country'].length; ++i) {
if ((country === "ALL" || original_data['country'][i] === country) &&
(year === "ALL" || original_data['year'][i] === year)) {
data[key].push(original_data[key][i]);
}
}
}
source.change.emit();
target_obj.change.emit();
"""
# define the filter widgets, without callbacks for now
country_list = ['ALL'] + df['country'].unique().tolist()
country_select = Select(title="Country:", value=country_list[0], options=country_list)
year_list = ['ALL'] + df['year'].unique().tolist()
year_select = Select(title="Year:", value=year_list[0], options=year_list)
# now define the callback objects now that the filter widgets exist
generic_callback = CustomJS(
args=dict(source=source,
original_source=original_source,
country_select_obj=country_select,
year_select_obj=year_select,
target_obj=data_table),
code=combined_callback_code
)
# finally, connect the callbacks to the filter widgets
country_select.js_on_change('value', generic_callback)
year_select.js_on_change('value', generic_callback)
p = column(country_select,year_select,data_table)
output_file('datatable_filter.html')
save(p)

Highlighting multiple hex_tiles by hovering in bokeh

I try to visualize my data in a hex map. For this I use python bokeh and the corresponding hex_tile function in the figure class. My data belongs to one of 8 different classes, each having a different color. The image below shows the current visualization:
I would like to add the possibility to change the color of the element (and ideally all its class members) when the mouse hovers over it.
I know, that it is somewhat possible, as bokeh themselves provide the following example:
https://docs.bokeh.org/en/latest/docs/gallery/hexbin.html
However, I do not know how to implement this myself (as this seems to be a feature for the hexbin function and not the simple hex_tile function)
Currently I provide my data in a ColumnDataSource:
source = ColumnDataSource(data=dict(
r=x_row,
q=y_col,
color=colors_array,
ipc_class=ipc_array
))
where "ipc_class" describes one of the 8 classes the element belongs to.
For the mouse hover tooltip I used the following code:
TOOLTIPS = [
("index", "$index"),
("(r,q)", "(#r, #q)"),
("ipc_class", "#ipc_class")
]
and then I visualized everything with:
p = figure(plot_width=1600, plot_height=1000, title="Ipc to Hexes with colors", match_aspect=True,
tools="wheel_zoom,reset,pan", background_fill_color='#440154', tooltips=TOOLTIPS)
p.grid.visible = False
p.hex_tile('q', 'r', source=source, fill_color='color')
I would like the visualization to add a function, where hovering over one element will result in one of the following:
1. Highlight the current element by changing its color
2. Highlight multiple elements of the same class when one is hovered over by changing its color
3. Change the color of the outer line of the hex_tile element (or complete class) when the element is hovered over
Which of these features is possible with bokeh and how would I go about it?
EDIT:
After trying to reimplement the suggestion by Tony, all elements will turn pink as soon as my mouse hits the graph and the color won´t turn back. My code looks like this:
source = ColumnDataSource(data=dict(
x=x_row,
y=y_col,
color=colors_array,
ipc_class=ipc_array
))
p = figure(plot_width=800, plot_height=800, title="Ipc to Square with colors", match_aspect=True,
tools="wheel_zoom,reset,pan", background_fill_color='#440154')
p.grid.visible = False
p.hex_tile('x', 'y', source=source, fill_color='color')
###################################
code = '''
for (i in cb_data.renderer.data_source.data['color'])
cb_data.renderer.data_source.data['color'][i] = colors[i];
if (cb_data.index.indices != null) {
hovered_index = cb_data.index.indices[0];
hovered_color = cb_data.renderer.data_source.data['color'][hovered_index];
for (i = 0; i < cb_data.renderer.data_source.data['color'].length; i++) {
if (cb_data.renderer.data_source.data['color'][i] == hovered_color)
cb_data.renderer.data_source.data['color'][i] = 'pink';
}
}
cb_data.renderer.data_source.change.emit();
'''
TOOLTIPS = [
("index", "$index"),
("(x,y)", "(#x, #y)"),
("ipc_class", "#ipc_class")
]
callback = CustomJS(args=dict(colors=colors), code=code)
hover = HoverTool(tooltips=TOOLTIPS, callback=callback)
p.add_tools(hover)
########################################
output_file("hexbin.html")
show(p)
basically, I removed the tooltips from the figure function and put them down to the hover tool. As I already have red in my graph, I replaced the hover color to "pink". As I am not quite sure what each line in the "code" variable is supposed to do, I am quite helpless with this. I think one mistake may be, that my ColumnDataSource looks somewhat different from Tony's and I do not know what was done to "classifiy" the first and third element, as well as the second and fourth element together. For me, it would be perfect, if the classification would be done by the "ipc_class" variable.

Following the discussion from previous post here comes the solution targeted for the OP code (Bokeh v1.1.0). What I did is:
1) Added a HoverTool
2) Added a JS callback to the HoverTool which:
Resets the hex colors to the original ones (colors_array passed in the callback)
Inspects the index of currently hovered hex (hovered_index)
Gets the ip_class of currently hovered hex (hovered_ip_class)
Walks through the data_source.data['ip_class'] and finds all hexagons with the same ip_class as the hovered one and sets a new color for it (pink)
Send source.change.emit() signal to the BokehJS to update the model
The code:
from bokeh.plotting import figure, show, output_file
from bokeh.models import ColumnDataSource, CustomJS, HoverTool
colors_array = ["green", "green", "blue", "blue"]
x_row = [0, 1, 2, 3]
y_col = [1, 1, 1, 1]
ipc_array = ['A', 'B', 'A', 'B']
source = ColumnDataSource(data = dict(
x = x_row,
y = y_col,
color = colors_array,
ipc_class = ipc_array
))
p = figure(plot_width = 800, plot_height = 800, title = "Ipc to Square with colors", match_aspect = True,
tools = "wheel_zoom,reset,pan", background_fill_color = '#440154')
p.grid.visible = False
p.hex_tile('x', 'y', source = source, fill_color = 'color')
###################################
code = '''
for (let i in cb_data.renderer.data_source.data['color'])
cb_data.renderer.data_source.data['color'][i] = colors[i];
if (cb_data.index.indices != null) {
const hovered_index = cb_data.index.indices[0];
const hovered_ipc_class = cb_data.renderer.data_source.data['ipc_class'][hovered_index];
for (let i = 0; i < cb_data.renderer.data_source.data['ipc_class'].length; i++) {
if (cb_data.renderer.data_source.data['ipc_class'][i] == hovered_ipc_class)
cb_data.renderer.data_source.data['color'][i] = 'pink';
}
}
cb_data.renderer.data_source.change.emit();
'''
TOOLTIPS = [
("index", "$index"),
("(x,y)", "(#x, #y)"),
("ipc_class", "#ipc_class")
]
callback = CustomJS(args = dict(ipc_array = ipc_array, colors = colors_array), code = code)
hover = HoverTool(tooltips = TOOLTIPS, callback = callback)
p.add_tools(hover)
########################################
output_file("hexbin.html")
show(p)
Result:

Maybe something like this to start with (Bokeh v1.1.0):
from bokeh.plotting import figure, show
from bokeh.models import ColumnDataSource, CustomJS, HoverTool
colors = ["green", "blue", "green", "blue"]
source = ColumnDataSource(dict(r = [0, 1, 2, 3], q = [1, 1, 1, 1], color = colors))
plot = figure(plot_width = 300, plot_height = 300, match_aspect = True)
plot.hex_tile('r', 'q', fill_color = 'color', source = source)
code = '''
for (i in cb_data.renderer.data_source.data['color'])
cb_data.renderer.data_source.data['color'][i] = colors[i];
if (cb_data.index.indices != null) {
hovered_index = cb_data.index.indices[0];
hovered_color = cb_data.renderer.data_source.data['color'][hovered_index];
for (i = 0; i < cb_data.renderer.data_source.data['color'].length; i++) {
if (cb_data.renderer.data_source.data['color'][i] == hovered_color)
cb_data.renderer.data_source.data['color'][i] = 'red';
}
}
cb_data.renderer.data_source.change.emit();
'''
callback = CustomJS(args = dict(colors = colors), code = code)
hover = HoverTool(tooltips = [('R', '#r')], callback = callback)
plot.add_tools(hover)
show(plot)
Result:

Another approach is to update cb_data.index.indices to include all those indices that have ipc_class in common, and add hover_color="pink" to hex_tile. So in the CustomJS code one would loop the ipc_class column and get the indices that match the ipc_class of the currently hovered item.
In this setup there is not need to update the color column in the data source.
Code below tested used Bokeh version 3.0.2.
from bokeh.plotting import figure, show, output_file
from bokeh.models import ColumnDataSource, CustomJS, HoverTool
colors_array = ["green", "green", "blue", "blue"]
x_row = [0, 1, 2, 3]
y_col = [1, 1, 1, 1]
ipc_array = ['A', 'B', 'A', 'B']
source = ColumnDataSource(data = dict(
x = x_row,
y = y_col,
color = colors_array,
ipc_class = ipc_array
))
plot = figure(
width = 800,
height = 800,
title = "Ipc to Square with colors",
match_aspect = True,
tools = "wheel_zoom,reset,pan",
background_fill_color = '#440154'
)
plot.grid.visible = False
plot.hex_tile(
'x', 'y',
source = source,
fill_color = 'color',
hover_color = 'pink' # Added!
)
code = '''
const hovered_index = cb_data.index.indices;
const src_data = cb_data.renderer.data_source.data;
if (hovered_index.length > 0) {
const hovered_ipc_class = src_data['ipc_class'][hovered_index];
var idx_common_ipc_class = hovered_index;
for (let i = 0; i < src_data['ipc_class'].length; i++) {
if (i === hovered_index[0]) {
continue;
}
if (src_data['ipc_class'][i] === hovered_ipc_class) {
idx_common_ipc_class.push(i);
}
}
cb_data.index.indices = idx_common_ipc_class;
cb_data.renderer.data_source.change.emit();
}
'''
TOOLTIPS = [
("index", "$index"),
("(x,y)", "(#x, #y)"),
("ipc_class", "#ipc_class")
]
callback = CustomJS(code = code)
hover = HoverTool(
tooltips = TOOLTIPS,
callback = callback
)
plot.add_tools(hover)
output_file("hexbin.html")
show(p)

Bokeh datatable filtering inconsistency

Apologies but I can't make any reproducible code for this question since it's pretty inconsistent.
So I have a bokeh data table, and I'm doing some filtering with it using 4 dropdown boxes. The data table updates based on dropdown box value, and the updates were written in JS. The filtering works as expected, but strangely enough for some very specific combinations of dropdown values it does not display anything in the data table. I was wondering if it was a problem with my data, but I coerced everything to strings and it still gave me the same problem.
The updates are written here:
combined_callback_code = """
var data = source.data;
var original_data = original_source.data;
var origin = origin_select_obj.value;
var classification = classification_select_obj.value;
var currency = currency_select_obj.value;
var grade = grade_select_obj.value;
for (var key in original_data) {
data[key] = [];
for (var i = 0; i < original_data['Origin'].length; ++i) {
if ((origin === "ALL" || original_data['Origin'][i] === origin) &&
(classification === "ALL" || original_data['Classification'][i] === classification) &&
(currency === "ALL" || original_data['Currency'][i] === currency) &&
(grade === "ALL" || original_data['BrokenPct'][i] === grade)){
data[key].push(original_data[key][i]);
}
}
}
target_obj.change.emit();
source.change.emit();
"""
# define the filter widgets, without callbacks for now
origin_list = ['ALL'] + df['Origin'].unique().tolist()
origin_select = Select(title="Origin:", value=origin_list[0], options=origin_list)
classification_list = ['ALL'] + df['Classification'].unique().tolist()
classification_select = Select(title="Classification:", value=classification_list[0], options=classification_list)
currency_list = ['ALL'] + df['Currency'].unique().tolist()
currency_select = Select(title = "Currency:", value=currency_list[0], options = currency_list)
grade_list = ["ALL"] + df['BrokenPct'].unique().tolist()
grade_select = Select(title = "Grade:", value = grade_list[0], options = grade_list)
# now define the callback objects now that the filter widgets exist
generic_callback = CustomJS(
args=dict(source=source,
original_source=original_source,
origin_select_obj=origin_select,
classification_select_obj=classification_select,
currency_select_obj = currency_select,
grade_select_obj = grade_select,
target_obj=data_table),
code=combined_callback_code
)
# finally, connect the callbacks to the filter widgets
origin_select.js_on_change('value', generic_callback)
classification_select.js_on_change('value', generic_callback)
currency_select.js_on_change('value', generic_callback)
grade_select.js_on_change('value', generic_callback)

The right way do update table data in your JS callback is this way:
var data = {};
//build your data here
source.data = data;
Where source is the Bokeh ColumnDataSource of you DataTable.
You don't need to use:
source.change.emit();
You do it only when you replace only a part of you data e.g. one table column.
And if data_table is your Bokeh DataTable object then also skip doing:
target_obj.change.emit();
The table date updates automatically when you update its ColumnDataSource.
See this simple example:
from bokeh.io import show
from bokeh.layouts import widgetbox
from bokeh.models import ColumnDataSource, Slider, DataTable, TableColumn, CustomJS
source = ColumnDataSource(dict(x = list(range(6)), y = [x ** 2 for x in range(6)]))
columns = [TableColumn(field = "x", title = "x"), TableColumn(field = "y", title = "x**2")]
table = DataTable(source = source, columns = columns, width = 320)
slider = Slider(start = 1, end = 20, value = 6, step = 1, title = "i", width = 300)
callback_code = """ i = slider.value;
new_data = {"x": [1, 2, 3, 4, 5], "y": [1, 4, 9, 16, 25]}
table.source.data = new_data
table.width = 320 + i * 25; """
callback = CustomJS(args = dict(slider = slider, table = table), code = callback_code)
slider.js_on_change('value', callback)
show(widgetbox(slider, table))

Changing colors on bokeh patches plot real time

I'm trying to create a bokeh plot of the US States, and color each of the state according to some data. Now using this tutorial I managed to create this, but I also want to enhance it, and add a slider to it, to change the values displayed. For example like displaying separate years.
With the help of this tutorial, I managed to add the slider, and the underlying data does change, according to the hover text, but the colors aren't recalculated, and so the visual representation does not match the values.
This is the code I've used, from a Jupyter notebook, so anybody who wants to try can reproduce
from bokeh.io import show, output_notebook
from bokeh.models import (
ColumnDataSource,
HoverTool,
LogColorMapper,
Range1d, CustomJS, Slider
)
from bokeh.palettes import Inferno256 as palette
from bokeh.plotting import figure
from bokeh.layouts import row, widgetbox
from bokeh.sampledata.us_counties import data as counties
from bokeh.sampledata.us_states import data as states
from bokeh.sampledata.unemployment import data as unemployment
import pandas as pd
import random
output_notebook()
palette.reverse()
states_accumulated ={}
available_state_codes = states.keys()
for key, value in counties.items():
state_name = value["state"].upper()
if state_name in states.keys() and "number" not in states[state_name]:
states[state_name]["number"] = key[0]
for key,state in states.items():
state["code"] = key
state_list = []
for key,state in states.items():
state_list.append(state)
unemployment_transf = []
for key,value in unemployment.items():
unemployment_transf.append({
"State":key[0],
"County":key[1],
"Value":value
})
unemp_df = pd.DataFrame(unemployment_transf)
unemp_sum = unemp_df.groupby("State").mean()["Value"]
unemp_sum = unemp_sum.sort_index()
unemp_sum_flat = {key:value for key, value in unemp_sum.items()}
for state in state_list:
state["value"] = unemp_sum_flat[state["number"]]
state_df = pd.DataFrame(state_list)
color_mapper = LogColorMapper(palette=palette)
state_xy = (list(state_df["lons"].values),list(state_df["lats"].values))
max_x = max([max(l) for l in state_xy[0]])
max_y = max([max(l) for l in state_xy[1]])
min_x = min([min(l) for l in state_xy[0]])
min_y = min([min(l) for l in state_xy[1]])
data=dict(
x=state_xy[0],
y=state_xy[1],
name=list(state_df["name"].values),
used = list(state_df["value"].values)
)
data['1999'] = list(state_df["value"].values)
data['2000'] = [random.randrange(0,10) for i in range(len(state_xy[0]))]
source = ColumnDataSource(data)
TOOLS = "pan,wheel_zoom,reset,hover,save"
p = figure(
title="States", tools=TOOLS,
x_axis_location=None, y_axis_location=None
)
p.width=450
p.height = 450
p.x_range= Range1d(-170,-60)
p.y_range = Range1d(min_y-10,max_y+10)
p.grid.grid_line_color = None
renderer = p.patches('x', 'y', source=source,
fill_color={'field': 'used', 'transform': color_mapper},
fill_alpha=0.7, line_color="white", line_width=0.5)
hover = p.select_one(HoverTool)
hover.point_policy = "follow_mouse"
hover.tooltips = [
("Name", "#name"),
("Unemployment rate)", "#used%"),
("(Long, Lat)", "($x, $y)"),
]
callback = CustomJS(args=dict(source=source,plot=p,color_mapper = color_mapper,renderer = renderer), code="""
var data = source.data;
var year = year.value;
used = data['used']
should_be = data[String(year)]
for (i = 0; i < should_be.length; i++) {
used[i] = should_be[i];
}
""")
year_slider = Slider(start=1999, end=2000, value=1999, step=1,
title="year", callback=callback)
callback.args["year"] = year_slider
layout = row(
p,
widgetbox(year_slider),
)
show(layout)
Sample images of the plot:
What I would like to accomplish, is that when I change the slider, the colors on the plot should change. Now I think the JS callback should call some kind of redraw or recalculate, but I haven't found any documentation about it. Is there a way to do this?

append source.change.emit() to the Javascipt code to trigger the change event.

Appending source.trigger("change"); to the CustomJS seems to solve the problem, now as the slider changes, the colors change.

Choropleth world map not showing all countries

I wanted to make a choropleth world map, which shows the hits(number of searches) of a word, on a World map.
Following is the code:
import plotly
import plotly.offline
import pandas as pd
df = pd.read_excel('F:\\Intern\\csir\\1yr\\news\\region_2016_2017.xlsx')
df = df.query('keyword==["addiction"]')
scl = [[0.0, 'rgb(242,240,247)'],[0.2, 'rgb(218,218,235)'],[0.4, 'rgb(188,189,220)'],\
[0.6, 'rgb(158,154,200)'],[0.8, 'rgb(117,107,177)'],[1.0, 'rgb(84,39,143)']]
data = [dict(
type='choropleth',
colorscale=scl,
locations = df['location'],
z = df['hits'].astype(int),
locationmode = "country names",
autocolorscale = False,
reversescale = False,
marker = dict(
line = dict (
color = 'rgb(180,180,180)',
width = 0.5)),
colorbar = dict(
autotick = False,
title = 'Hits'),)]
layout = dict(
title = 'Addiction keyword 1yr analysis',
geo = dict(
showframe = False,
showcoastlines = False,
projection = dict(
type = 'Mercator'
)
)
)
fig = dict(data = data,layout = layout)
plotly.offline.plot(fig,validate=False,filename = 'd3-world-map.html')
And the plotted map is:
As one can see clearly, many countries are missing. This may be due to the fact that many countries didn't have entries which explicitly stated that they have zero hits.
I don't want to explicitly do that with my data. Is there any other way out of this? So that we can see all of the countries.
Data set can be found here.
Note that the dataset that I've linked is an .csv file whereas the file used in the program is an .xlsx version of the file.

You need to turn on country outlines under layout...
"geo":{
"countriescolor": "#444444",
"showcountries": true
},