We Keep Coding

bokeh: How to edit a df or CDS-object through box_select?

I'm trying to label a pandas-df (containing timeseries data) with the help of
a bokeh-lineplot, box_select tool and a TextInput widget in a jupyter-notebook. How can I access the by the box_select selected data points?
I tried to edit a similar problems code (Get selected data contained within box select tool in Bokeh) by changing the CustomJS to something like:
source.callback = CustomJS(args=dict(p=p), code="""
var inds = cb_obj.get('selected')['1d'].indices;
[source.data['xvals'][i] for i in inds] = 'b'
"""
)
but couldn't apply a change on the source of the selected points.
So the shortterm goal is to manipulate a specific column of source of the selected points.
Longterm I want to use a TextInput widget to label the selected points by the supplied Textinput. That would look like:
EDIT:
That's the current code I'm trying in the notebook, to reconstruct the issue:
from random import random
import bokeh as bk
from bokeh.layouts import row
from bokeh.models import CustomJS, ColumnDataSource, HoverTool
from bokeh.plotting import figure, output_file, show, output_notebook
output_notebook()
x = [random() for x in range(20)]
y = [random() for y in range(20)]
hovertool=HoverTool(tooltips=[("Index", "$index"), ("Label", "#label")])
source = ColumnDataSource(data=dict(x=x, y=y, label=[i for i in "a"*20]))
p1 = figure(plot_width=400, plot_height=400, tools="box_select", title="Select Here")
p1.circle('x', 'y', source=source, alpha=0.6)
p1.add_tools(hovertool)
source.selected.js_on_change('indices', CustomJS(args=dict(source=source), code="""
var inds = cb_obj.indices;
for (var i = 0; i < inds.length; i++) {
source.data['label'][inds[i]] = 'b'
}
source.change.emit();
""")
)
layout = row(p1)
show(layout)

The main thing to note is that BokehJS can only automatically notice updates when actual assignments are made, e.g.
source.data = some_new_data
That would trigger an update. If you update the data "in place" then BokehJS is not able to notice that. You will have to be explicit and call source.change.emit() to let BokehJS know something has been updated.
However, you should also know that you are using three different things that are long-deprecated and will be removed in the release after next.
cb_obj.get('selected')
There is no need to ever use .get You can just access properties directly:
cb_obj.selected
The ['1d'] syntax. This dict approach was very clumsy and will be removed very soon. For most selections you want the indices property of the selection:
source.selected.indices
source.callback
This is an ancient ad-hoc callback. There is a newer general mechanism for callbacks on properties that should always be used instead
source.selected.js_on_change('indices', CustomJS(...))
Note that in this case, the cb_obj is the selection, not the data source.

With the help of this guide on how to embed a bokeh server in the notebook I figured out the following minimal example for my purpose:
from random import random
import pandas as pd
import numpy as np
from bokeh.io import output_notebook, show
from bokeh.layouts import column
from bokeh.models import Button
from bokeh.plotting import figure
from bokeh.models import HoverTool, ColumnDataSource, BoxSelectTool
from bokeh.models.widgets import TextInput
output_notebook()
def modify_doc(doc):
# create a plot and style its properties
TOOLS="pan,wheel_zoom,reset"
p = figure(title = "My chart", tools=TOOLS)
p.xaxis.axis_label = 'X'
p.yaxis.axis_label = 'Y'
hovertool=HoverTool(tooltips=[("Index", "$index"), ("Label", "#label")])
source = ColumnDataSource(
data=dict(
xvals=list(range(0, 10)),
yvals=list(np.random.normal(0, 1, 10)),
label = [i for i in "a"*10]
))
p.scatter("xvals", "yvals",source=source, color="white")
p.line("xvals", "yvals",source=source)
p.add_tools(BoxSelectTool(dimensions="width"))
p.add_tools(hovertool)
# create a callback that will add a number in a random location
def callback():
inds = source.selected.indices
for i in inds:
source.data['label'][i] = label_input.value.strip()
print(source.data)
new_data = pd.DataFrame(source.data)
new_data.to_csv("new_data.csv", index=False)
# TextInput to specify the label
label_input = TextInput(title="Label")
# add a button widget and configure with the call back
button = Button(label="Label Data")
button.on_click(callback)
# put the button and plot in a layout and add to the document
doc.add_root(column(button,label_input, p))
show(modify_doc, notebook_url="http://localhost:8888")
That generates the following UI:
BTW: Due to the non-existing box_select tool for the line glyph I use a workaround by combining it with invisible scatter points.
So far so good, is there a more elegant way to access the data.source/new_data df in the notebook outside modify_doc() than exporting it within the callback?

interactive scatter highlight in bokeh

I am trying to visualise sensor output in relation to its path.
I plot path as scatter in one figure and some range of signal amplitude in the second figure. I need to visualise (highlight) a path point at which the particular reading was taken.
I started using bokeh as a backend and in general, got very good results with visualisations I need. But I am stuck on this particular interaction.
I would like to have some marker like a vertical line anchored in the middle of the figure. When I move/scroll the amplitude plot (the bottom one), I would like to highlight the point on the path plot where the reading closest to the marker line was taken.
The example code:
(I would like to anchor the marker line and add interaction between the red dot and the vertical line taking an index of the signal, which is not implemented.)
import numpy as np
import pandas as pd
from bokeh.io import output_file
from bokeh.models import ColumnDataSource, HoverTool, Span
from bokeh.plotting import figure, show
from bokeh.layouts import gridplot
output_file('interactive_path_sig.html', title="interactive path")
class InteractivePath():
def __init__(self):
x = np.arange(0, 1000, 0.5)
self.df = pd.DataFrame({"x": x,
"y": np.sin(x),
"z": np.cos(x)})
self.source = ColumnDataSource(self.df)
def plot_path(self):
plt = figure(title = "Sensor Path")
plt.scatter(x="x", y="y",source=self.source,
line_color=None, size = 6)
# TODO implement interaction instead of hard coded index
index=500 # this is where I think I need to create working callback
print("x={}, y={}".format(self.df['x'][index], self.df['y'][index]))
plt.circle(x=self.df['x'][index], y=self.df['y'][index],
fill_color="red", size=15)
hover = HoverTool()
hover.tooltips=[("index", "#index"), ("senosr","#z")]
plt.add_tools(hover)
return plt
def plot_signal(self):
plt = figure(x_range=(450, 550), title="Signal Amplitude")
plt.line(x="index", y="z", source=self.source, line_color="black", line_width=2)
# TODO implement interaction instead of hard coded index
index = 500 # I think this needs emit some singal to other plot
vline = Span(location=index, dimension='height', line_color='red', line_width=3)
plt.renderers.extend([vline])
return plt
def get_grid(self):
""" place visualisation in a grid and display"""
grid = gridplot([[self.plot_path()], [self.plot_signal()]],
sizing_mode='stretch_both',)
return grid
def vis_main(self):
""" use all visualisations"""
show(self.get_grid())
if __name__=="__main__":
vis = InteractivePath()
vis.vis_main()

So a few pointers:
I think you'll want both of those plots in the same method because the columndatasource is common between them, and you can set CustomJS behaviors between them if they're in the same scope.
The index that you're using already exists within your self.df which will be easier to interact with once it's on your plot, since you can handle it with JS plot behavior instead of going back to a python variable and reloading data.
Instead of drawing a new glyph for your 'highlighted' point, consider using the 'hover' or 'selected' functionality built in. hover_color='red' for example could replace drawing and moving another class of glyph. If you want to leave statically selected so you can generate a nice report without a mouse in a screenshot, defining a callback using the built-in selected property of ColumnDataSource
I can post some actual code blocks with more specific examples, but if any of these points is a hard stop for your actual use case, it'll drive solution.
Edit:
So I got pretty close using one class method - the issue is being able to edit the second plot from the first method, not the actual change to the ColumnDataSource itself.
def plot_it(self):
self.plot_signal = figure(x_range=(450, 550), y_range=(-1, 1), title='signal')
self.plot_signal.line(x='index', y='z', source=self.source)
self.plot_signal.segment(x0=500, y0=-2, x1=500, y1=2, source=self.source)
self.plot_path = figure(title='sensor')
self.plot_path.scatter(x='x', y='y', source=self.source, hover_color='red')
jscode='''
var data = source.data;
var plot_signal = plot_signal;
var index = cb_data.index['1d'].indices;
var xmin = 0;
var xmax = 0;
if (index > 0) {
xmin = index[0] - 50;
xmax = index[0] + 50;
plot_signal.x_range.end = xmax;
plot_signal.x_range.start = xmin;
plot_signal.change.emit();
}
hover_callback = CustomJS(args=dict(source=self.source, plot_signal=self.plot_signal), code=jscode)
hover.tooltips = [('index', '#index'), ('sensor', '#z')]
self.plot_path.add_tools(hover)
def get_grid(self):
self.plot_it()
grid = gridplot([[self.plot_path], [self.plot_signal]])
return grid
That should do everything but move the line segment. I couldn't find the segment naming convention to add plot_signal.SOMEOBJECT.x0 and .x1 but it would just get added to the if (index > 0) block just like using index[0]. I took some of the style options out because I'm transcribing from another computer.
This question on moving a line segment might give you the syntax on the segment JSON object.

How to make a Bokeh scatter plot interactive (with slider)

I'm trying to make a scatter plot in Bokeh based on the simple example code posted here.
The following code produces a working demo for a line plot:
from bokeh.layouts import column
from bokeh.models import CustomJS, ColumnDataSource, Slider
from bokeh.plotting import Figure, show
# fetch and clear the document
from bokeh.io import curdoc
curdoc().clear()
x = [x*0.005 for x in range(0, 100)]
y = x
source = ColumnDataSource(data=dict(x=x, y=y))
plot = Figure(plot_width=400, plot_height=400)
plot.line(x='x', y='y', source=source)
def callback(source=source, window=None):
data = source.data
f = cb_obj.value
x, y = data['x'], data['y']
for i in range(len(x)):
y[i] = window.Math.pow(x[i], f)
source.trigger('change')
slider = Slider(start=0.1, end=4, value=1, step=.1, title="Start week",
callback=CustomJS.from_py_func(callback))
layout = column(slider, plot)
show(layout)
It looks like this:
In this demo, when you adjust the slider and press the 'reset' icon, the plot re-draws itself based on the updated formula for y=f(x).
However, I want to make a scatter plot that changes, not a line plot.
Problem:
When I simply change plot.line in above code to plot.circle, the plot renders okay but it is static - it does not change when you shift the slider and press 'reset'. No error messages that I can see.

I found the answer in the documentation.
The final line in callback should be source.change.emit() not source.trigger('change'). I do not know the difference between these two but the latter works with circle plots.
I.e.
def callback(source=source, window=None):
data = source.data
f = cb_obj.value
x, y = data['x'], data['y']
for i in range(len(x)):
y[i] = window.Math.pow(x[i], f)
source.change.emit()

Set the zoom level of a bokeh map when using a tile provider

I've followed the example here: http://docs.bokeh.org/en/latest/docs/user_guide/geo.html#tile-providers
I got a basic map loading a GeoJSON file with a list of polygons (already projected to Web Mercator EPSG:3857) so then I could use STAMEN_TONER as a tile provider.
from bokeh.io import output_file, show
from bokeh.plotting import figure
from bokeh.tile_providers import STAMEN_TONER, STAMEN_TERRAIN
from bokeh.models import Range1d, GeoJSONDataSource
# bokeh configuration for jupyter
from bokeh.io import output_notebook
output_notebook()
# bounding box (x,y web mercator projection, not lon/lat)
mercator_extent_x = dict(start=x_low, end=x_high, bounds=None)
mercator_extent_y = dict(start=y_low, end=y_high, bounds=None)
x_range1d = Range1d(**mercator_extent_x)
y_range1d = Range1d(**mercator_extent_y)
fig = figure(
tools='pan, zoom_in, zoom_out, box_zoom, reset, save',
x_range=x_range1d,
y_range=y_range1d,
plot_width=800,
plot_height=600
)
fig.axis.visible = False
fig.add_tile(STAMEN_TERRAIN)
# the GeoJSON is already in x,y web mercator projection, not lon/lat
with open('/path/to/my_polygons.geojson', 'r') as f:
my_polygons_geo_json = GeoJSONDataSource(geojson=f.read())
fig.multi_line(
xs='xs',
ys='ys',
line_color='black',
line_width=1,
source=my_polygons_geo_json
)
show(fig)
However I am not able to set a default zoom level for the tiles. I thought it could have been a tool setting (http://docs.bokeh.org/en/latest/docs/user_guide/tools.html) but in there I can not find a default value for the zoom capabilities.
How can I set a default value for the zoom level of the tiles?

Old question but answering if someone would have the same problem. Set range for your map and this way you can zoom into the desired area on load. Below example with Papua New Guinea
p = figure(title="PNG Highlands Earthquake 7.5 Affected Villages",y_range=(-4.31509, -7.0341),x_range=( 141.26667, 145.56598))
p.xaxis.axis_label = 'longitude'
p.yaxis.axis_label = 'latitude'

I've just run into this issue myself, and found a good solution that should work under most circumstances. This requires making sure the data and the x_range/y_range to be projected properly (I used Proj and transform from pyproj but I'm sure there are other packages that will work the same).
Import modules:
import pandas as pd
import numpy as np
from pyproj import Proj, transform
import datashader as ds
from datashader import transfer_functions as tf
from datashader.bokeh_ext import InteractiveImage
from datashader.utils import export_image
from datashader.colors import colormap_select, Greys9, Hot, viridis, inferno
from IPython.core.display import HTML, display
from bokeh.plotting import figure, output_notebook, output_file, show
from bokeh.tile_providers import CARTODBPOSITRON
from bokeh.tile_providers import STAMEN_TONER
from bokeh.tile_providers import STAMEN_TERRAIN
from bokeh.embed import file_html
from functools import partial
output_notebook()
Read in data (I took a few extra steps to try and clean the coordinates since I'm working with an extremely messy dataset that contains NaN and broken text in the coordinates columns):
df = pd.read_csv('data.csv', usecols=['latitude', 'longitude'])
df.apply(lambda x: pd.to_numeric(x,errors='coerced')).dropna()
df = df.loc[(df['latitude'] > - 90) & (df['latitude'] < 90) & (df['longitude'] > -180) & (df['longitude'] < 180)]
Reproject data:
# WGS 84
inProj = Proj(init='epsg:4326')
# WGS84 Pseudo Web Mercator, projection for most WMS services
outProj = Proj(init='epsg:3857')
df['xWeb'],df['yWeb'] = transform(inProj,outProj,df['longitude'].values,df['latitude'].values)
Reproject the x_range, y_range. This is critical as these values set the extent of the bokeh map - the coordinates of these values need to match the projection. To make sure you have the correct coordinates, I suggest using http://bboxfinder.com to create a bounding box AOI and get the correct min/max and min/max coordinates (making sure EPSG:3857 - WGS 84/Pseudo-Mercator is selected). Using this method, just copy the coodinates next to "box" - these are in the order of minx,miny,maxx,maxy and should then be reordered as minx,maxx,miny,maxy (x_range = (minx,maxx))(y_range=(miny,maxy)):
world = x_range, y_range = ((-18706892.5544, 21289852.6142), (-7631472.9040, 12797393.0236))
plot_width = int(950)
plot_height = int(plot_width//1.2)
def base_plot(tools='pan,wheel_zoom,save,reset',plot_width=plot_width,
plot_height=plot_height, **plot_args):
p = figure(tools=tools, plot_width=plot_width, plot_height=plot_height,
x_range=x_range, y_range=y_range, outline_line_color=None,
min_border=0, min_border_left=0, min_border_right=0,
min_border_top=0, min_border_bottom=0, **plot_args)
p.axis.visible = False
p.xgrid.grid_line_color = None
p.ygrid.grid_line_color = None
return p
options = dict(line_color=None, fill_color='blue', size=1.5, alpha=0.25)
background = "black"
export = partial(export_image, export_path="export", background=background)
cm = partial(colormap_select, reverse=(background=="white"))
def create_image(x_range, y_range, w=plot_width, h=plot_height):
cvs = ds.Canvas(plot_width=w, plot_height=h, x_range=x_range, y_range=y_range)
agg = cvs.points(df, 'xWeb', 'yWeb')
magma = ['#3B0F6F', '#8C2980', '#DD4968', '#FD9F6C', '#FBFCBF']
img = tf.shade(agg, cmap=magma, how='eq_hist') # how='linear', 'log', 'eq_hist'
return tf.dynspread(img, threshold=.05, max_px=15)
p = base_plot()
p.add_tile("WMS service")
#used to export image (without the WMS)
export(create_image(*world),"TweetGeos")
#call interactive image
InteractiveImage(p, create_image)

The notion of a zoom "level" only applies to GMapPlot and there only because google controls the presentation of the maps very carefully, and that is the API they provide. All other Bokeh plots have explicitly user-settable x_range and y_range properties. You can set the start and end of these ranges to be whatever you want, and the plot will display the corresponding area defined by those bounds.

Bokeh Sliders associated with a plot (without bokeh server)

I'm trying to plot my 1D PDE in time using bokeh. I have a nparray U where each line is a time slice( dt ) and each column is a space slice (dx), therefore U[0] is my initial condition and U[n] is my n-th iteration.
Of course if I ask to plot(x,U[t]) for every t, I get the correct plot (as good as my approximations can be ;-), but I would like associate this with a time slider for interactivity (of course in future I would like to "play" it as animation ;-)
For now I would like to avoid to use bokeh server, since I want that this HTML be an stand alone "application"
The issue here is that callbacks just don't work, or at least I'm not being able to update the graph. I don't want to run a "bokeh" server since all my data is already inside a numpy data structure (U array). Every line is a iteration in time and every column is a dx.
#!/usr/bin/env python
from __future__ import division
from bokeh.models import ColumnDataSource, HBox, VBoxForm, HoverTool
from bokeh.models.widgets import Slider, TextInput
from bokeh.plotting import Figure, output_file, show
import numpy as np
def linearconv(nx,c=1,sigma=0.5,tmax=1,xmax=3):
nt=int((tmax/xmax)*((nx-1)/(c*sigma))+1) # Time Grid
x,dx=np.linspace(0,xmax,nx,retstep=True)
t,dt=np.linspace(0,tmax,nt,retstep=True)
# Initial conditions
#
# u=2 if 0.5 <= x <= 1
# u=1 everywhere else in the support
U = np.ones((nt,nx))
U[0][np.where((.5<=x) & (x<=1))]=2
# Calculate the wave over the time
for n in range(1,nt):
for i in range(1,nx):
U[n][i]= U[n-1][i] - c*dt/dx* ( U[n-1][i]-U[n-1][i-1] )
return U,x,t,dx,dt,nt
def prepareplot(height=400, width=400,title="Wave #"):
plot = Figure(plot_height=height, plot_width=width, title=title,
tools="crosshair,pan,reset,resize,save,wheel_zoom")
return plot
def update_ttime(attrname, old, new):
plot.title = "Wave #{}s".format(title.value)
def update_graph(attrname, old, new):
# Get the current slider values
t = time.value
source = ColumnDataSource(data=dict(x=x, t=U[t]))
plot.line('x','t', source=source)
plot.line('x','t', source=source)
nx=101
# Set up data
U,x,t,dx,dt,nt = linearconv(nx)
plot = prepareplot()
time = Slider(title="Time", value=t[-1], start=t[0], end=t[-1], step=dt)
ttime = TextInput(title="Time", value="{}".format(t[-1]))
source = ColumnDataSource(data=dict(x=x, t=U[-1]))
plot.line('x','t', source=source)
# Set up callbacks
ttime.on_change('value', update_ttime)
time.on_change('value', update_graph)
# Setup layouts
inputs = VBoxForm(children=[ttime,time])
layout = HBox(children=[inputs,plot], width=800)
# Plot the plot
output_file("sli.html")
show(layout)