We Keep Coding

How to cache bokeh plots using redis

I'm using bokeh server to render a timeseries graph over a map. As the timeseries progresses, the focus of the map moves.
The code below works, but each progression creates a call that goes off to the google api (GMAP) to get the backdrop. This then takes time to render. At points where the timeseries has shifted the focus a few times in quick succession, the backdrop hasn't had time to render before it is updated.
I've been trying to work out if/how these requests can be made in advance, cached (using redis), enabling the user is able to view the cache with all data already loaded for each tick on the timeseries.
main.py
import settings
from bokeh.plotting import figure, gmap
from bokeh.embed import components
from bokeh.models import CustomJS, ColumnDataSource, Slider, GMapOptions, GMapPlot, Range1d, Button
from bokeh.models.widgets import DataTable, TableColumn, HTMLTemplateFormatter
from bokeh.layouts import column, row, gridplot, layout
from bokeh.io import show, export_png, curdoc
from filehandler import get_graph_data
"""
Get arguments from request
"""
try:
args = curdoc().session_context.request.arguments
pk = int(args.get('pk')[0])
except:
pass
"""
get data for graph from file and initialise variables
"""
#load data into dictionary from file referenced by pk
data_dict = get_graph_data(pk)
no_of_markers = data_dict.get('markers')
length_of_series = data_dict.get('length')
series_data = data_dict.get('data') #lat/lon position of each series at each point in time
series_names = series_data.get('series_names') #names of series
range_x_axis = data_dict.get('xaxis') #min/max lat co-ords
range_y_axis = data_dict.get('yaxis') #min/max lon co-ords
"""
Build data
"""
graph_source = ColumnDataSource(series_data)
"""
Build markers to show current location
"""
markers = ColumnDataSource(data=dict(lon=[], lat=[]))
"""
Build mapping layer
"""
def create_map_backdrop(centroid, zoom, tools):
"""
Create the map backdrop, centered on the starting point
Using GoogleMaps api
"""
map_options = GMapOptions(lng=centroid[1],
lat=centroid[0],
map_type='roadmap',
zoom=zoom,
)
return gmap(google_api_key=settings.MAP_KEY,
map_options=map_options,
tools=tools,
)
#set map focus
centroid = (graph_source.data['lats'][0][0],
graph_source.data['lons'][0][0],
)
"""
Build Plot
"""
tools="pan, wheel_zoom, reset"
p = create_map_backdrop(centroid, 18, tools)
p.multi_line(xs='lons',
ys='lats',
source=graph_source,
line_color='color',
)
p.toolbar.logo = None
p.circle(x='lon', y='lat', source=markers)
"""
User Interactions
"""
def animate_update():
tick = slider.value + 1
slider.value = tick
def slider_update(attr, old, new):
"""
Updates all of the datasources, depending on current value of slider
"""
start = timer()
if slider.value>series_length:
animate()
else:
tick = slider.value
i=0
lons, lats = [], []
marker_lons, marker_lats = [], []
while i < no_of_markers:
#update lines
lons.append(series_data['lons'][i][0:tick])
lats.append(series_data['lats'][i][0:tick])
#update markers
marker_lons.append(series_data['lons'][i][tick])
marker_lats.append(series_data['lats'][i][tick])
#update iterators
i += 1
#update marker display
markers.data['lon'] = marker_lons
markers.data['lat'] = marker_lats
#update line display
graph_source.data['lons'] = lons
graph_source.data['lats'] = lats
#set map_focus
map_focus_lon = series_data['lons'][tick]
map_focus_lat = series_data['lats'][tick]
#update map focus
p.map_options.lng = map_focus_lon
p.map_options.lat = map_focus_lat
slider = Slider(start=0, end=series_length, value=0, step=5)
slider.on_change('value', slider_update)
callback_id = None
def animate():
global callback_id
if button.label == "► Play":
button.label = "❚❚ Pause"
callback_id = curdoc().add_periodic_callback(animate_update, 1)
else:
button.label = "► Play"
curdoc().remove_periodic_callback(callback_id)
button = Button(label="► Play", width=60)
button.on_click(animate)
"""
Display plot
"""
grid = layout([[p, data_table],
[slider, button],
])
curdoc().add_root(grid)
I've tried caching the plot data (p), but it looks like this is persisted before the call to the google api is made.
I've explored caching the map tiles direct from the api and then stitching them into the plot as a background image (using bokeh ImageURL), but I can't get ImageUrl to recognise the in-memory image.
The server documentation suggests that redis can be used as a backend so I wondered whether this might speed thing up, but when I try to start it bokeh serve myapp --allow-websocket-origin=127.0.0.1:5006 --backend=redis I get --backend is not a recognised command.
Is there a way to either cache the fully rendered graph (possibly the graph document itself), whilst retaining the ability for users to interact with the plot; or to cache the gmap plot once it has been rendered and then add it to the rest of the plot?

If this was standalone Bokeh content (i.e. not a Bokeh server app) then you serialize the JSON representation of the plot with json_items and re-hydrate it explicitly in the browser with Bokeh.embed_items. That JSON could potentially be stored in Redis, and maybe that would be relevant. But a Bokeh server is not like that. After the initial session creation, there is never any "whole document" to store or cache, just a sequence of incremental, partial updates that happen over a websocket protocol. E.g. the server says "this specific data source changed" and the browser says "OK I should recompute bounds and re-render".
That said, there are some changes I would suggest.
The first is that you should not update CDS columns one by one. You should not do this:
# BAD
markers.data['lon'] = marker_lons
markers.data['lat'] = marker_lats
This will generate two separate update events and two separate re-render requests. Apart from the extra work this causes, it's also the case that the first update is guaranteed to have mismatched old/new coordinates. Instead, you should always update CDS .data dict "atomically", in one go:
source.data = new_data_dict
Addtionally, you might try curdoc().hold to collect updates into fewer events.

Python Bokeh Google Maps Location History: multiline() missing 2 required positional arguments: 'xs' and 'ys'

I have written a script which can use the location history from Google and plot this on a hmtl website:
import json
from datetime import datetime
from bokeh.io import output_file, show
from bokeh.models import ColumnDataSource, GMapOptions
from bokeh.plotting import gmap
#place your history file in your project deirectory
with open("Standortverlauf.json") as f:
data = json.load(f)
print("JSON Load Successfull!")
lats = []
longs = []
#retrieving the lats and longs
for entry in data["locations"]:
ts = int(entry["timestampMs"][:10])
time = datetime.utcfromtimestamp(ts)
# place your start date as unix timestamp below
if int(entry["timestampMs"][:10]) > 1554336000:
lats.append(entry["latitudeE7"]/(10**7))
longs.append(entry["longitudeE7"]/(10**7))
print(len(lats))
output_file("gmap.html")
map_options = GMapOptions(lat=37.81, lng=-122.45, map_type="roadmap", zoom=12)
# For GMaps to function, Google requires you obtain and enable an API key:
# https://developers.google.com/maps/documentation/javascript/get-api-key
# or: https://console.cloud.google.com/apis/credentials?project=hybrid-zephyr-271316 "Anmeldedaten", "Anmeldedaten erstellen", "API-Schlüssel erstellen"
# Replace the value below with your personal API key
p = gmap("API key", map_options, title="Location History",
tools = "box_zoom, pan, wheel_zoom", width=1500, height=700)
source = ColumnDataSource(
data=dict(lat=lats,
lon=longs)
)
# change color and size of the dots below
p.circle(x="lon", y="lat", size=6, fill_color="red", fill_alpha=0.8, source=source)
#This adds lines but can take much longer
p.multi_line(x='lon', y='lat', source=source, color='red', line_width=3)
show(p)
The code works fine apart from the multi_line line which leads to following error:
JSON Load Successfull!
6373
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-d49b67550055> in <module>
48 p.circle(x="lon", y="lat", size=6, fill_color="red", fill_alpha=0.8, source=source)
49 #This adds lines but can take much longer
---> 50 p.multi_line(x='lon', y='lat', source=source, color='red', line_width=3)
51 show(p)
TypeError: multiline() missing 2 required positional arguments: 'xs' and 'ys'
Can you help me how I could debug the line and have connecting lines between my points?
Thank you for your help!

Bokeh attribute update doesn't update Model without removing a model from curdoc

The below code is a minimal example of an issues where a bokeh model doesn't update when it's attribute is set via a callback. I've found that removing and adding back a model object (not even the suspect one) from the layout of the curdoc forces it to refresh. I've shown this via the first button press.
Is there a more elegant way to force bokeh to redraw the figure?
The example is for DataTable.columns.formatter, but I've noticed that this applies to other model attributes as well (including axis ranges, where I've seen a workaround involving setting the range explicitly at figure creation to allow updates).
from bokeh.models.widgets import Dropdown, RadioButtonGroup, CheckboxGroup, \
Toggle, DataTable, TableColumn, NumberFormatter
from bokeh.plotting import figure, curdoc, ColumnDataSource
from bokeh.layouts import column, layout
def update_format(attr, old, new):
if toggle_commas.active == 1:
(t.columns[1].formatter)
# remove the commas
t.columns[1].formatter = NumberFormatter(format='0,0.[00]')
# show that it updates the actual attribute
print(t.columns[1].formatter)
del doc_layout.children[-1]
doc_layout.children.insert(1, toggle_commas)
else:
# change the formatter back and note that it doesn't update the table unless you remove and add something
(t.columns[1].formatter)
# remove the commas
t.columns[1].formatter = NumberFormatter(format='0.[00]')
# show that it updates the actual attribute
print(t.columns[1].formatter)
table_data = dict(
percentiles=['min', '1st', '5th', '10th', '25th', '50th',
'75th', '90th', '95th', '99th', 'max', '', 'mean', 'std'],
values=[i for i in range(1000, 1014)]
)
table_source = ColumnDataSource(table_data)
table_columns = [
TableColumn(field="percentiles", title="Percentile"),
TableColumn(field="values", title="Value", formatter=NumberFormatter(format='0.[00]'))
]
t = DataTable(source=table_source, columns=table_columns, width=400, height=600,
name='pct_table')
toggle_commas = Toggle(label='Commas', active=False)
toggle_commas.on_change('active', update_format)
doc_layout = layout(t, toggle_commas)
curdoc().add_root(doc_layout)

hide annotations in bokeh

I have a Bokeh plot where I add some data in the form of a LabelSet and BoxAnnotation as overlay, but I want to be able to dynamically enable/disable this overlay.
I can enable/hide some of the lines in the plot already, but the system for the Annotations seems to be different. I've got so far already
Initialising
from ipywidgets import interact
from bokeh.plotting import figure as bf
from bokeh.layouts import layout as bl
from bokeh.models import Toggle, BoxAnnotation, CustomJS
from bokeh.io import push_notebook, show, output_notebook
output_notebook()
Widget generation
p = bf(title='test', x_range=(0,1), y_range=(0,1))
x = [1/3, 2/3]
y=[1/3, 2/3]
p.circle(x=x, y=y, size=15)
box = BoxAnnotation(left=None, right=0.5, fill_color='red', fill_alpha=0.1)
p.add_layout(box)
Interactivity
code = '''\
if toggle.active
box.visible = true
console.log 'enabling box'
else
box.visible = false
console.log 'disabling box'
'''
callback = CustomJS.from_coffeescript(code=code, args={})
toggle = Toggle(label="Red Box", button_type="success", callback=callback)
callback.args = {'toggle': toggle, 'box': box}
layout = bl([p], [toggle])
show(layout)
When I check the JS console, the if/else clauses get triggered as expected, so the Toggle works but the red box stays in place, both in Firefox as in IE

I think there might be some plumbing that is not hooked up on the BokehJS side to respond to visible. If so, that's a bug. Please make an issue with all this information in the Project Issue Tracker.
In the mean time, you can accomplish the same visual effect by manipulating the alpha values instead:
code = '''\
if toggle.active
box.fill_alpha = 0.1
box.line_alpha = 1
console.log 'enabling box'
else
box.fill_alpha = 0
box.line_alpha = 0
console.log 'disabling box'
'''
callback = CustomJS.from_coffeescript(code=code, args={})
toggle = Toggle(label="Red Box", button_type="success", callback=callback)
callback.args = {'toggle': toggle, 'box': box}
layout = bl([p], [toggle])
show(layout)

How do I work with images in Bokeh (Python)

For example you can plot an image in matplotlib using this code:
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
img=mpimg.imread('image.png')
plt.imshow(img)
Is something like this possible with Bokeh(0.10)?

You can use the ImageURL glyph (image_url plot method)to load images locally or from the web.
from bokeh.plotting import figure, show, output_file
output_file('image.html')
p = figure(x_range=(0,1), y_range=(0,1))
p.image_url(url=['tree.png'], x=0, y=1, w=0.8, h=0.6)
## could also leave out keywords
# p.image_url(['tree.png'], 0, 1, 0.8, h=0.6)
show(p)
One gotcha - if you graph only an image (and no other data), you'll have to explicitly set the plot ranges.
Here's the docs:
http://docs.bokeh.org/en/latest/docs/reference/models/glyphs.html#bokeh.models.glyphs.ImageURL

The earlier answer was helpful. However, I wanted an image only option without any additional object. So, adding the answer for Bokeh version 0.12.0 and removed all the grids, axes and toolbar.
from bokeh.plotting import figure, curdoc
from bokeh.models import ColumnDataSource, Range1d
bosch_logo = "static/tree.jpg"
logo_src = ColumnDataSource(dict(url = [bosch_logo]))
page_logo = figure(plot_width = 500, plot_height = 500, title="")
page_logo.toolbar.logo = None
page_logo.toolbar_location = None
page_logo.x_range=Range1d(start=0, end=1)
page_logo.y_range=Range1d(start=0, end=1)
page_logo.xaxis.visible = None
page_logo.yaxis.visible = None
page_logo.xgrid.grid_line_color = None
page_logo.ygrid.grid_line_color = None
page_logo.image_url(url='url', x=0.05, y = 0.85, h=0.7, w=0.9, source=logo_src)
page_logo.outline_line_alpha = 0
curdoc().add_root(page_logo)

Another option is to display the image in a div.:
from bokeh.io import output_notebook, show
from bokeh.models.widgets import Div
output_notebook()
div_image = Div(text="""<img src="https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/7.png" alt="div_image">""", width=150, height=150)
show(div_image)
ImageURL can't get updated dynamically with a callback. However, using a div, you can do so by treating the div_image.text as a regular Python string, for example:
from ipywidgets import interact
from bokeh.io import output_notebook, show, push_notebook
from bokeh.models.widgets import Div
output_notebook()
div_image = Div(text="""<img src="https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/1.png" alt="div_image">""", width=100, height=100)
def update(pokemon_number=1):
div_image.text = """<img src="https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/{}.png" alt="div_image">""".format(pokemon_number)
push_notebook()
show(div_image, notebook_handle=True)
interact(update, pokemon_number=[1, 4, 7])
Of course, the image source can also point to a local file.
(Tested in Python 3.7.3 and bokeh 1.2.0)

Running this example using bokeh serve is a bit more tricky. I suggest to setup working directory properly:
server_folder/
+main.py
+static/
+logo.png
.. and run bokeh serve command from directory ABOVE server_folder
bokeh serve server_folder --show
Then this code works for me
#main.py file
from bokeh.plotting import figure, curdoc
x_range = (-20,-10) # could be anything - e.g.(0,1)
y_range = (20,30)
p = figure(x_range=x_range, y_range=y_range)
#img_path = 'https://docs.bokeh.org/en/latest/_static/images/logo.png'
img_path = 'server_folder/static/logo.png'
p.image_url(url=[img_path],x=x_range[0],y=y_range[1],w=x_range[1]-x_range[0],h=y_range[1]-y_range[0])
doc = curdoc()
doc.add_root(p)

Here is a simple example that works, almost the same simple format as you requested:
from PIL import Image
import numpy as np
from bokeh.plotting import figure, output_notebook, show
output_notebook()
#load image
im = Image.open('Segment_image.png') # just replace any image that you want here
p = figure()
imarray = np.array(im.convert("RGBA"))
plotted_image = p.image_rgba(image=[imarray.view("uint32").reshape(imarray.shape[:2])], x=0, y=0, dw=imarray.shape[0], dh=imarray.shape[1])
show(p)