We Keep Coding

How to build an Altair layered chart w/ dual axis?

Description
This code shows three Altair charts:
scatter
rate
line_plot
Goal
The goal is to combine all charts into a layered chart w/ these specifications:
show the y-axis for both scatter and rate (ie. dual axis chart)
facet by Series
show the line_plot.
Code
import altair as alt
from vega_datasets import data
import pandas as pd
source = data.anscombe().copy()
source['line-label'] = 'x=y'
source = pd.concat([source,source.groupby('Series').agg(x_diff=('X','diff'), y_diff=('Y','diff'))],axis=1)
source['rate'] = source.y_diff/source.x_diff
source['rate-label'] = 'rate of change'
source['line-label'] = 'line y=x'
source_linear = source.groupby(by=['Series']).agg(x_linear=('X','max'), y_linear=('X', 'max')).reset_index().sort_values(by=['Series'])
source_origin = source_linear.copy()
source_origin['y_linear'] = 0
source_origin['x_linear'] = 0
source_linear = pd.concat([source_origin,source_linear]).sort_values(by=['Series'])
source = source.merge(source_linear,on='Series').drop_duplicates()
scatter = alt.Chart(source).mark_circle(size=60, opacity=0.60).encode(
x=alt.X('X', title='X'),
y=alt.Y('Y', title='Y'),
color='Series:N',
tooltip=['X','Y','rate']
)
line_plot = alt.Chart(source).mark_line(color= 'black', strokeDash=[3,8]).encode(
x=alt.X('x_linear', title = ''),
y=alt.Y('y_linear', title = ''),
shape = alt.Shape('line-label', title = 'Break Even'),
color = alt.value('black')
)
rate = alt.Chart(source).mark_line(strokeDash=[5,3]).encode(
x=alt.X('X', title = 'X'),
y=alt.Y('rate:Q'),
color = alt.Color('rate-label',),
tooltip=['rate','X','Y']
)
Current solution
The issue with the current solution is that the rate chart's y-axis is not displaying as a dual axis. Any suggestions?
alt.layer(rate,scatter,line_plot).facet(
'Series:N'
, columns=2
).resolve_scale(
x='independent',
y='independent'
).display()

Well, I got it, but this probably isn't the best solution. I've followed the method described in the following link where we manually facet the charts:
Thread on Facets
To get the dual axis, I just added .resolve_scale(y='independent') to the manual step. Below is the solution:
import altair as alt
from vega_datasets import data
import pandas as pd
source = data.anscombe().copy()
source\['line-label'\] = 'x=y'
source = pd.concat(\[source,source.groupby('Series').agg(x_diff=('X','diff'), y_diff=('Y','diff'))\],axis=1)
source\['rate'\] = source.y_diff/source.x_diff
source\['rate-label'\] = 'rate of change'
source\['line-label'\] = 'line y=x'
source_linear = source.groupby(by=\['Series'\]).agg(x_linear=('X','max'), y_linear=('X', 'max')).reset_index().sort_values(by=\['Series'\])
source_origin = source_linear.copy()
source_origin\['y_linear'\] = 0
source_origin\['x_linear'\] = 0
source_linear = pd.concat(\[source_origin,source_linear\]).sort_values(by=\['Series'\])
source = source.merge(source_linear,on='Series').drop_duplicates()
scatter = alt.Chart().mark_circle(size=60, opacity=0.60).encode(
x=alt.X('X', title='X'),
y=alt.Y('Y', title='Y'),
color='Series:N',
tooltip=\['X','Y','rate'\]
)
line_plot = alt.Chart().mark_line(color= 'black', strokeDash=\[3,8\]).encode(
x=alt.X('x_linear', title = '', axis=None),
y=alt.Y('y_linear', title = '', axis=None),
shape = alt.Shape('line-label', title = 'Break Even'),
color = alt.value('black')
)
rate = alt.Chart().mark_line(strokeDash=\[5,3\]).encode(
x=alt.X('X', title = 'X'),
y=alt.Y('rate:Q'),
color = alt.Color('rate-label',),
tooltip=\['rate','X','Y'\]
)
scatter_rate = alt.layer(scatter, rate, data=source)
chart_generator = (alt.layer(scatter, rate, line_plot, data = source, title=f"{val}: Duplicated Points w/ Line at Y=X").transform_filter(alt.datum.Series == val).resolve_scale(y='independent') \
for val in source.Series.unique())
chart = alt.concat(*(
chart_generator
), columns=2).display()

Adding a Title or Text to a Folium Map

I'm wondering if there's a way to add a title or text on a folium map in python?
I have 8 maps to show and I want the user to know which map they're looking at without having to click on a marker. I attempted to add an image of the map, but couldn't because I don't have high enough reputation score.
My code:
#marker cluster
corpus_chris_loc = [27.783889, -97.510556]
harvey_avg_losses_map = folium.Map(location = corpus_chris_loc, zoom_start = 5)
marker_cluster = MarkerCluster().add_to(harvey_avg_losses_map)
#inside the loop add each marker to the cluster
for row_index, row_values in harvey_losses.iterrows():
loc = [row_values['lat'], row_values['lng']]
pop = ("zip code: " + str(row_values["loss_location_zip"]) + "\nzip_avg: " + "$" + str(row_values['zip_avg'])) #show the zip and it's avg
icon = folium.Icon(color='red')
marker = folium.Marker(
title = "Harvey: " + "$" + str(row_values['harvey_avg']),
location = loc,
popup = pop,
icon=icon)
marker.add_to(marker_cluster)
#save an interactive HTML map by calling .save()
harvey_avg_losses_map.save('../data/harveylossclustermap.html')
harvey_avg_losses_map[map of hurricane harvey insurance claims][1]

Of course you can add a title to a Folium map.
For example:
import folium
loc = 'Corpus Christi'
title_html = '''
<h3 align="center" style="font-size:16px"><b>{}</b></h3>
'''.format(loc)
m = folium.Map(location=[27.783889, -97.510556],
zoom_start=12)
m.get_root().html.add_child(folium.Element(title_html))
m.save('map-with-title.html')
m

Changing colors on bokeh patches plot real time

I'm trying to create a bokeh plot of the US States, and color each of the state according to some data. Now using this tutorial I managed to create this, but I also want to enhance it, and add a slider to it, to change the values displayed. For example like displaying separate years.
With the help of this tutorial, I managed to add the slider, and the underlying data does change, according to the hover text, but the colors aren't recalculated, and so the visual representation does not match the values.
This is the code I've used, from a Jupyter notebook, so anybody who wants to try can reproduce
from bokeh.io import show, output_notebook
from bokeh.models import (
ColumnDataSource,
HoverTool,
LogColorMapper,
Range1d, CustomJS, Slider
)
from bokeh.palettes import Inferno256 as palette
from bokeh.plotting import figure
from bokeh.layouts import row, widgetbox
from bokeh.sampledata.us_counties import data as counties
from bokeh.sampledata.us_states import data as states
from bokeh.sampledata.unemployment import data as unemployment
import pandas as pd
import random
output_notebook()
palette.reverse()
states_accumulated ={}
available_state_codes = states.keys()
for key, value in counties.items():
state_name = value["state"].upper()
if state_name in states.keys() and "number" not in states[state_name]:
states[state_name]["number"] = key[0]
for key,state in states.items():
state["code"] = key
state_list = []
for key,state in states.items():
state_list.append(state)
unemployment_transf = []
for key,value in unemployment.items():
unemployment_transf.append({
"State":key[0],
"County":key[1],
"Value":value
})
unemp_df = pd.DataFrame(unemployment_transf)
unemp_sum = unemp_df.groupby("State").mean()["Value"]
unemp_sum = unemp_sum.sort_index()
unemp_sum_flat = {key:value for key, value in unemp_sum.items()}
for state in state_list:
state["value"] = unemp_sum_flat[state["number"]]
state_df = pd.DataFrame(state_list)
color_mapper = LogColorMapper(palette=palette)
state_xy = (list(state_df["lons"].values),list(state_df["lats"].values))
max_x = max([max(l) for l in state_xy[0]])
max_y = max([max(l) for l in state_xy[1]])
min_x = min([min(l) for l in state_xy[0]])
min_y = min([min(l) for l in state_xy[1]])
data=dict(
x=state_xy[0],
y=state_xy[1],
name=list(state_df["name"].values),
used = list(state_df["value"].values)
)
data['1999'] = list(state_df["value"].values)
data['2000'] = [random.randrange(0,10) for i in range(len(state_xy[0]))]
source = ColumnDataSource(data)
TOOLS = "pan,wheel_zoom,reset,hover,save"
p = figure(
title="States", tools=TOOLS,
x_axis_location=None, y_axis_location=None
)
p.width=450
p.height = 450
p.x_range= Range1d(-170,-60)
p.y_range = Range1d(min_y-10,max_y+10)
p.grid.grid_line_color = None
renderer = p.patches('x', 'y', source=source,
fill_color={'field': 'used', 'transform': color_mapper},
fill_alpha=0.7, line_color="white", line_width=0.5)
hover = p.select_one(HoverTool)
hover.point_policy = "follow_mouse"
hover.tooltips = [
("Name", "#name"),
("Unemployment rate)", "#used%"),
("(Long, Lat)", "($x, $y)"),
]
callback = CustomJS(args=dict(source=source,plot=p,color_mapper = color_mapper,renderer = renderer), code="""
var data = source.data;
var year = year.value;
used = data['used']
should_be = data[String(year)]
for (i = 0; i < should_be.length; i++) {
used[i] = should_be[i];
}
""")
year_slider = Slider(start=1999, end=2000, value=1999, step=1,
title="year", callback=callback)
callback.args["year"] = year_slider
layout = row(
p,
widgetbox(year_slider),
)
show(layout)
Sample images of the plot:
What I would like to accomplish, is that when I change the slider, the colors on the plot should change. Now I think the JS callback should call some kind of redraw or recalculate, but I haven't found any documentation about it. Is there a way to do this?

append source.change.emit() to the Javascipt code to trigger the change event.

Appending source.trigger("change"); to the CustomJS seems to solve the problem, now as the slider changes, the colors change.

Python: Bokeh hover tool tips date time [duplicate]

I am trying to get a line plot via Bokeh in Python. I am new to Bokeh and I am trying to apply hover tool tips over the plot. The x-axis of the plot has Timestamp values which are converted into epoch string. I've reviewed some same problems here and tried to use the workaround for my case but it doesn't seem to work. On the plot it gives ??? where the time should show up.
Any suggestions for my code?
Timestamp is in format 2016-12-29 02:49:12
Also can someone tell how do I format x-axis ticks to show up vertically ?
p = figure(width=1100,height=300,tools='resize,pan,wheel_zoom,box_zoom,reset,previewsave,hover',logo=None)
p.title.text = "Time Series for Price in Euros"
p.grid.grid_line_alpha = 0
p.xaxis.axis_label = "Day"
p.yaxis.axis_label = "Euros"
p.ygrid.band_fill_color = "olive"
p.ygrid.band_fill_alpha = 0.1
p.circle(df['DateTime'],df['EuP'], size=4, legend='close',
color='darkgrey', alpha=0.2)
p.xaxis.formatter = DatetimeTickFormatter(formats=dict(
hours=["%d %B %Y"],
days=["%d %B %Y"],
months=["%d %B %Y"],
years=["%d %B %Y"],
))
source = ColumnDataSource(data=dict(time=[x.strftime("%Y-%m-%d %H:%M:%S")for x in df['DateTime']]))
hover = p.select(dict(type=HoverTool))
hover.tooltips = {"time":'#time', "y":"$y"}
hover.mode = 'mouse'
p.line(df['DateTime'],df['EuP'],legend='Price',color='navy',alpha=0.7)

Since this answer was originally posted, new work has gone into Bokeh to make things simpler. A datetime field can be formatted as a datetime directly by the hover tool, by specifying a formatter, e.g.:
HoverTool(tooltips=[('date', '#DateTime{%F}')],
formatters={'#DateTime': 'datetime'})
It is no longer necessary to pre-format date fields in the data source as below. For more information see Formatting Tooltip Fields
OLD ANSWER:
The problem with your tooltip is you created a source with the string representation of the dates, but the p.line() call is unaware of it. So you have to pass in a columndatasource that has the tooltip, the x and y values.
Here is a working variant of your code:
from bokeh.plotting import figure, show
from bokeh.models.formatters import DatetimeTickFormatter
from bokeh.models import ColumnDataSource
from bokeh.models.tools import HoverTool
import pandas as pd
import numpy as np
data = {
'DateTime' : pd.Series(
['2016-12-29 02:49:12',
'2016-12-30 02:49:12',
'2016-12-31 02:49:12'],
dtype='datetime64[ns]'),
'EuP' : [20,40,15]
}
df = pd.DataFrame(data)
df['tooltip'] = [x.strftime("%Y-%m-%d %H:%M:%S") for x in df['DateTime']]
p = figure(width=1100,height=300,tools='resize,pan,wheel_zoom,box_zoom,reset,previewsave,hover',logo=None)
p.title.text = "Time Series for Price in Euros"
p.grid.grid_line_alpha = 0
p.xaxis.axis_label = "Day"
p.yaxis.axis_label = "Euros"
p.ygrid.band_fill_color = "olive"
p.ygrid.band_fill_alpha = 0.1
p.circle(df['DateTime'],df['EuP'], size=4, legend='close',
color='darkgrey', alpha=0.2)
p.xaxis.formatter = DatetimeTickFormatter(formats=dict(
hours=["%d %B %Y"],
days=["%d %B %Y"],
months=["%d %B %Y"],
years=["%d %B %Y"],
))
hover = p.select(dict(type=HoverTool))
tips = [('when','#tooltip'), ('y','$y')]
hover.tooltips = tips
hover.mode = 'mouse'
p.line(x='DateTime', y='EuP', source=ColumnDataSource(df),
legend='Price',color='navy',alpha=0.7)
show(p)
Also note there is an open issue about the lack of formatting options in the bokeh tooltip. There might be an easier way to not have to format the datestrings as a separate column:
https://github.com/bokeh/bokeh/issues/1239
Also can someone tell how do I format x-axis ticks to show up vertically ?
They look fine to me, sorry I cannot help on that one.
Hope this helps!
PS it would be better next time if you posted a working script with import statements, and a mocked up dataframe to make it possible to test. It took some time to sort it all out. But I am learning Bokeh so that is fine :)

Sorry for not commenting, I don't have enough reputation for that.
The accepted answer by #Alex doesn't work for me (Bokeh 2.0.1), because it is lacking a simple #-sign in the formatter. The working code is this:
HoverTool(tooltips=[('date', '#DateTime{%F}')],
formatters={'#DateTime': 'datetime'})

I have created a wrapper for ScatterPlot in bokeh.
class Visualization():
WIDTH = 1000
TOOLS = "pan,wheel_zoom,box_zoom,reset,save"
class ScatterChart(Visualization):
def __init__(self, data, spec:Dict):
self.data = data
self.x_column = spec["x_axis"]
self.y_column = spec["y_axis"]
self.series_ = spec["series_column"]
self.xlabel = spec['xlabel']
self.ylabel = spec['ylabel']
self.title = spec['title']
def prepare_data(self):
# Get Axis Type
self.xtype = 'datetime' if self.data.dtypes[self.x_column].type is np.datetime64 else 'linear'
self.ytype = 'datetime' if self.data.dtypes[self.x_column].type is np.datetime64 else 'linear'
return self.data
def render(self):
df_ = self.prepare_data()
format_= {}
tool_tip=[]
# For axis
for col in [self.x_column, self.y_column , self.series_]:
if self.data.dtypes[col].type is np.datetime64:
format_['#' + str(col) ] = "datetime" # formatter
tool_tip.append(tuple([str(col) , '#' + str(col) + '{%F}'])) # tool-tip
else:
format_['#' + str(col) ] = "numeral" #
tool_tip.append(tuple([str(col) , '#' + str(col)]))
# print(format_)
# print(tool_tip)
# Add Hover parameters
hover = HoverTool(tooltips= tool_tip
, formatters=format_ )
p=figure(
width = super().WIDTH,
height = 500,
x_axis_label = self.xlabel,
x_axis_type=self.xtype,
y_axis_label = self.ylabel,
y_axis_type=self.ytype,
title = self.title,
tools = super().TOOLS
)
# Get Only Top 10 groups/series to display
for value, color in zip(islice(self.data.groupby(by=[self.series_]
)[self.series_].count().sort_values(ascending=False).rename('cnt').reset_index()[self.series_].tolist(), 10), Category10[10]):
p.scatter(
x=self.x_column,
y=self.y_column,
source=df_.loc[(df_[self.series_]==value)],
color=color,
legend_group=self.series_
)
p.add_tools(hover)
p.toolbar.logo = None
p.legend.location = "top_left"
p.legend.click_policy="hide"
return p
# end of ScatterChart
This is how I initialize this
from visualization import ScatterChart
sc = ScatterChart(
df,
{'x_axis' :'ts',
'y_axis': 'Discus',
'series_column': 'Competition',
'xlabel':'Discus',
'ylabel':'Javeline',
'title':'Discus Vs Javeline'
})
d = sc.render()
show(d)

Get selected data contained within box select tool in Bokeh

If I have a scatter plot in bokeh and I've enabled the Box Select Tool, suppose I select a few points with the Box Select Tool. How can I access the (x,y) position location information of the points that I've selected?
%matplotlib inline
import numpy as np
from random import choice
from string import ascii_lowercase
from bokeh.models.tools import *
from bokeh.plotting import *
output_notebook()
TOOLS="pan,wheel_zoom,reset,hover,poly_select,box_select"
p = figure(title = "My chart", tools=TOOLS)
p.xaxis.axis_label = 'X'
p.yaxis.axis_label = 'Y'
source = ColumnDataSource(
data=dict(
xvals=list(range(0, 10)),
yvals=list(np.random.normal(0, 1, 10)),
letters = [choice(ascii_lowercase) for _ in range(10)]
)
)
p.scatter("xvals", "yvals",source=source,fill_alpha=0.2, size=5)
select_tool = p.select(dict(type=BoxSelectTool))[0]
show(p)
# How can I know which points are contained in the Box Select Tool?
I can't call the "callback" attribute and the "dimensions" attribute just returns a list ["width", "height"]. If I can just get the dimensions and the location of the Selected Box, I can figure out which points are in my dataset from there.

You can use a callback on the ColumnDataSource that updates a Python variable with the indices of the selected data:
%matplotlib inline
import numpy as np
from random import choice
from string import ascii_lowercase
from bokeh.models.tools import *
from bokeh.plotting import *
from bokeh.models import CustomJS
output_notebook()
TOOLS="pan,wheel_zoom,reset,hover,poly_select,box_select"
p = figure(title = "My chart", tools=TOOLS)
p.xaxis.axis_label = 'X'
p.yaxis.axis_label = 'Y'
source = ColumnDataSource(
data=dict(
xvals=list(range(0, 10)),
yvals=list(np.random.normal(0, 1, 10)),
letters = [choice(ascii_lowercase) for _ in range(10)]
)
)
p.scatter("xvals", "yvals",source=source,fill_alpha=0.2, size=5)
select_tool = p.select(dict(type=BoxSelectTool))[0]
source.callback = CustomJS(args=dict(p=p), code="""
var inds = cb_obj.get('selected')['1d'].indices;
var d1 = cb_obj.get('data');
console.log(d1)
var kernel = IPython.notebook.kernel;
IPython.notebook.kernel.execute("inds = " + inds);
"""
)
show(p)
Then you can access the desired data attributes using something like
zip([source.data['xvals'][i] for i in inds],
[source.data['yvals'][i] for i in inds])

Here is a working example with Python 3.7.5 and Bokeh 1.4.0
public github link to this jupyter notebook:
https://github.com/surfaceowl-ai/python_visualizations/blob/master/notebooks/bokeh_save_linked_plot_data.ipynb
environment report:
virtual env python version: Python 3.7.5
virtual env ipython version: 7.9.0
watermark package reports:
bokeh 1.4.0
jupyter 1.0.0
numpy 1.17.4
pandas 0.25.3
rise 5.6.0
watermark 2.0.2
# Generate linked plots + TABLE displaying data + save button to export cvs of selected data
from random import random
from bokeh.io import output_notebook # prevent opening separate tab with graph
from bokeh.io import show
from bokeh.layouts import row
from bokeh.layouts import grid
from bokeh.models import CustomJS, ColumnDataSource
from bokeh.models import Button # for saving data
from bokeh.models.widgets import DataTable, DateFormatter, TableColumn
from bokeh.models import HoverTool
from bokeh.plotting import figure
# create data
x = [random() for x in range(500)]
y = [random() for y in range(500)]
# create first subplot
plot_width = 400
plot_height = 400
s1 = ColumnDataSource(data=dict(x=x, y=y))
fig01 = figure(
plot_width=plot_width,
plot_height=plot_height,
tools=["lasso_select", "reset", "save"],
title="Select Here",
)
fig01.circle("x", "y", source=s1, alpha=0.6)
# create second subplot
s2 = ColumnDataSource(data=dict(x=[], y=[]))
# demo smart error msg: `box_zoom`, vs `BoxZoomTool`
fig02 = figure(
plot_width=400,
plot_height=400,
x_range=(0, 1),
y_range=(0, 1),
tools=["box_zoom", "wheel_zoom", "reset", "save"],
title="Watch Here",
)
fig02.circle("x", "y", source=s2, alpha=0.6, color="firebrick")
# create dynamic table of selected points
columns = [
TableColumn(field="x", title="X axis"),
TableColumn(field="y", title="Y axis"),
]
table = DataTable(
source=s2,
columns=columns,
width=400,
height=600,
sortable=True,
selectable=True,
editable=True,
)
# fancy javascript to link subplots
# js pushes selected points into ColumnDataSource of 2nd plot
# inspiration for this from a few sources:
# credit: https://stackoverflow.com/users/1097752/iolsmit via: https://stackoverflow.com/questions/48982260/bokeh-lasso-select-to-table-update
# credit: https://stackoverflow.com/users/8412027/joris via: https://stackoverflow.com/questions/34164587/get-selected-data-contained-within-box-select-tool-in-bokeh
s1.selected.js_on_change(
"indices",
CustomJS(
args=dict(s1=s1, s2=s2, table=table),
code="""
var inds = cb_obj.indices;
var d1 = s1.data;
var d2 = s2.data;
d2['x'] = []
d2['y'] = []
for (var i = 0; i < inds.length; i++) {
d2['x'].push(d1['x'][inds[i]])
d2['y'].push(d1['y'][inds[i]])
}
s2.change.emit();
table.change.emit();
var inds = source_data.selected.indices;
var data = source_data.data;
var out = "x, y\\n";
for (i = 0; i < inds.length; i++) {
out += data['x'][inds[i]] + "," + data['y'][inds[i]] + "\\n";
}
var file = new Blob([out], {type: 'text/plain'});
""",
),
)
# create save button - saves selected datapoints to text file onbutton
# inspriation for this code:
# credit: https://stackoverflow.com/questions/31824124/is-there-a-way-to-save-bokeh-data-table-content
# note: savebutton line `var out = "x, y\\n";` defines the header of the exported file, helpful to have a header for downstream processing
savebutton = Button(label="Save", button_type="success")
savebutton.callback = CustomJS(
args=dict(source_data=s1),
code="""
var inds = source_data.selected.indices;
var data = source_data.data;
var out = "x, y\\n";
for (i = 0; i < inds.length; i++) {
out += data['x'][inds[i]] + "," + data['y'][inds[i]] + "\\n";
}
var file = new Blob([out], {type: 'text/plain'});
var elem = window.document.createElement('a');
elem.href = window.URL.createObjectURL(file);
elem.download = 'selected-data.txt';
document.body.appendChild(elem);
elem.click();
document.body.removeChild(elem);
""",
)
# add Hover tool
# define what is displayed in the tooltip
tooltips = [
("X:", "#x"),
("Y:", "#y"),
("static text", "static text"),
]
fig02.add_tools(HoverTool(tooltips=tooltips))
# display results
# demo linked plots
# demo zooms and reset
# demo hover tool
# demo table
# demo save selected results to file
layout = grid([fig01, fig02, table, savebutton], ncols=3)
output_notebook()
show(layout)
# things to try:
# select random shape of blue dots with lasso tool in 'Select Here' graph
# only selected points appear as red dots in 'Watch Here' graph -- try zooming, saving that graph separately
# selected points also appear in the table, which is sortable
# click the 'Save' button to export a csv
# TODO: export from Bokeh to pandas dataframe