We Keep Coding

`update` function doesn't work correctly for bokeh interactors in python

I have a source code that plots the alphashape of a stock price. There's a slider to update the plot dynamically. But the update function doesn't work as expected.
Here's the source code.
x=[76.84,76.85,76.86,76.87,76.88,76.9,76.91,76.92,76.93,76.94,76.97,76.97,76.98,76.99,77.0,77.03,77.03,77.04,77.05,77.06,77.09,77.09,77.1,77.11,77.12,77.15,77.16,77.16,77.17,77.18,77.21,77.22,77.22,77.23,77.24,77.27,77.28,77.28,77.29,77.3,77.33,77.34,77.35,77.35,77.36,77.39,77.4,77.41,77.41,77.42,77.45,77.46,77.47,77.47,77.48,77.51,77.52,77.53,77.54,77.54,77.57,77.58,77.59,77.6,77.6,77.63,77.64,77.65,77.66,77.66,77.69,77.7,77.71,77.72,77.73,77.75,77.76,77.77,77.78,77.79,77.81,77.82,77.83,77.84,77.85,77.87,77.88,77.89,77.9,77.91,77.93,77.94,77.95,77.96,77.97,77.99,78.0,78.01,78.02,78.03,78.05,78.06,78.07,78.08,78.09,78.13,78.14,78.15,78.17,78.18,78.19,78.2,78.21,78.24,78.24,78.25,78.26,78.27,78.3,78.3,78.31,78.32,78.33,78.36,78.36,78.37,78.38,78.39,78.42,78.43,78.43,78.44,78.45,78.48,78.49,78.49,78.5,78.51,78.54,78.55,78.55,78.56,78.57,78.6,78.61,78.62,78.62,78.63,78.66,78.67,78.68,78.68,78.69,78.72,78.73,78.74,78.74,78.75,78.78,78.79,78.8,78.81,78.81,78.84,78.85,78.86,78.87,78.87,78.91,78.92,78.93,78.94,78.96,78.97,78.98,78.99,79.0,79.02,79.03,79.04,79.05,79.06,79.08,79.09,79.1,79.11,79.12,79.2,79.21,79.22,79.23,79.24,79.26,79.27,79.28,79.29,79.3,79.32,79.33,79.34,79.35,79.36,79.38,79.39,79.4,79.41,79.42,79.44,79.45,79.46,79.47,79.48,79.51,79.51,79.52,79.53,79.54,79.57,79.57,79.58,79.59,79.6,79.63,79.63,79.64,79.65,79.66,79.69,79.7,79.7,79.71,79.72,79.75,79.76,79.76,79.77,79.78,79.81,79.82,79.82,79.83,79.84,79.87,79.88,79.89,79.89,79.9,79.94,79.95,79.95,79.96,79.99,80.0,80.01,80.02,80.02,80.05,80.06,80.07,80.08,80.08,80.11,80.12,80.13,80.14,80.14,80.17,80.18,80.19,80.2,80.21,80.23,80.24,80.25,80.26,80.27,80.29,80.3,80.31,80.32,80.33,80.35,80.36,80.37,80.38,80.39,80.41,80.42,80.43,80.44,80.45,80.47,80.48,80.49,80.5,80.51,80.53,80.54,80.55,80.56,80.57,80.59,80.6,80.61,80.62,80.63,80.65,80.66,80.67,80.68,80.69,80.71,80.72,80.73,80.74,80.75,80.78,80.78,80.79,80.8,80.81,80.84,80.84,80.85,80.86,80.87,80.9,80.9,80.91,80.92,80.93,80.96,80.97,80.97,80.98,80.99,81.02,81.03,81.03,81.04,81.05,81.08,81.09,81.1,81.1,81.11,81.14,81.15,81.16,81.16,81.17,81.2,81.21,81.22,81.22,81.23,81.28,81.29,81.29,81.32,81.33,81.34,81.35,81.35,81.38,81.39,81.4,81.41,81.41,81.44,81.45,81.46,81.47,81.48,81.5,81.51,81.52,81.53,81.54,81.56,81.57,81.58,81.59,81.6,81.62,81.63,81.64,81.65,81.66,81.68,81.69,81.7,81.71,81.72,81.74,81.75,81.76,81.77,81.78,81.8,81.81,81.82,81.83,81.84,81.86,81.87,81.88,81.89,81.9,81.92,81.93,81.94,81.95,81.96,81.98,81.99,82.0,82.01,82.02,82.05,82.06,82.07,82.08,82.11,82.11,82.12,82.13,82.14,82.17,82.18,82.18,82.19,82.2,82.23,82.24,82.24,82.25,82.26,82.29,82.3,82.3,82.31,82.32,82.35,82.36,82.37,82.37,82.38,82.41,82.42,82.43,82.43,82.44,82.59,82.6,82.61,82.62,82.62,82.65,82.66,82.67,82.68,82.68,82.71,82.72,82.73,82.74,82.75,82.77,82.78,82.79,82.8,82.81,82.83,82.84,82.85,82.86,82.87,82.89,82.9,82.91,82.92,82.93,82.95,82.96,82.97,82.98,82.99,83.01,83.02,83.03,83.04,83.05,83.07,83.08,83.1,83.11,83.13,83.14,83.15,83.16,83.17,83.19,83.2,83.21,83.22,83.23,83.26,83.26,83.27,83.28,83.29,83.32,83.32,83.33,83.34,83.35,83.38,83.38,83.39,83.4,83.41,83.44,83.45,83.45,83.46,83.47,83.5,83.51,83.51,83.52,83.53,83.56,83.57,83.57,83.58,83.59,83.62,83.63,83.64,83.64,83.65,83.68,83.69,83.7,83.7,83.71,83.74,83.75,83.76,83.76,83.77,83.8,83.81,83.82,83.83,83.83,83.86,83.87,83.88,83.89,83.89,83.92,83.93,83.94,83.95,83.95,83.98,83.99,84.0,84.01,84.02,84.04,84.05,84.06,84.07,84.08,84.1,84.11,84.12,84.13,84.14,84.16,84.17,84.18,84.19,84.2,84.22,84.23,84.24,84.25,84.26,84.28,84.29,84.3,84.31,84.32,84.34,84.35,84.36,84.37,84.38,84.43,84.44,84.46,84.47,84.48,84.49,84.5,84.53,84.53,84.54,84.55,84.56,84.59,84.59,84.6,84.61,84.62,84.65,84.65,84.66,84.67,84.68,84.71,84.72,84.72,84.73,84.74,84.77,84.78,84.78,84.79,84.8,84.83,84.84,84.84,84.85,84.86,84.89,84.9,84.91,84.91,84.92,84.95,84.96,84.97,84.97,84.98,85.01,85.02,85.03,85.03,85.04,85.07,85.08,85.09,85.1,85.1,85.13,85.14,85.15,85.16,85.16,85.19,85.2,85.22,85.22,85.25,85.26,85.27,85.28,85.29,85.31,85.32,85.33,85.34,85.35,85.37,85.38,85.39,85.4,85.41,85.43,85.44,85.45,85.46,85.47,85.61,85.62,85.63,85.64,85.65,85.67,85.68,85.69,85.7,85.71,85.73,85.74,85.75,85.76,85.77,85.8,85.8,85.81,85.82,85.83,85.86,85.86,85.87,85.88,85.89,85.92,85.92,85.93,85.94,85.95,85.98,85.99,85.99,86.0,86.01,86.04,86.05,86.05,86.06,86.07,86.1,86.11,86.11,86.12,86.13,86.16,86.17,86.18,86.18,86.19,86.22,86.23,86.24,86.28,86.29,86.3,86.3,86.31,86.34,86.35,86.36,86.37,86.37,86.4,86.41,86.42,86.43,86.43,86.46,86.47,86.48,86.49,86.5,86.52,86.53,86.54,86.55,86.56,86.58,86.59,86.6,86.61,86.62,86.64,86.65,86.66,86.67,86.68,86.7,86.71,86.72,86.73,86.74,86.78,86.79,86.8,86.82,86.83,86.84,86.85,86.86,86.88,86.89,86.9,86.91,86.92,86.94,86.95,86.96,86.97,86.98,87.0,87.01,87.02,87.03,87.04,87.07,87.07,87.08,87.09,87.1,87.13,87.13,87.14,87.15,87.16,87.19,87.19,87.2,87.21,87.22,87.25,87.26,87.26,87.27,87.28,87.31,87.32,87.32,87.33,87.34,87.37,87.38,87.38,87.39,87.4,87.43,87.44,87.45,87.45,87.46,87.49,87.5,87.51,87.51,87.52,87.55,87.56,87.61,87.62,87.63,87.64,87.64,87.67,87.68,87.69,87.7,87.7,87.73,87.74,87.75,87.76,87.77,87.79,87.8,87.81,87.82,87.83,87.85,87.86,87.87,87.88,87.89,87.91,87.92,87.93,87.94,87.95,87.97,87.98,87.99,88.0,88.01,88.03,88.04,88.05,88.06,88.07,88.09,88.1,88.11,88.12,88.13,88.15,88.16,88.17,88.18,88.19,88.21,88.22,88.23,88.24,88.25,88.27,88.28,88.29,88.3,88.31,88.34,88.34,88.35,88.4,88.4,88.41,88.42,88.43,88.46,88.46,88.47,88.48,88.49,88.52,88.53,88.53,88.54,88.55,88.7,88.71,88.72,88.72,88.73,88.76,88.77,88.78,88.78,88.79,88.82,88.83,88.84,88.85,88.85,88.88,88.89,88.9,88.91,88.91,88.94,88.95,88.96,88.97,88.97,89.0,89.01,89.02,89.03,89.04,89.06,89.07,89.08,89.09,89.1,89.12,89.13,89.14,89.15,89.16,89.18,89.19,89.2,89.21,89.22,89.24,89.25,89.26,89.27,89.28,89.3,89.31,89.32,89.33,89.34,89.36,89.37,89.38,89.39,89.42,89.43,89.44,89.45,89.46,89.48,89.49,89.5,89.51,89.52,89.54,89.55,89.56,89.57,89.58,89.61]
y=[2.29,2.41,2.4,2.38,2.43,2.42,2.38,2.36,2.4,2.37,2.36,2.37,2.34,2.32,2.31,2.25,2.25,2.21,2.2,2.21,2.21,2.21,2.21,2.19,2.17,2.1,2.08,2.08,2.12,2.15,2.1,2.09,2.1,2.08,2.08,2.01,2.0,1.98,1.98,1.95,1.92,1.92,1.92,1.92,1.92,1.88,1.88,1.91,1.91,1.88,1.89,1.87,1.85,1.84,1.83,1.88,1.93,1.88,1.82,1.82,2.08,2.13,2.35,2.32,2.37,2.34,2.25,2.35,2.33,2.34,2.32,2.34,2.39,2.53,2.49,2.53,2.54,2.55,2.53,2.52,2.52,2.54,2.66,2.71,2.81,2.92,3.09,2.99,3.03,2.98,3.01,2.98,2.93,2.91,2.93,2.91,2.89,2.92,2.9,2.87,2.9,2.9,2.93,2.83,2.78,2.67,2.6,2.66,2.61,2.61,2.61,2.54,2.56,2.51,2.52,2.55,2.6,2.6,2.67,2.63,2.62,2.63,2.61,2.58,2.59,2.59,2.62,2.59,2.58,2.61,2.63,2.6,2.63,2.63,2.61,2.6,2.58,2.58,2.57,2.58,2.58,2.58,2.58,2.57,2.58,2.58,2.58,2.58,2.55,2.52,2.53,2.53,2.51,2.46,2.48,2.45,2.54,2.53,2.49,2.51,2.49,2.48,2.49,2.47,2.48,2.49,2.48,2.5,2.5,2.55,2.53,2.52,2.51,2.49,2.5,2.49,2.49,2.47,2.46,2.48,2.45,2.45,2.43,2.43,2.45,2.45,2.45,2.45,2.45,2.45,2.45,2.45,2.46,2.45,2.44,2.44,2.45,2.45,2.47,2.56,2.52,2.48,2.47,2.5,2.54,2.54,2.58,2.61,2.63,2.63,2.63,2.61,2.59,2.59,2.56,2.57,2.58,2.56,2.57,2.61,2.59,2.6,2.6,2.58,2.6,2.59,2.6,2.61,2.61,2.59,2.6,2.62,2.62,2.6,2.61,2.59,2.59,2.59,2.59,2.61,2.67,2.65,2.63,2.63,2.6,2.56,2.59,2.59,2.59,2.58,2.58,2.57,2.58,2.55,2.55,2.58,2.58,2.57,2.58,2.83,2.88,2.93,2.79,2.82,2.81,2.86,2.86,2.85,2.82,2.82,2.82,2.78,2.78,2.82,2.79,2.8,2.79,2.79,2.78,2.72,2.73,2.71,2.72,2.73,2.73,2.74,2.74,2.72,2.73,2.73,2.71,2.68,2.71,2.75,2.84,2.91,2.89,2.92,2.97,2.96,2.94,2.99,3.04,2.97,2.99,2.97,2.99,2.98,2.99,3.0,3.01,2.99,2.98,2.99,2.99,2.99,3.01,2.96,2.97,3.0,2.98,2.97,2.96,2.96,3.0,3.0,2.99,2.98,2.99,2.99,2.99,2.99,2.99,2.99,2.98,2.98,2.98,2.98,3.02,3.03,3.03,3.05,3.09,3.08,3.1,3.12,3.14,3.13,3.12,3.14,3.15,3.13,3.15,3.14,3.14,3.14,3.14,3.13,3.11,3.08,3.08,3.08,3.08,3.1,3.11,3.11,3.11,3.09,3.13,3.17,3.28,3.43,3.52,3.47,3.45,3.45,3.45,3.44,3.46,3.46,3.45,3.44,3.45,3.45,3.45,3.45,3.45,3.47,3.5,3.54,3.52,3.5,3.5,3.5,3.44,3.45,3.45,3.45,3.43,3.45,3.48,3.48,3.45,3.46,3.43,3.46,3.45,3.43,3.43,3.42,3.42,3.43,3.42,3.41,3.39,3.38,3.38,3.38,3.4,3.39,3.38,3.39,3.37,3.37,3.38,3.38,3.38,3.38,3.38,3.38,3.37,3.36,3.37,3.36,3.36,3.37,3.36,3.41,3.41,3.4,3.39,3.39,3.37,3.37,3.36,3.36,3.36,3.36,3.36,3.37,3.36,3.37,3.39,3.45,3.42,3.39,3.4,3.4,3.39,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.38,3.42,3.42,3.41,3.39,3.39,3.39,3.37,3.38,3.4,3.41,3.44,3.43,3.43,3.43,3.43,3.42,3.42,3.42,3.47,3.46,3.47,3.53,3.65,3.59,3.76,3.85,3.77,3.9,3.76,3.75,3.8,3.73,3.7,3.66,3.68,3.66,3.69,3.68,3.69,3.69,3.61,3.61,3.61,3.59,3.59,3.59,3.63,3.61,3.62,3.63,3.62,3.61,3.61,3.62,3.69,3.66,3.69,3.68,3.66,3.65,3.66,3.68,3.78,3.76,3.77,3.74,3.75,3.77,3.75,3.7,3.7,3.73,3.74,3.79,3.83,3.87,3.86,3.8,3.81,3.78,3.8,3.78,3.78,3.84,3.81,3.81,3.82,3.78,3.75,3.76,3.74,3.72,3.71,3.72,3.78,3.78,3.77,3.76,3.74,3.74,3.75,3.75,3.73,3.72,3.71,3.68,3.7,3.67,3.64,3.56,3.57,3.56,3.61,3.62,3.59,3.57,3.59,3.55,3.54,3.53,3.52,3.53,3.53,3.58,3.6,3.57,3.53,3.53,3.54,3.55,3.57,3.57,3.58,3.64,3.63,3.6,3.6,3.6,3.59,3.6,3.6,3.61,3.61,3.62,3.64,3.64,3.64,3.69,3.73,3.71,3.69,3.69,3.69,3.65,3.66,3.66,3.72,3.73,3.7,3.7,3.72,3.74,3.74,3.74,3.79,3.85,3.9,3.88,3.93,3.86,3.94,4.0,4.0,3.97,3.94,3.93,3.91,3.92,3.94,3.94,3.94,3.99,3.98,4.01,3.99,3.92,3.82,3.71,3.81,3.77,3.76,3.81,3.79,3.83,3.83,3.88,3.89,3.84,3.84,3.83,3.79,3.81,3.8,3.81,3.82,3.83,3.8,3.81,3.81,3.83,3.83,3.86,3.92,3.93,3.97,3.97,3.96,3.95,3.94,3.96,3.98,3.88,3.98,4.0,4.02,4.04,4.08,4.09,4.09,4.16,4.22,4.21,4.19,4.19,4.18,4.19,4.2,4.19,4.2,4.21,4.27,4.3,4.29,4.26,4.29,4.29,4.34,4.36,4.35,4.33,4.33,4.36,4.34,4.33,4.34,4.37,4.35,4.36,4.39,4.38,4.41,4.4,4.4,4.39,4.39,4.41,4.42,4.46,4.48,4.53,4.63,4.65,4.71,4.81,4.91,5.0,4.95,5.04,5.01,4.98,4.9,4.95,4.91,4.8,4.9,4.86,4.76,4.77,4.77,4.79,4.8,4.79,4.81,4.89,4.87,4.87,4.87,4.8,4.79,4.75,4.69,4.69,4.71,4.78,4.76,4.74,4.73,4.8,4.81,4.84,4.83,4.83,4.83,4.79,4.75,4.75,4.66,4.69,4.7,4.68,4.7,4.73,4.72,4.75,4.75,4.75,4.71,4.72,4.71,4.69,4.68,4.64,4.65,4.65,4.66,4.66,4.64,4.65,4.64,4.62,4.63,4.6,4.52,4.45,4.53,4.49,4.5,4.48,4.37,4.39,4.4,4.41,4.43,4.47,4.46,4.45,4.42,4.44,4.45,4.45,4.44,4.43,4.41,4.41,4.44,4.41,4.38,4.38,4.37,4.37,4.38,4.32,4.24,4.29,4.31,4.29,4.27,4.28,4.28,4.28,4.32,4.32,4.33,4.33,4.32,4.33,4.39,4.47,4.47,4.53,4.53,4.53,4.52,4.54,4.51,4.53,4.53,4.53,4.54,4.54,4.58,4.56,4.58,4.56,4.55,4.53,4.54,4.54,4.55,4.54,4.53,4.52,4.49,4.45,4.45,4.46,4.46,4.48,4.46,4.47,4.47,4.49,4.47,4.47,4.48,4.51,4.57,4.57,4.59,4.61,4.57,4.57,4.6,4.64,4.64,4.63,4.65,4.65,4.64,4.64,4.66,4.72,4.73,4.76,4.74,4.8,4.78,4.72,4.76,4.86,4.86,4.88,4.86,4.83,4.85,4.85,4.84,4.81,4.82,4.82,4.82,4.81,4.82,4.85,4.85,4.84,4.82,4.81,4.78,4.81,4.79,4.75,4.78,4.8,4.79,4.78,4.76,4.77,4.77,4.77,4.78,4.79,4.79,4.76,4.75,4.74,4.73,4.74,4.75,4.8,4.81,4.84,4.82,4.8,4.81,4.8,4.77,4.81,4.8,4.81,4.84,4.86,4.83,4.82,4.81,4.8,4.78,4.81,4.81,4.82,4.88,4.84,4.84,4.83,4.83,4.85,4.85,4.83,4.81,4.82,4.79,4.8,4.79,4.78,4.8,4.79,4.78,4.77,4.78,4.77,4.76,]
from alphashape import alphashape
from shapely.geometry import mapping
from bokeh.plotting import figure
from ipywidgets import interact
from bokeh.io import output_notebook, show, push_notebook
def alphashape_func(x, y, alpha):
length = range(len(x))
# date count
pnt = [[x[i],y[i]] for i in length]
# return a shapely.polygon/multipolygon
alpha_shape = alphashape(pnt, alpha=alpha)
# convert shapely.polygon/multipolygon to list
map = mapping(alpha_shape)['coordinates']
poly_shp = [i[0] for i in map]
bound_len = len(poly_shp)
# single alpha shape case
if bound_len == 1:
bound_x = [i[0] for i in poly_shp]
bound_y = [i[1] for i in poly_shp]
# multiple alpha shape case
else:
bound_x = [[i[0] for i in poly_shp[j]] for j in range(bound_len)]
bound_y = [[i[1] for i in poly_shp[j]] for j in range(bound_len)]
# return a dict containing 2 lists: x & y.
return {'x':bound_x, 'y':bound_y}
alpha = 5
alpha_high_pnt = alphashape_func(x,y,alpha)
plot = figure(sizing_mode='stretch_width', output_backend="webgl")
# line_pnt(plot, max_processed_xy['x'], max_processed_xy['y'],legend_label ='processed_xy',line_color='yellow', line_width=2)
alpha_shape_plt = plot.multi_line(xs=alpha_high_pnt['x'],ys=alpha_high_pnt['y'], line_color='cyan',legend_label = 'alpha_high_pnt')
# create an update function
def update(alpha=5):
alpha_high_pnt = alphashape_func(x,y,alpha)
alpha_shape_plt.data_source.data['xs'] = alpha_high_pnt['x']
alpha_shape_plt.data_source.data['ys'] = alpha_high_pnt['y']
# push new values to the notebook
push_notebook()
output_notebook()
show(plot)
interact(update, alpha=(0,25,1))
(the dynamic slider only works when you run it in jupyter in a web browser)
When I drag the slider, it shows an error message:
BokehUserWarning: ColumnDataSource's columns must be of the same length. Current lengths: ('xs', 54), ('ys', 99)
I don't see the reason of this error, since when I manually adjust the alpha value, the lengths of xs and ys equal.
Can anyone help?
===================== update ======================
Based on #bigreddot suggestion, I update the code to this, the doesn't match problem is resolved, but the plot doesn't refresh yet.
from alphashape import alphashape
from shapely.geometry import mapping
from bokeh.plotting import figure
from bokeh.io import output_notebook, show, push_notebook
from bokeh.models import ColumnDataSource
from ipywidgets import interact
output_notebook()
def alphashape_func(x, y, alpha):
length = range(len(x))
# date count
pnt = [[x[i],y[i]] for i in length]
# return a shapely.polygon/multipolygon
alpha_shape = alphashape(pnt, alpha=alpha)
# convert shapely.polygon/multipolygon to list
map = mapping(alpha_shape)['coordinates']
poly_shp = [i[0] for i in map]
bound_len = len(poly_shp)
# single alpha shape case
if bound_len == 1:
bound_x = [i[0] for i in poly_shp]
bound_y = [i[1] for i in poly_shp]
# multiple alpha shape case
else:
bound_x = [[i[0] for i in poly_shp[j]] for j in range(bound_len)]
bound_y = [[i[1] for i in poly_shp[j]] for j in range(bound_len)]
# return a dict containing 2 lists: x & y.
return {'x':bound_x, 'y':bound_y}
alpha = 5
plot = figure(sizing_mode='stretch_width', output_backend="webgl")
source = ColumnDataSource(data=alphashape_func(x,y,alpha))
alpha_shape_plt = plot.multi_line(source=source, xs='x',ys='y', line_color='cyan',legend_label = 'alpha_high_pnt')
print
# create an update function
def update(alpha=5):
source.data = alphashape_func(x,y,alpha)
# push new values to the notebook
push_notebook()
interact(update, alpha=(0,25,1))
show(plot)

In between this line:
alpha_shape_plt.data_source.data['xs'] = alpha_high_pnt['x']
and this line:
alpha_shape_plt.data_source.data['ys'] = alpha_high_pnt['y']
the CDS columns are not all the same length. If you need to update with data that has a new length you should collect all the updates up front in a new_data dict and then set
source.data = new_data
to update the CDS "all at once". This is more efficient in any case, as well, since it results in fewer property update change events being sent out.

Grouping data in Python using Bokeh and visualizing it

I am trying to build a visual that tracks widget counts by category using hbar. The source data is not aggregated. This is what it looks like:
This data is aggregated at MktCatKey level, but I want to group by category and then perform a calculation on the counts. Lets say if the category is Category_A, I want to add +10 to the counts. Finally, I want to display both current and projected on a visual.
This is how far I have gotten:
query = open('workingsql.sql')
dataset = pd.read_sql_query(query.read(), cnxn)
query.close()
p = figure()
CurrentCount = dataset.Current
ProjCount = dataset.Projected
Cat = dataset.Category
grouped = dataset.groupby('Category')['Current','Projected'].sum()
source = ColumnDataSource(grouped)
p = figure(y_range=Cat)
p.hbar(y=Cat, right = CurrentCount, left = 0, height = 0.5,source=source, fill_color="#D7D7D7")
p.hbar(y=Cat, right = ProjCount, left = 0, height = 0.5,source=source, fill_color="#E21150")
hover = HoverTool()
hover.tooltips = [("Totals", "#Current Current Count")]
hover.mode = 'hline'
p.add_tools(hover)
show(p)
I was able to get this to work if I source directly from the dataset. But since I’m trying to perform a calculation, I cant use the source directly. I’m not fully familiar on how to do an if statement on CurrentCount to see if it’s for Category_A or not but that’s where I’m at.
I have additional things I want to do on this dataset (like bring in a goals dataset and plot against that), but taking small steps for now. Any help is appreciated.
Working code below:
import pyodbc
import pandas as pd
from bokeh.plotting import figure, output_file, show
from bokeh.models import ColumnDataSource, Div, Select, Slider, TextInput
from bokeh.embed import components
from bokeh.models.tools import HoverTool
query = open('workingsql.sql')
dataset = pd.read_sql_query(query.read(), cnxn)
query.close()
p = figure()
CurrentCount = dataset.Current
ProjCount = dataset.Projected
Cat = dataset.Category
grouped = dataset.groupby('Category')['Current','Projected'].sum()
source = ColumnDataSource(pd.DataFrame(grouped))
Category = source.data['Category'].tolist()
p = figure(y_range=Category)
p.hbar(y='Category', right = 'Current', left = 0, height = 0.5,source=source, fill_color="#D7D7D7")
p.hbar(y='Category', right = 'Projected', left = 0, height = 0.5,source=source, fill_color="#E21150")
hover = HoverTool()
hover.tooltips = [("Totals", "#Current Current Count")]
hover.mode = 'hline'
p.add_tools(hover)
show(p)

How to make an interactive time serie plot using plotly?

I am trying to make an interactive time serie visualization using plotly and jupyter notebook.
I want to have a simple plot where I can filter the index of a dataframe using plotly and ipywidget and store the new index I have. But, I have no idea how to do so. I am investigating the documentation without any success. What I am doing so far :
import pandas as pd
import numpy as np
import plotly.graph_objs as go
from ipywidgets import interactive
index = pd.date_range(start='2020-01-01', end='2020-01-15', freq='D')
timeserie = pd.DataFrame(np.random.normal(0,1,size=index.size), index=index, columns=['sensor'])
fig = go.FigureWidget([
go.Scatter(
x=timeserie.index.values,
y=timeserie.values,
mode='markers'
)
])
def update_training_dataset(index_min, index_max, sensor):
scatter = fig.data[0]
index = timeserie.loc[(timeserie.index >= index_min) & (timeserie.index <= index_max)].index
sensor_value = timeserie.loc[scatter.x, sensor].values
with fig.batch_update():
fig.layout.yaxis.title = sensor
scatter.x = index
scatter.y = sensor_value
interactive(update_training_dataset, index_min=index, index_max=index, sensor=timeserie.columns)
But, it leads to a strange error..
KeyError : "None of [Int64Index([15778368000000000000, ... are in the [index]"
This is weird as the index of my timeserie has datetimeindex as type.
This code would lead to updating the dataframe according to the values of sensor, index_min, index_max that the user set. Also, I note that the date are provided in a select widget... I would love to have a date picker here. Can someone help me ? Provide any code that I can get some insights from ? Thank you :)
EDIT
The solution is provided below thanks to Serge :)
fig = go.FigureWidget([
go.Scatter(
x=timeserie.index,
y=timeserie.values,
mode='markers'
)
])
def update_training_dataset(index_min, index_max, Sensor):
scatter = fig.data[0]
index = timeserie.loc[(timeserie.index >= index_min) & (timeserie.index <= index_max)].index
sensor_value = timeserie.loc[scatter.x, Sensor].values
with fig.batch_update():
fig.layout.yaxis.title = Sensor
scatter.x = index
scatter.y = sensor_value
date_picker_max = DatePicker(
description='End date',
disabled=False,
value = index.max()
)
date_picker_min = DatePicker(
description='Start date',
disabled=False,
value = index.min()
)
interact(
update_training_dataset,
index_min=date_picker_min,
index_max=date_picker_max,
Sensor=timeserie.columns
)
I am still working on a way to have hours:minutes:seconds in the date picker.
EDIT 2
By the way, no need to use interact instead of interactive : they seem to support widgets as parameters. Also, you need to import ipydatetime as below to get datetime picker.
# usual imports
from ipydatetime import DatetimePicker
fig = go.FigureWidget([
go.Scatter(
x=timeserie.index,
y=timeserie.values,
mode='markers'
)
])
def update_training_dataset(index_min, index_max, Sensor):
scatter = fig.data[0]
index = timeserie.loc[(timeserie.index >= index_min) & (timeserie.index <= index_max)].index
sensor_value = timeserie.loc[scatter.x, Sensor].values
with fig.batch_update():
fig.layout.yaxis.title = Sensor
scatter.x = index
scatter.y = sensor_value
date_picker_max = DatetimePicker(
description='End date',
disabled=False,
value = index.max()
)
date_picker_min = DatetimePicker(
description='Start date',
disabled=False,
value = index.min()
)
interact(
update_training_dataset,
index_min=date_picker_min,
index_max=date_picker_max,
Sensor=timeserie.columns
)

Actually, your code is all good. You did a simple mistake in the definition of fig. Try the following
fig = go.FigureWidget([
go.Scatter(
x=timeserie.index,
y=timeserie.values,
mode='markers'
)
])
def update_training_dataset(index_min, index_max, sensor):
scatter = fig.data[0]
index = timeserie.loc[(timeserie.index >= index_min) & (timeserie.index <= index_max)].index
sensor_value = timeserie.loc[scatter.x, sensor].values
with fig.batch_update():
fig.layout.yaxis.title = sensor
scatter.x = index
scatter.y = sensor_value
interactive(update_training_dataset, index_min=index, index_max=index, sensor=timeserie.columns)
You simly made the error of defining x=timeserie.index.values when it actually should be x=timeserie.index.
The result is fine when this is changed.

Bokeh callback not updating chart [duplicate]

Struggling to understand why this bokeh visual will not allow me to change plots and see the predicted data. The plot and select (dropdown-looking) menu appears, but I'm not able to change the plot for items in the menu.
Running Bokeh 1.2.0 via Anaconda. The code has been run both inside & outside of Jupyter. No errors display when the code is run. I've looked through the handful of SO posts relating to this same issue, but I've not been able to apply the same solutions successfully.
I wasn't sure how to create a toy problem out of this, so in addition to the code sample below, the full code (including the regression code and corresponding data) can be found at my github here (code: Regression&Plotting.ipynb, data: pred_data.csv, historical_data.csv, features_created.pkd.)
import pandas as pd
import datetime
from bokeh.io import curdoc, output_notebook, output_file
from bokeh.layouts import row, column
from bokeh.models import Select, DataRange1d, ColumnDataSource
from bokeh.plotting import figure
#Must be run from the command line
def get_historical_data(src_hist, drug_id):
historical_data = src_hist.loc[src_hist['ndc'] == drug_id]
historical_data.drop(['Unnamed: 0', 'date'], inplace = True, axis = 1)#.dropna()
historical_data['date'] = pd.to_datetime(historical_data[['year', 'month', 'day']], infer_datetime_format=True)
historical_data = historical_data.set_index(['date'])
historical_data.sort_index(inplace = True)
# csd_historical = ColumnDataSource(historical_data)
return historical_data
def get_prediction_data(src_test, drug_id):
#Assign the new date
#Write a new dataframe with values for the new dates
df_pred = src_test.loc[src_test['ndc'] == drug_id].copy()
df_pred.loc[:, 'year'] = input_date.year
df_pred.loc[:, 'month'] = input_date.month
df_pred.loc[:, 'day'] = input_date.day
df_pred.drop(['Unnamed: 0', 'date'], inplace = True, axis = 1)
prediction = lin_model.predict(df_pred)
prediction_data = pd.DataFrame({'drug_id': prediction[0][0], 'predictions': prediction[0][1], 'date': pd.to_datetime(df_pred[['year', 'month', 'day']], infer_datetime_format=True, errors = 'coerce')})
prediction_data = prediction_data.set_index(['date'])
prediction_data.sort_index(inplace = True)
# csd_prediction = ColumnDataSource(prediction_data)
return prediction_data
def make_plot(historical_data, prediction_data, title):
#Historical Data
plot = figure(plot_width=800, plot_height = 800, x_axis_type = 'datetime',
toolbar_location = 'below')
plot.xaxis.axis_label = 'Time'
plot.yaxis.axis_label = 'Price ($)'
plot.axis.axis_label_text_font_style = 'bold'
plot.x_range = DataRange1d(range_padding = 0.0)
plot.grid.grid_line_alpha = 0.3
plot.title.text = title
plot.line(x = 'date', y='nadac_per_unit', source = historical_data, line_color = 'blue', ) #plot historical data
plot.line(x = 'date', y='predictions', source = prediction_data, line_color = 'red') #plot prediction data (line from last date/price point to date, price point for input_date above)
return plot
def update_plot(attrname, old, new):
ver = vselect.value
new_hist_source = get_historical_data(src_hist, ver) #calls the function above to get the data instead of handling it here on its own
historical_data.data = ColumnDataSource.from_df(new_hist_source)
# new_pred_source = get_prediction_data(src_pred, ver)
# prediction_data.data = new_pred_source.data
#Import data source
src_hist = pd.read_csv('data/historical_data.csv')
src_pred = pd.read_csv('data/pred_data.csv')
#Prep for default view
#Initialize plot with ID number
ver = 781593600
#Set the prediction date
input_date = datetime.datetime(2020, 3, 31) #Make this selectable in future
#Select-menu options
menu_options = src_pred['ndc'].astype(str) #already contains unique values
#Create select (dropdown) menu
vselect = Select(value=str(ver), title='Drug ID', options=sorted((menu_options)))
#Prep datasets for plotting
historical_data = get_historical_data(src_hist, ver)
prediction_data = get_prediction_data(src_pred, ver)
#Create a new plot with the source data
plot = make_plot(historical_data, prediction_data, "Drug Prices")
#Update the plot every time 'vselect' is changed'
vselect.on_change('value', update_plot)
controls = row(vselect)
curdoc().add_root(row(plot, controls))
UPDATED: ERRORS:
1) No errors show up in Jupyter Notebook.
2) CLI shows a UserWarning: Pandas doesn't allow columns to be careated via a new attribute name, referencing `historical_data.data = ColumnDatasource.from_df(new_hist_source).
Ultimately, the plot should have a line for historical data, and another line or dot for predicted data derived from sklearn. It also has a dropdown menu to select each item to plot (one at a time).

Your update_plot is a no-op that does not actually make any changes to Bokeh model state, which is what is necessary to change a Bokeh plot. Changing Bokeh model state means assigning a new value to a property on a Bokeh object. Typically, to update a plot, you would compute a new data dict and then set an existing CDS from it:
source.data = new_data # plain python dict
Or, if you want to update from a DataFame:
source.data = ColumnDataSource.from_df(new_df)
As an aside, don't assign the .data from one CDS to another:
source.data = other_source.data # BAD
By contrast, your update_plot computes some new data and then throws it away. Note there is never any purpose to returning anything at all from any Bokeh callback. The callbacks are called by Bokeh library code, which does not expect or use any return values.
Lastly, I don't think any of those last JS console errors were generated by BokehJS.

Create plotly scattermapbox from pandas dataframe

I would like to create a scattermapbox for indonesia for various statistics (population, GDP, etc.) on a regional basis.
I am working with a geopandas file from github.
The example on the plotly website creates multiple files for each layer and then uses the github link as source.
#republican counties
source = 'https://raw.githubusercontent.com/plotly/datasets/master/florida-red-data.json'
#democrat counties
source = 'https://raw.githubusercontent.com/plotly/datasets/master/florida-blue-data.json'
My question therefore is, how can I use the pandas dataframe to create layer dict for every region and use that as a source (also colouring of each region by specific values in other dataframes).
Should that not be possible at all and it is necessary to create a seperate file for each region how would I do that? My attempt (lines 16-20) doesn't seem to work
import pandas as pd
import json
import string
import plotly
from plotly.graph_objs import Scattermapbox, Layout
ID_regions = pd.read_json('https://raw.githubusercontent.com/N1x0/indonesia-geojson/master/indonesia-edit.geojson')
region_names = []
for region in ID_regions['features']:
region_names.append(state['properties']['name'])
print(region_names)
#This shit creates json and doesn't work
def create_region_files():
for i in range(len(ID_regions)):
region_data = ID_regions.iloc[i,:]
region_data.to_json(f'C:\\Users\\nicho\\Desktop\\Waste Management\\Map_Maker\\ID_regions\\{region_names[i]}.json')
i += 1
def create_Chloropleth():
mapbox_access_token = 'My Access Key'
data = [
Scattermapbox(
lat=['45.5017'],
lon=['-73.5673'],
mode='markers',
)
]
layout = Layout(
height=900,
autosize=True,
showlegend=False,
hovermode='closest',
mapbox=dict(
layers=[
dict(
sourcetype = 'geojson',
source = 'https://raw.githubusercontent.com/N1x0/indonesia-geojson/master/indonesia-edit.geojson',
type = 'fill',
color = 'green'
),
dict(
sourcetype = 'geojson',
source = 'https://raw.githubusercontent.com/N1x0/indonesia-geojson/master/west-sulawesi.json',
type = ' fill',
color = 'red',
)
],
accesstoken=mapbox_access_token,
bearing=0,
center=dict(
lat=0.7893,
lon=113.9213
),
pitch=0,
zoom=4.5,
style='light'
),
)
fig = dict(data=data, layout=layout)
plotly.offline.plot(fig, filename='Chloropleth_Province_Population.html')
create_Chloropleth()
Thank you for the help!

Ok took me a while but i figured it all out. Big thanks to Emma Grimaldi over at Medium and Vince Pota. Their posts were what helped me through most of it.
So here the answers to my own question in order:
It is not necessary to create an individual file for each region. I.e. you can use a pandas dataframe to match names of the regions in the json and that'll work just fine.
with open('indonesia-en.geojson') as f:
geojson = json.load(f)
def make_sources(downsample = 0):
sources = []
geojson_copy = copy.deepcopy(geojson['features']) # do not overwrite the original file
for feature in geojson_copy:
if downsample > 0:
coords = np.array(feature['geometry']['coordinates'][0][0])
coords = coords[::downsample]
feature['geometry']['coordinates'] = [[coords]]
sources.append(dict(type = 'FeatureCollection',
features = [feature])
)
return sources
So you just extract the coordinates from the geojson and append them to a a list of dicts[{}].
How to use this list to dynamically create layers:
MAPBOX_APIKEY = "Your API Key"
data = dict(type='scattermapbox',
lat=lats,
lon=lons,
mode='markers',
text=hover_text,
marker=dict(size=1,
color=scatter_colors,
showscale = True,
cmin = minpop/1000000,
cmax = maxpop/1000000,
colorscale = colorscale,
colorbar = dict(
title='Population in Millions'
)
),
showlegend=False,
hoverinfo='text'
)
layers=([dict(sourcetype = 'geojson',
source =sources[k],
below="water",
type = 'line', # the borders
line = dict(width = 1),
color = 'black',
) for k in range(n_provinces) # where n_provinces = len(geojson['features'])
] +
[dict(sourcetype = 'geojson',
source =sources[k],
type = 'fill', # the area inside the borders
color = scatter_colors[k],
opacity=0.8
) for k in range(n_provinces) # where n_provinces = len(geojson['features'])
]
)
So the solution here is too set sources = sources[k] I.e. the list with the dict of lat/long values created in make_sources()
How to color the layers accordingly color=scatter_colors[k]
Using the linked example I used 3 functions
3.1 scalarmappable
#sets colors based on min and max values
def scalarmappable(cmap, cmin, cmax):
colormap = cm.get_cmap(cmap)
norm = Normalize(vmin=cmin, vmax=cmax+(cmax*0.10)) #vmax get's increased 10 percent because otherwise the most populous region doesnt get colored
return cm.ScalarMappable(norm=norm, cmap=colormap)
3.2 scatter_colors
#uses matplotlib to create colors based on values and sets grey for isnan value
def get_scatter_colors(sm, df):
grey = 'rgba(128,128,128,1)'
return ['rgba' + str(sm.to_rgba(m, bytes = True, alpha = 1)) if not np.isnan(m) else grey for m in df]
3.3 colorscale
#defines horizontal range and corresponding values for colorscale
def get_colorscale(sm, df, cmin, cmax):
xrange = np.linspace(0, 1, len(df))
values = np.linspace(cmin, cmax, len(df))
return [[i, 'rgba' + str(sm.to_rgba(v, bytes = True))] for i,v in zip(xrange, values) ]
Then variables using the functions are set
#assigning values
colormap = 'nipy_spectral'
minpop = stats['population'].min()
maxpop = stats['population'].max()
sources = make_sources(downsample=0)
lons, lats = get_centers()
sm = scalarmappable(colormap, minpop, maxpop)
scatter_colors = get_scatter_colors(sm, stats['population'])
colorscale = get_colorscale(sm, stats, minpop, maxpop)
hover_text = get_hover_text(stats['population'])
So if anyone had some problems with this answer can help you progress :)