I have seen the TripsLayer example in the deck.gl website (this one) and it looks really cool. I would like to accomplish the same but using pydeck, the python bindings for deck.gl. The example in pydeck's webpage (this one) is not animated and I am not sure how should I do it to get a smooth animation as shown in the javascript example. I have tried multiple things (passing lists, functions, variables with changing value etc.) but non of them have worked and I can't find any example with pydeck.
Thanks!
It's true that the example should include more trips. Here is how to achieve the animation of multiple trips in a jupyter notebook.
import time
import pandas as pd
import pydeck as pdk
data = '[{"agent_id":0,"path":[[-0.63968,50.83091,0.0],[-0.78175,50.83205,0.0]],"time":[65100,65520],"color":[228,87,86]},{"agent_id":1,"path":[[-0.63968,50.83091,0.0],[-0.78175,50.83205,0.0]],"time":[65940,66420],"color":[178,121,162]},{"agent_id":2,"path":[[-0.63968,50.83091,0.0],[-0.37617,50.8185,0.0]],"time":[65340,66360],"color":[157,117,93]},{"agent_id":3,"path":[[-0.63968,50.83091,0.0],[-0.78175,50.83205,0.0]],"time":[65940,66420],"color":[238,202,59]},{"agent_id":4,"path":[[-0.63968,50.83091,0.0],[-0.78175,50.83205,0.0]],"time":[67740,68160],"color":[157,117,93]}]'
df = pd.read_json(data)
view = {"bearing": 0, "latitude": 50.85, "longitude": -0.16, "pitch": 0, "zoom": 9}
time_min = 65_000
time_max = 80_000
layer = pdk.Layer(
"TripsLayer",
df,
get_path='path',
get_timestamps='time',
get_color='color',
opacity=0.8,
width_min_pixels=3,
rounded=True,
trail_length=900,
current_time=0
)
# Render
r = pdk.Deck(layers=[layer], initial_view_state=view, map_style='dark_no_labels')
r.show()
# Animate
for ct in range(time_min, time_max, 100):
layer.current_time = ct
r.update()
time.sleep(0.1)
Related
Objective: Create a plotly HISTOGRAM with a dropdown menu and put it in a html. The html will have multiple different types of graphs (not mentioned in this question).
I actually have a large dataframe/csv with many columns; but for this question I'm considering a very SIMPLE csv/dataframe.
The csv/dataframe has three columns - HOST,REQUEST,INCIDENT. Below is the sample csv.
HOST,REQUEST,INCIDENT
host1,GET,error
host1,GET,warning
host1,GET,warning
host1,POST,warning
host1,POST,error
host1,POST,warning
host2,GET,warning
host2,GET,error
host2,GET,warning
host2,POST,error
host2,POST,warning
host2,POST,error
host3,GET,error
host3,GET,error
host3,GET,error
host3,POST,error
host3,POST,error
host3,POST,warning
host4,GET,warning
host4,GET,error
host4,GET,error
host4,POST,error
host4,POST,warning
host4,POST,warning
Currently I'm plottting separate HISTOGRAM graphs for 'REQUEST Vs INCIDENT' for each HOST and then creating a html out of it. Means if there're four different hosts, then I'm plotting four different HISTOGRAM graphs in my html.
Below is my code.
import pandas as pd
import plotly.express as px
print(f"START")
df = pd.read_csv("dropdown.csv")
hosts = list(df['HOST'].unique())
print(hosts)
for host in hosts:
title = "Dropdown grap for host = " + host
df1 = df.loc[(df['HOST'] == host)]
graph = px.histogram(df1, x='REQUEST', color='INCIDENT', title=title)
with open("dropdown.html", 'a') as f:
f.write(graph.to_html(full_html=False, include_plotlyjs=True))
print(f"END")
Below is my output html having four graphs
But My Objective is to plot a single HISTOGRAM graph in my output html, with HOST being the dropdown. I should be able to select different HOSTs from the dropdown to get graph for each respective HOST.
Using plotly express I'm NOT getting any option to achieve my required output. Need help with this. Especially if I can achieve this using plotly.express itself that'll be great!
Other options are also welcome.
You can loop through all possible hosts, and create a corresponding fig using fig = px.histogram(df_subset_by_host, x='REQUEST', color='INCIDENT'), then extract the x array data stored in the fig._data object, and assign this data to the "x" arg of each host selection button.
For example:
from io import StringIO
import pandas as pd
import plotly.express as px
data_str = StringIO("""HOST,REQUEST,INCIDENT
host1,GET,error
host1,GET,warning
host1,GET,warning
host1,POST,warning
host1,POST,error
host1,POST,warning
host2,GET,warning
host2,GET,error
host2,GET,warning
host2,POST,error
host2,POST,warning
host2,POST,error
host3,GET,error
host3,GET,error
host3,GET,error
host3,POST,error
host3,POST,error
host3,POST,warning
host4,GET,warning
host4,GET,error
host4,GET,error
host4,POST,error
host4,POST,warning
host4,POST,warning""")
df = pd.read_csv(data_str)
hosts = list(df['HOST'].unique())
host_default = "host1"
title = f"Dropdown grap for host = {host_default}"
fig = px.histogram(df.loc[df['HOST'] == host_default], x='REQUEST', color='INCIDENT', title=title)
buttons = []
for host in hosts:
df_host = df.loc[(df['HOST'] == host)]
fig_host = px.histogram(df_host, x='REQUEST', color='INCIDENT')
buttons.append(
dict(
label=host,
method="update",
args=[
{
"x": [trace['x'] for trace in fig_host._data],
"title": f"Dropdown group for host {host}"
}
]
)
)
fig.update_layout(
updatemenus=[
dict(
type="dropdown",
direction="down",
showactive=True,
buttons=buttons
)
]
)
fig.show()
Cross-post: https://discourse.bokeh.org/t/add-calculated-horizontal-lines-corresponding-to-categories-on-the-x-axis/5544
I would like to duplicate this plot in Python:
Here is my attempt, using pandas and bokeh:
Imports:
import pandas as pd
from bokeh.io import output_notebook, show, reset_output
from bokeh.palettes import Spectral5, Turbo256
from bokeh.plotting import figure
from bokeh.transform import factor_cmap
from bokeh.models import Band, Span, FactorRange, ColumnDataSource
Create data:
fruits = ['Apples', 'Pears']
years = ['2015', '2016']
data = {'fruit' : fruits,
'2015' : [2, 1],
'2016' : [5, 3]}
fruit_df = pd.DataFrame(data).set_index("fruit")
tidy_df = (pd.DataFrame(data)
.melt(id_vars=["fruit"], var_name="year")
.assign(fruit_year=lambda df: list(zip(df['fruit'], df['year'])))
.set_index('fruit_year'))
Create bokeh plot:
p = figure(x_range=FactorRange(factors=tidy_df.index.unique()),
plot_height=400,
plot_width=400,
tooltips=[('Fruit', '#fruit'), # first string is user-defined; second string must refer to a column
('Year', '#year'),
('Value', '#value')])
cds = ColumnDataSource(tidy_df)
index_cmap = factor_cmap("fruit",
Spectral5[:2],
factors=sorted(tidy_df["fruit"].unique())) # this is a reference back to the dataframe
p.circle(x='fruit_year',
y='value',
size=20,
source=cds,
fill_color=index_cmap,
line_color=None,
)
# how do I add a median just to one categorical section?
median = Span(location=tidy_df.loc[tidy_df["fruit"] == "Apples", "value"].median(), # median value for Apples
#dimension='height',
line_color='red',
line_dash='dashed',
line_width=1.0
)
p.add_layout(median)
# how do I add this standard deviation(ish) band to just the Apples or Pears section?
band = Band(
base='fruit_year',
lower=2,
upper=4,
source=cds,
)
p.add_layout(band)
show(p)
Output:
Am I up against this issue? https://github.com/bokeh/bokeh/issues/8592
Is there any other data visualization library for Python that can accomplish this? Altair, Holoviews, Matplotlib, Plotly... ?
Band is a connected area, but your image of the desired output has two disconnected areas. Meaning, you actually need two bands. Take a look at the example here to better understand bands: https://docs.bokeh.org/en/latest/docs/user_guide/annotations.html#bands
By using Band(base='fruit_year', lower=2, upper=4, source=cds) you ask Bokeh to plot a band where for each value of fruit_year, the lower coordinate will be 2 and the upper coordinate will be 4. Which is exactly what you see on your Bokeh plot.
A bit unrelated but still a mistake - notice how your X axis is different from what you wanted. You have to specify the major category first, so replace list(zip(df['fruit'], df['year'])) with list(zip(df['year'], df['fruit'])).
Now, to the "how to" part. Since you need two separate bands, you cannot provide them with the same data source. The way to do it would be to have two extra data sources - one for each band. It ends up being something like this:
for year, sd in [('2015', 0.3), ('2016', 0.5)]:
b_df = (tidy_df[tidy_df['year'] == year]
.drop(columns=['year', 'fruit'])
.assign(lower=lambda df: df['value'].min() - sd,
upper=lambda df: df['value'].max() + sd)
.drop(columns='value'))
p.add_layout(Band(base='fruit_year', lower='lower', upper='upper',
source=ColumnDataSource(b_df)))
There are two issues left however. The first one is a trivial one - the automatic Y range (an instance of DataRange1d class by default) will not take the bands' heights into account. So the bands can easily go out of bounds and be cropped by the plot. The solution here is to use manual ranging that takes the SD values into account.
The second issue is that the width of band is limited to the X range factors, meaning that the circles will be partially outside of the band. This one is not that easy to fix. Usually a solution would be to use a transform to just shift the coordinates a bit at the edges. But since this is a categorical axis, we cannot do it. One possible solution here is to create a custom Band model that adds an offset:
class MyBand(Band):
# language=TypeScript
__implementation__ = """
import {Band, BandView} from "models/annotations/band"
export class MyBandView extends BandView {
protected _map_data(): void {
super._map_data()
const base_sx = this.model.dimension == 'height' ? this._lower_sx : this._lower_sy
if (base_sx.length > 1) {
const offset = (base_sx[1] - base_sx[0]) / 2
base_sx[0] -= offset
base_sx[base_sx.length - 1] += offset
}
}
}
export class MyBand extends Band {
__view_type__: MyBandView
static init_MyBand(): void {
this.prototype.default_view = MyBandView
}
}
"""
Just replace Band with MyBand in the code above and it should work. One caveat - you will need to have Node.js installed and the startup time will be longer for a second or two because the custom model code needs compilation. Another caveat - the custom model code knows about internals of BokehJS. Meaning, that while it's working with Bokeh 2.0.2 I can't guarantee that it will work with any other Bokeh version.
Relatively new here and new to Python in general, but trying to work with Plotly Express to create myself a Choropleth map that allows me to color-code custom data on countries. However, I'm having trouble actually loading the map up. I've been able to load the native Choropleth map with none of my data, but when I link my own geojson data set to the function it gets stuck loading on line 30. I haven't quite been able to figure out why, since it just gets stuck and gives me nothing else to troubleshoot with.
Any help is greatly appreciated!
import random
from token_file import token
import pandas as pd
import plotly.express as px
import json
with open('documents/countries.geojson') as f:
data = json.load(f)
countries = {}
for features in data["features"]:
if features["properties"]["ISO_A3"] != "-99":
name = features["properties"]["ADMIN"]
iso = features["properties"]["ISO_A3"]
geo = features["geometry"]
val = random.randint(0, 100)
values = pd.Series([name, iso, val], index=["Name", "ISO", "Val"])
countries[name] = values
else:
continue
countries = pd.DataFrame(countries)
countries = countries.transpose()
print(countries)
px.set_mapbox_access_token(token)
map = px.choropleth_mapbox(countries, locations="ISO", zoom=1, hover_name="Name", hover_data=["ISO"],
color="Val", color_continuous_scale="Viridis", mapbox_style="carto-positron")
map.update_layout(margin={"r": 0, "t": 0, "l": 0, "b": 0})
map.show()
I am having an issue with matching up the color table/brightness on CMI01 through CMI06 when creating GOES16 imagery with MetPy. I've tried using stock color tables and using random vmin/vmax to try and get a match. I've also tried using custom made color tables and even tried integrating things like min_reflectance_factor && max_reflectance_factor as vmin/vmax values.
Maybe I'm making this way more difficult than it is? Is there something I'm missing? Below are excerpts of code helping to create the current image output that I have:
grayscale = {"colors": [(0,0,0),(0,0,0),(255,255,255),(255,255,255)], "position": [0, 0.0909, 0.74242, 1]}
CMI_C02 = {"name": "C02", "commonName": "Visible Red Band", "grayscale": True, "baseDir": "visRed", "colorMap": grayscale}
dat = data.metpy.parse_cf('CMI_'+singleChannel['name'])
proj = dat.metpy.cartopy_crs
maxConcat = "max_reflectance_factor_"+singleChannel['name']
vmax = data[maxConcat]
sat = ax.pcolormesh(x, y, dat, cmap=make_cmap(singleChannel['colorMap']['colors'], position=singleChannel['colorMap']['position'], bit=True), transform=proj, vmin=0, vmax=vmax)
make_cmap is a handy dandy method I found that helps to create custom color tables. This code is part of a multiprocessing process, so singleChannel is actually CMI_C02.
For reference, the first image is from College of DuPage and the second is my output...
Any help/guidance would be greatly appreciated!
So your problem is, I believe, because there's a non-linear transformation being applied to the data on College of DuPage, in this case a square root (sqrt). This has been applied to GOES imagery in the past, as mentioned in the GOES ABI documentation. I think that's what is being done by CoD.
Here's a script to compare with and without sqrt:
import cartopy.feature as cfeature
from datetime import datetime, timedelta
import matplotlib.pyplot as plt
import metpy
import numpy as np
from siphon.catalog import TDSCatalog
# Trying to find the most recent image from around ~18Z
cat = TDSCatalog('http://thredds.ucar.edu/thredds/catalog/satellite/goes16'
'/GOES16/CONUS/Channel02/current/catalog.xml')
best_time = datetime.utcnow().replace(hour=18, minute=0, second=0, microsecond=0)
if best_time > datetime.utcnow():
best_time -= timedelta(days=1)
ds = cat.datasets.filter_time_nearest(best_time)
# Open with xarray and pull apart with some help using MetPy
data = ds.remote_access(use_xarray=True)
img_data = data.metpy.parse_cf('Sectorized_CMI')
x = img_data.metpy.x
y = img_data.metpy.y
# Create a two panel figure: one with no enhancement, one using sqrt()
fig = plt.figure(figsize=(10, 15))
for panel, func in enumerate([None, np.sqrt]):
if func is not None:
plot_data = func(img_data)
title = 'Sqrt Enhancement'
else:
plot_data = img_data
title = 'No Enhancement'
ax = fig.add_subplot(2, 1, panel + 1, projection=img_data.metpy.cartopy_crs)
ax.imshow(plot_data, extent=(x[0], x[-1], y[-1], y[0]),
cmap='Greys_r', origin='upper')
ax.add_feature(cfeature.COASTLINE, edgecolor='cyan')
ax.add_feature(cfeature.BORDERS, edgecolor='cyan')
ax.add_feature(cfeature.STATES, edgecolor='cyan')
ax.set_title(title)
Which results in:
The lower image, with the sqrt transformation applied seems to match the CoD image pretty well.
After polling some meteorologists, I ended up making a color table that was in between the two images as the agreed general consensus was that they thought my version was too dark and the standard was too light.
I still used vmax and vmin for pcolormesh() and simplified my grayscale object to just two colors with a slightly darker gray than the standard.
Thanks to all who looked at this.
I am using Bokeh in an experiment to plot data in realtime and the library provides a convenient way to do that.
Here a snippet of my code to accomplish this tasks:
# do the imports
import pandas as pd
import numpy as np
import time
from bokeh.plotting import *
from bokeh.models import ColumnDataSource
# here is simulated fake time series data
ts = pd.date_range("8:00", "10:00", freq="5S")
ts.name = 'timestamp'
ms = pd.Series(np.arange(0, len(ts)), index=ts)
ms.name = 'measurement'
data = pd.DataFrame(ms)
data['state'] = np.random.choice(3, len(ts))
data['observation'] = np.random.choice(2, len(ts))
data.reset_index(inplace=True)
data.head()
This is how the data looks like.
Next I have used the following snipped to push the data to the server in real time
output_server("observation")
p = figure(plot_width=800, plot_height=400, x_axis_type="datetime")
x = np.array(data.head(2).timestamp, dtype=np.datetime64)
y = np.array(data.head(2).observation)
p.diamond_cross(x,y, size=30, fill_color=None, line_width=2, name='observation')
show(p)
renderer = p.select(dict(name="observation"))[0]
ds = renderer.data_source
for mes in range(len(data)):
x = np.append(x, np.datetime64(data.loc[mes].timestamp))
y = np.append(y, np.int64(data.loc[mes].observation))
ds.data["x"] = x
ds.data["y"] = y
ds._dirty = True
cursession().store_objects(ds)
time.sleep(.1)
This produces a very nice result, however I need to change the color of each data point conditioned on a value.
In this case, the condition is the state variable which takes three values -- 0, 1, and 2. So my data should be able to reflect that.
I have spent hours trying to figure it out (admittedly I an very new to Bokeh) and any help will be greatly appreciated.
When you push the data, you have to separate the groups by desired color, and then supply the corresponding colors as a palette. There's a longer discussion with several variations at https://github.com/bokeh/bokeh/issues/1967, such as the simple boteh.charts dot example bryevdv posted on 28 Feb:
cat = ['foo', 'bar', 'baz']
xyvalues=dict(x=[1,4,5], y=[2,7,3], z=[3,4,5])
dots = Dot(
xyvalues, cat=cat, title="Data",
ylabel='FP Rate', xlabel='Vendors',
legend=False, palette=["red", "green", "blue"])
show(dots)
Please remember to read and follow the posting guidelines at https://stackoverflow.com/help/how-to-ask; I found this and several other potentially useful hits with my first search attempt, "Bokeh 'change color' plot". If none of these solve your problem, you need to differentiate what you're doing from the answers already out there.