Related
I wanted to try to parse this nested json using Pandas, and I'm confused when i wanted to extract the data from column "amount" and "items", and the data has so many rows like hundreds, this is one of the example
{
"_id": "62eaa99b014c9bb30203e48a",
"amount": {
"product": 291000,
"shipping": 75000,
"admin_fee": 4500,
"order_voucher_deduction": 0,
"transaction_voucher_deduction": 0,
"total": 366000,
"paid": 366000
},
"status": 32,
"items": [
{
"_id": "62eaa99b014c9bb30203e48d",
"earning": 80400,
"variants": [
{
"name": "Color",
"value": "Black"
},
{
"name": "Size",
"value": "38"
}
],
"marketplace_price": 65100,
"product_price": 62000,
"reseller_price": 145500,
"product_id": 227991,
"name": "Heels",
"sku_id": 890512,
"internal_markup": 3100,
"weight": 500,
"image": "https://product-asset.s3.ap-southeast-1.amazonaws.com/1659384575578.jpeg",
"quantity": 1,
"supplier_price": 60140
}
I've tried using this and it'd only shows the index
dfjson=pd.json_normalize(datasetjson)
dfjson.head(3)
##UPDATE
I tried added the pd.Dataframe , yes it works to become dataframe, but i still haven't got to know how to extract the _id, earning, variants
Given:
data = {
'_id': '62eaa99b014c9bb30203e48a',
'amount': {'admin_fee': 4500,
'order_voucher_deduction': 0,
'paid': 366000,
'product': 291000,
'shipping': 75000,
'total': 366000,
'transaction_voucher_deduction': 0},
'items': [{'_id': '62eaa99b014c9bb30203e48d',
'earning': 80400,
'image': 'https://product-asset.s3.ap-southeast-1.amazonaws.com/1659384575578.jpeg',
'internal_markup': 3100,
'marketplace_price': 65100,
'name': 'Heels',
'product_id': 227991,
'product_price': 62000,
'quantity': 1,
'reseller_price': 145500,
'sku_id': 890512,
'supplier_price': 60140,
'variants': [{'name': 'Color', 'value': 'Black'},
{'name': 'Size', 'value': '38'}],
'weight': 500}],
'status': 32
}
Doing:
df = pd.json_normalize(data, ['items'], ['amount'])
df = df.join(df.amount.apply(pd.Series))
df = df.join(df.variants.apply(pd.DataFrame)[0].set_index('name').T.reset_index(drop=True))
df = df.drop(['amount', 'variants'], axis=1)
print(df)
Output:
_id earning marketplace_price product_price reseller_price product_id name sku_id internal_markup weight image quantity supplier_price product shipping admin_fee order_voucher_deduction transaction_voucher_deduction total paid Color Size
0 62eaa99b014c9bb30203e48d 80400 65100 62000 145500 227991 Heels 890512 3100 500 https://product-asset.s3.ap-southeast-1.amazon... 1 60140 291000 75000 4500 0 0 366000 366000 Black 38
There's probably a better way to do some of this, but the sample provided wasn't even a valid json object, so I can't be sure what the real data actually looks like.
Try pd.json_normalize(datasetjson, max_level=0)
I guess you are confuse working with dictionaries or json format.
This line is the same sample you have but it's missed ]} at the end. I formatted removing blank spaces but it's the same:
dfjson = {"_id":"62eaa99b014c9bb30203e48a","amount":{"product":291000,"shipping":75000,"admin_fee":4500,"order_voucher_deduction":0,"transaction_voucher_deduction":0,"total":366000,"paid":366000},"status":32,"items":[{"_id":"62eaa99b014c9bb30203e48d","earning":80400,"variants":[{"name":"Color","value":"Black"},{"name":"Size","value":"38"}],"marketplace_price":65100,"product_price":62000,"reseller_price":145500,"product_id":227991,"name":"Heels","sku_id":890512,"internal_markup":3100,"weight":500,"image":"https://product-asset.s3.ap-southeast-1.amazonaws.com/1659384575578.jpeg","quantity":1,"supplier_price":60140}]}
Now, if you want to call amount:
dfjson['amount']
# Output
{'product': 291000,
'shipping': 75000,
'admin_fee': 4500,
'order_voucher_deduction': 0,
'transaction_voucher_deduction': 0,
'total': 366000,
'paid': 366000}
If you want to call items:
dfjson['items']
# Output
[{'_id': '62eaa99b014c9bb30203e48d',
'earning': 80400,
'variants': [{'name': 'Color', 'value': 'Black'},
{'name': 'Size', 'value': '38'}],
'marketplace_price': 65100,
'product_price': 62000,
'reseller_price': 145500,
'product_id': 227991,
'name': 'Heels',
'sku_id': 890512,
'internal_markup': 3100,
'weight': 500,
'image': 'https://product-asset.s3.ap-southeast-1.amazonaws.com/1659384575578.jpeg',
'quantity': 1,
'supplier_price': 60140}]
For getting the items, you can create a list:
list_items = []
for i in dfjson['items']:
list_items.append(i)
For how to import the entire json data into a pandas DataFrame check out the answer given by BeRT2me
but if you are only after extracting _id, earning, variants to a Pandas DataFrame giving:
_id _id_id earning variants
0 62eaa99b014c9bb30203e48a 62eaa99b014c9bb30203e48d 80400 [{'name': 'Color', 'value': 'Black'}, {'name':...
as you state in your question:
but I still haven't got to know how to extract the _id, earning, variants
notice that the problem with extracting _id, earning, variants is that this values are 'hidden" within a list of one item. Resolving this issue with indexing it by [0] gives the required values:
json_text = """\
{'_id': '62eaa99b014c9bb30203e48a',
'amount': {'admin_fee': 4500,
'order_voucher_deduction': 0,
'paid': 366000,
'product': 291000,
'shipping': 75000,
'total': 366000,
'transaction_voucher_deduction': 0},
'items': [{'_id': '62eaa99b014c9bb30203e48d',
'earning': 80400,
'image': 'https://product-asset.s3.ap-southeast-1.amazonaws.com/1659384575578.jpeg',
'internal_markup': 3100,
'marketplace_price': 65100,
'name': 'Heels',
'product_id': 227991,
'product_price': 62000,
'quantity': 1,
'reseller_price': 145500,
'sku_id': 890512,
'supplier_price': 60140,
'variants': [{'name': 'Color', 'value': 'Black'},
{'name': 'Size', 'value': '38'}],
'weight': 500}],
'status': 32}
json_dict = eval(json_text)
print(f'{(_id := json_dict["items"][0]["_id"])=}')
print(f'{(earning := json_dict["items"][0]["earning"])=}')
print(f'{(variants := json_dict["items"][0]["variants"])=}')
print('---')
print(f'{_id=}')
print(f'{earning=}')
print(f'{variants=}')
gives:
(_id := json_dict["items"][0]["_id"])='62eaa99b014c9bb30203e48d'
(earning := json_dict["items"][0]["earning"])=80400
(variants := json_dict["items"][0]["variants"])=[{'name': 'Color', 'value': 'Black'}, {'name': 'Size', 'value': '38'}]
---
_id='62eaa99b014c9bb30203e48d'
earning=80400
variants=[{'name': 'Color', 'value': 'Black'}, {'name': 'Size', 'value': '38'}]```
If you want in addition a Pandas DataFrame with rows holding all this extracted values you can loop over all your json data files adding a row to a DataFrame using:
# Create an empty DataFrame:
df = pd.DataFrame(columns=['_id', '_id_id', 'earning', 'variants'])
# Add rows to df in a loop processing the json data files:
df_to_append = pd.DataFrame(
[[json_dict['_id'], _id, earning, variants]],
columns=['_id', '_id_id', 'earning', 'variants']
)
df = df.append(df_to_append)
pd.set_option('display.max_columns', None)
print(df.to_dict())
I want to display a shape over the Canada map.
The idea is 2 shapes in different years.
But my slide at the end says:
"Time Not Available"
I tried to find here at the community, but I haven't found a problem like it.
Here you can find my file and here you can find my code:
import folium
from folium.plugins import TimestampedGeoJson
import json
import pandas as pd
with open('outputfile.json') as f:
poly = json.load(f)
features = [
{
'type': 'Feature',
'geometry': {
'type': 'MultiPolygon',
'coordinates': pol['coordinates'],
},
'properties': {
'ABBREVNAME': pol['ABBREVNAME'],
'time': pol['date'],
}
} for pol in poly
]
mapa = folium.Map(
location = [56.130,-106.35],
tiles='openstreetmap',
zoom_start = 3
)
TimestampedGeoJson({'type': 'FeatureCollection', 'features': features}).add_to(mapa)
mapa
Thanks!!!
I had the same problem of yours of time not available and solved by following the example on the documentation and this other post.
Basically, some key points from doc:
1- It's is not 'time' but "times" and it must be the same length of the list of coordinates
2- Lookout for time format it only takes ISO or ms epoch
enter image description here
Here is a code example of a store location code i was working, I didn't try Polygon yet but hope it helps u:
m = folium.Map([-23.579782, -46.687754], zoom_start=6, tiles="cartodbpositron")
TimestampedGeoJson({
'type': 'FeatureCollection',
'features': [
{
'type': 'Feature',
'geometry': {
'type': 'LineString',
'coordinates': [[-46.687754, -23.579782]],
},
'properties': {
'icon': 'marker',
'iconstyle': {
'iconSize': [20, 20],
'iconUrl':
'https://img.icons8.com/ios-filled/50/000000/online-store.png'
},
'id': 'house',
'popup': 1,
'times': [1633046400000.0]
}
}, {
'type': 'Feature',
'geometry': {
'type': 'LineString',
'coordinates': [[-46.887754, -23.579782]],
},
'properties': {
'icon': 'marker',
'iconstyle': {
'iconSize': [20, 20],
'iconUrl':
'https://img.icons8.com/ios-filled/50/000000/online-store.png'
},
'id': 'house',
'popup': 1,
'times': [1635046400000.0]
}
}
]
}).add_to(m)
folium_static(m)
m.save('test.html')
A geocode api returns no location information for coordinates in ocean/sea. For those records, I would like to find the nearest possible coordinates that has a valid location information (that is closest land coordinates)
Below is the code for fetching location information by passing coordinates
import requests
request_url = "https://api.mapbox.com/geocoding/v5/mapbox.places/{0}%2C{1}.json?access_token={2}&types=country&limit=1".format(lng,lat,key)
response = requests.get(request_url)
output = response.json()
I have no clue in finding the nearest location. I'm also new to Python
Sample output:
{'type': 'FeatureCollection',
'query': [32.12, 54.21],
'features': [{'id': 'country.10008046970720960',
'type': 'Feature',
'place_type': ['country'],
'relevance': 1,
'properties': {'short_code': 'ru', 'wikidata': 'Q159'},
'text': 'Russia',
'place_name': 'Russia',
'bbox': [19.608673, 41.185353, 179.9, 81.961618],
'center': [37.61667, 55.75],
'geometry': {'type': 'Point', 'coordinates': [37.61667, 55.75]}}],
'attribution': 'NOTICE: © 2020 Mapbox and its suppliers. All rights reserved. Use of this data is subject to the Mapbox Terms of Service (https://www.mapbox.com/about/maps/). This response and the information it contains may not be retained. POI(s) provided by Foursquare.'}
Output when the coordinates are ocean:
{'type': 'FeatureCollection',
'query': [0, 0],
'features': [],
'attribution': 'NOTICE: © 2020 Mapbox and its suppliers. All rights reserved. Use of this data is subject to the Mapbox Terms of Service (https://www.mapbox.com/about/maps/). This response and the information it contains may not be retained. POI(s) provided by Foursquare.'}
Using Haversine Formula to find the nearest point (eg. city) based on latitude and longitude.
def dist_between_two_lat_lon(*args):
from math import asin, cos, radians, sin, sqrt
lat1, lat2, long1, long2 = map(radians, args)
dist_lats = abs(lat2 - lat1)
dist_longs = abs(long2 - long1)
a = sin(dist_lats/2)**2 + cos(lat1) * cos(lat2) * sin(dist_longs/2)**2
c = asin(sqrt(a)) * 2
radius_earth = 6378 # the "Earth radius" R varies from 6356.752 km at the poles to 6378.137 km at the equator.
return c * radius_earth
def find_closest_lat_lon(data, v):
try:
return min(data, key=lambda p: dist_between_two_lat_lon(v['lat'],p['lat'],v['lon'],p['lon']))
except TypeError:
print('Not a list or not a number.')
# city = {'lat_key': value, 'lon_key': value} # type:dict()
new_york = {'lat': 40.712776, 'lon': -74.005974}
washington = {'lat': 47.751076, 'lon': -120.740135}
san_francisco = {'lat': 37.774929, 'lon': -122.419418}
city_list = [new_york, washington, san_francisco]
city_to_find = {'lat': 29.760427, 'lon': -95.369804} # Houston
print(find_closest_lat_lon(city_list, city_to_find))
Which Yields:
{'lat': 47.751076, 'lon': -120.740135} # Corresponds to Washington
Let's suppose you got four json answers from mapbox and you saved them in a list:
json_answers = list() # = []
json_answers.append({'type': 'FeatureCollection',
'query': [32.12, 54.21],
'features': [{'id': 'country.10008046970720960',
'type': 'Feature',
'place_type': ['country'],
'relevance': 1,
'properties': {'short_code': 'ru', 'wikidata': 'Q159'},
'text': 'Russia',
'place_name': 'Russia',
'bbox': [19.608673, 41.185353, 179.9, 81.961618],
'center': [37.61667, 55.75],
'geometry': {'type': 'Point', 'coordinates': [37.61667, 55.75]}}],
'attribution': 'NOTICE: ...'})
# I changed only the 'coordinates' value for this example
json_answers.append({'type': 'FeatureCollection',
'query': [32.12, 54.21],
'features': [{'id': 'country.10008046970720960',
'type': 'Feature',
'place_type': ['country'],
'relevance': 1,
'properties': {'short_code': 'ru', 'wikidata': 'Q159'},
'text': 'Russia',
'place_name': 'Russia',
'bbox': [19.608673, 41.185353, 179.9, 81.961618],
'center': [37.61667, 55.75],
'geometry': {'type': 'Point', 'coordinates': [38.21667, 56.15]}}],
'attribution': 'NOTICE: ...'})
# I changed only the 'coordinates' value for this example
json_answers.append({'type': 'FeatureCollection',
'query': [32.12, 54.21],
'features': [{'id': 'country.10008046970720960',
'type': 'Feature',
'place_type': ['country'],
'relevance': 1,
'properties': {'short_code': 'ru', 'wikidata': 'Q159'},
'text': 'Russia',
'place_name': 'Russia',
'bbox': [19.608673, 41.185353, 179.9, 81.961618],
'center': [37.61667, 55.75],
'geometry': {'type': 'Point', 'coordinates': [33.21667, 51.15]}}],
'attribution': 'NOTICE: ...'})
# The last answer is "null"
json_answers.append({'type': 'FeatureCollection',
'query': [0, 0],
'features': [],
'attribution': 'NOTICE: ...'})
coord_list = []
for answer in json_answers:
if answer['features']: # check if ['features'] is not empty
# I'm not sure if it's [lat, lon] or [lon, lat] (you can verify it on mapbox)
print(f"Coordinates in [lat, lon]: {answer['features'][0]['geometry']['coordinates']}")
lat = answer['features'][0]['geometry']['coordinates'][0]
lon = answer['features'][0]['geometry']['coordinates'][1]
temp_dict = {'lat': lat, 'lon': lon}
coord_list.append(temp_dict)
print(f"coord_list = {coord_list}")
point_to_find = {'lat': 37.41667, 'lon': 55.05} # Houston
print(f"point_to_find = {point_to_find}")
print(f"find_closest_lat_lon = {find_closest_lat_lon(coord_list, point_to_find)}")
Which yields:
{'lat': 47.751076, 'lon': -120.740135}
Coordinates in [lat, lon]: [37.61667, 55.75]
Coordinates in [lat, lon]: [38.21667, 56.15]
Coordinates in [lat, lon]: [33.21667, 51.15]
coord_list = [{'lat': 37.61667, 'lon': 55.75}, {'lat': 38.21667, 'lon': 56.15}, {'lat': 33.21667, 'lon': 51.15}]
point_to_find = {'lat': 37.41667, 'lon': 55.05}
find_closest_lat_lon = {'lat': 38.21667, 'lon': 56.15}
Use reverse_geocode library in python to get nearest city with country.
Example:
import reverse_geocode
coordinates = (-37.81, 144.96), (31.76, 35.21)
reverse_geocode.search(coordinates)
Result:
[{'city': 'Melbourne', 'code': 'AU', 'country': 'Australia'},
{'city': 'Jerusalem', 'code': 'IL', 'country': 'Israel'}]
Here is an unoptimized solution.
What's going on under the hood of the function:
Run a GeoPy reverse look-up on a point.
If the point is found, return its country name.
If the point is not found, search for the nearest point of land in the world_geometry variable.
Perform a reverse lookup on that closest point.
Return that point's country name (if it exists) or the locality name (if no country name).
from geopy.geocoders import Nominatim
from shapely.ops import nearest_points
def country_lookup(query, geocoder, land_geometry):
try:
loc = geocoder.reverse((query.y, query.x))
return loc.raw['address']['country']
except (KeyError, AttributeError):
_, p2 = nearest_points(query, land_geometry)
loc = geocoder.reverse((p2.y, p2.x)).raw['address']
if 'country' in loc.keys():
return loc['country']
else:
return loc['locality']
# get world (or any land) geometry, instantiate geolocator service
world = gp.read_file(gp.datasets.get_path('naturalearth_lowres'))
world_geometry = world.geometry.unary_union
geolocator = Nominatim(user_agent="GIW")
# Create a column of country names from points in a GDF's geometry.
gdf['country'] = gdf.geometry.apply(country_lookup, args=(geolocator, world_geometry))
The accuracy of the results depends on the accuracy of the land geometry you provide. For example, geopandas's world geometry is pretty good. I was able to find names for all countries except for some of the smallest of the islands in the Bahamas. Those that it could not find were labelled "Bermuda Triangle" by the function, which is good enough for me.
Different package to try out is reverse_geocoder, which will return the nearest city, state, and country. Seems to be better than the reverse_geocode package.
import reverse_geocoder as rg
coordinates = (29,-84.1),(37,-125) #Both located in the ocean
rg.search(coordinates)
Output:
[OrderedDict([('lat', '29.67106'),
('lon', '-83.38764'),
('name', 'Steinhatchee'),
('admin1', 'Florida'),
('admin2', 'Taylor County'),
('cc', 'US')]),
OrderedDict([('lat', '38.71519'),
('lon', '-123.45445'),
('name', 'Sea Ranch'),
('admin1', 'California'),
('admin2', 'Sonoma County'),
('cc', 'US')])]
an animation example from this plotly tutorial is not working with Plotly 2.0.12. I put the error output below. Is there any way to solve the problem? I am using plotly on a Jupyter Notebook.
PlotlyError: Invalid 'figure_or_data' argument. Plotly will not be
able to properly parse the resulting JSON. If you want to send this
'figure_or_data' to Plotly anyway (not recommended), you can set
'validate=False' as a plot option.
Here's why you're seeing this error:
'slider' is not allowed in 'layout'
Path To Error: ['layout']['slider']
Valid attributes for 'layout' at path ['layout'] under parents
['figure']:
['angularaxis', 'annotations', 'autosize', 'bargap', 'bargroupgap',
'barmode', 'barnorm', 'boxgap', 'boxgroupgap', 'boxmode', 'calendar',
'direction', 'dragmode', 'font', 'geo', 'height', 'hiddenlabels',
'hiddenlabelssrc', 'hidesources', 'hoverlabel', 'hovermode',
'images',
'legend', 'mapbox', 'margin', 'orientation', 'paper_bgcolor',
'plot_bgcolor', 'radialaxis', 'scene', 'separators', 'shapes',
'showlegend', 'sliders', 'smith', 'ternary', 'title', 'titlefont',
'updatemenus', 'width', 'xaxis', 'yaxis']
Run `<layout-object>.help('attribute')` on any of the above.
'<layout-object>' is the object at ['layout']
EDIT: Just noticed the link is broken. Here is the full code:
from plotly.offline import init_notebook_mode, iplot
from IPython.display import display, HTML
import pandas as pd
init_notebook_mode(connected=True)
url = 'https://raw.githubusercontent.com/plotly/datasets/master/gapminderDataFiveYear.csv'
dataset = pd.read_csv(url)
years = ['1952', '1962', '1967', '1972', '1977', '1982', '1987', '1992', '1997', '2002', '2007']
# make list of continents
continents = []
for continent in dataset['continent']:
if continent not in continents:
continents.append(continent)
# make figure
figure = {
'data': [],
'layout': {},
'frames': [],
'config': {'scrollzoom': True}
}
# fill in most of layout
figure['layout']['xaxis'] = {'range': [30, 85], 'title': 'Life Expectancy'}
figure['layout']['yaxis'] = {'title': 'GDP per Capita', 'type': 'log'}
figure['layout']['hovermode'] = 'closest'
figure['layout']['slider'] = {
'args': [
'slider.value', {
'duration': 400,
'ease': 'cubic-in-out'
}
],
'initialValue': '1952',
'plotlycommand': 'animate',
'values': years,
'visible': True
}
figure['layout']['updatemenus'] = [
{
'buttons': [
{
'args': [None, {'frame': {'duration': 500, 'redraw': False},
'fromcurrent': True, 'transition': {'duration': 300, 'easing': 'quadratic-in-out'}}],
'label': 'Play',
'method': 'animate'
},
{
'args': [[None], {'frame': {'duration': 0, 'redraw': False}, 'mode': 'immediate',
'transition': {'duration': 0}}],
'label': 'Pause',
'method': 'animate'
}
],
'direction': 'left',
'pad': {'r': 10, 't': 87},
'showactive': False,
'type': 'buttons',
'x': 0.1,
'xanchor': 'right',
'y': 0,
'yanchor': 'top'
}
]
sliders_dict = {
'active': 0,
'yanchor': 'top',
'xanchor': 'left',
'currentvalue': {
'font': {'size': 20},
'prefix': 'Year:',
'visible': True,
'xanchor': 'right'
},
'transition': {'duration': 300, 'easing': 'cubic-in-out'},
'pad': {'b': 10, 't': 50},
'len': 0.9,
'x': 0.1,
'y': 0,
'steps': []
}
# make data
year = 1952
for continent in continents:
dataset_by_year = dataset[dataset['year'] == year]
dataset_by_year_and_cont =
dataset_by_year[dataset_by_year['continent'] == continent]
data_dict = {
'x': list(dataset_by_year_and_cont['lifeExp']),
'y': list(dataset_by_year_and_cont['gdpPercap']),
'mode': 'markers',
'text': list(dataset_by_year_and_cont['country']),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'size': list(dataset_by_year_and_cont['pop'])
},
'name': continent
}
figure['data'].append(data_dict)
# make frames
for year in years:
frame = {'data': [], 'name': str(year)}
for continent in continents:
dataset_by_year = dataset[dataset['year'] == int(year)]
dataset_by_year_and_cont =
dataset_by_year[dataset_by_year['continent'] == continent]
data_dict = {
'x': list(dataset_by_year_and_cont['lifeExp']),
'y': list(dataset_by_year_and_cont['gdpPercap']),
'mode': 'markers',
'text': list(dataset_by_year_and_cont['country']),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'size': list(dataset_by_year_and_cont['pop'])
},
'name': continent
}
frame['data'].append(data_dict)
figure['frames'].append(frame)
slider_step = {'args': [
[year],
{'frame': {'duration': 300, 'redraw': False},
'mode': 'immediate',
'transition': {'duration': 300}}
],
'label': year,
'method': 'animate'}
sliders_dict['steps'].append(slider_step)
figure['layout']['sliders'] = [sliders_dict]
iplot(figure)
I do not know if I have to downgrade the version (and, in case, to which one) but I'd rather not.
I validated your code and found some errors.
On Line 29 you should have given
figure['layout']['sliders'] instead of figure['layout']['slider']
Plotly offline's iplot function has a separate parameter of inputting config of the plot.
So the below line
# make figure
figure = {
'data': [],
'layout': {},
'frames': [],
'config': {'scrollzoom': True}
}
and the line
iplot(figure)
should actually be written as
# make figure
figure = {
'data': [],
'layout': {},
'frames': []
}
config = {'scrollzoom': True}
and
iplot(figure, config=config)
So the final working code should be
from plotly.offline import init_notebook_mode, iplot
from IPython.display import display, HTML
import pandas as pd
init_notebook_mode(connected=True)
url = 'https://raw.githubusercontent.com/plotly/datasets/master/gapminderDataFiveYear.csv'
dataset = pd.read_csv(url)
years = ['1952', '1962', '1967', '1972', '1977', '1982', '1987', '1992', '1997', '2002', '2007']
# make list of continents
continents = []
for continent in dataset['continent']:
if continent not in continents:
continents.append(continent)
# make figure
figure = {
'data': [],
'layout': {},
'frames': []
}
config = {'scrollzoom': True}
# fill in most of layout
figure['layout']['xaxis'] = {'range': [30, 85], 'title': 'Life Expectancy'}
figure['layout']['yaxis'] = {'title': 'GDP per Capita', 'type': 'log'}
figure['layout']['hovermode'] = 'closest'
figure['layout']['sliders'] = {
'args': [
'slider.value', {
'duration': 400,
'ease': 'cubic-in-out'
}
],
'initialValue': '1952',
'plotlycommand': 'animate',
'values': years,
'visible': True
}
figure['layout']['updatemenus'] = [
{
'buttons': [
{
'args': [None, {'frame': {'duration': 500, 'redraw': False},
'fromcurrent': True, 'transition': {'duration': 300, 'easing': 'quadratic-in-out'}}],
'label': 'Play',
'method': 'animate'
},
{
'args': [[None], {'frame': {'duration': 0, 'redraw': False}, 'mode': 'immediate',
'transition': {'duration': 0}}],
'label': 'Pause',
'method': 'animate'
}
],
'direction': 'left',
'pad': {'r': 10, 't': 87},
'showactive': False,
'type': 'buttons',
'x': 0.1,
'xanchor': 'right',
'y': 0,
'yanchor': 'top'
}
]
sliders_dict = {
'active': 0,
'yanchor': 'top',
'xanchor': 'left',
'currentvalue': {
'font': {'size': 20},
'prefix': 'Year:',
'visible': True,
'xanchor': 'right'
},
'transition': {'duration': 300, 'easing': 'cubic-in-out'},
'pad': {'b': 10, 't': 50},
'len': 0.9,
'x': 0.1,
'y': 0,
'steps': []
}
# make data
year = 1952
for continent in continents:
dataset_by_year = dataset[dataset['year'] == year]
dataset_by_year_and_cont=dataset_by_year[dataset_by_year['continent'] == continent]
data_dict = {
'x': list(dataset_by_year_and_cont['lifeExp']),
'y': list(dataset_by_year_and_cont['gdpPercap']),
'mode': 'markers',
'text': list(dataset_by_year_and_cont['country']),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'size': list(dataset_by_year_and_cont['pop'])
},
'name': continent
}
figure['data'].append(data_dict)
# make frames
for year in years:
frame = {'data': [], 'name': str(year)}
for continent in continents:
dataset_by_year = dataset[dataset['year'] == int(year)]
dataset_by_year_and_cont=dataset_by_year[dataset_by_year['continent'] == continent]
data_dict = {
'x': list(dataset_by_year_and_cont['lifeExp']),
'y': list(dataset_by_year_and_cont['gdpPercap']),
'mode': 'markers',
'text': list(dataset_by_year_and_cont['country']),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'size': list(dataset_by_year_and_cont['pop'])
},
'name': continent
}
frame['data'].append(data_dict)
figure['frames'].append(frame)
slider_step = {'args': [
[year],
{'frame': {'duration': 300, 'redraw': False},
'mode': 'immediate',
'transition': {'duration': 300}}
],
'label': year,
'method': 'animate'}
sliders_dict['steps'].append(slider_step)
figure['layout']['sliders'] = [sliders_dict]
iplot(figure, config=config)
I hope this helps you resolve your issue, the slider looks great :)
I was trying to recreate this example in a Jupyter notebook.
https://plot.ly/python/gapminder-example/
but was getting this error:
PlotlyDictKeyError: 'slider' is not allowed in 'layout'
Path To Error: ['layout']['slider']
Valid attributes for 'layout' at path ['layout'] under parents ['figure']:
['angularaxis', 'annotations', 'autosize', 'bargap', 'bargroupgap',
'barmode', 'barnorm', 'boxgap', 'boxgroupgap', 'boxmode', 'calendar',
'direction', 'dragmode', 'font', 'geo', 'height', 'hiddenlabels',
'hiddenlabelssrc', 'hidesources', 'hoverlabel', 'hovermode', 'images',
'legend', 'mapbox', 'margin', 'orientation', 'paper_bgcolor',
'plot_bgcolor', 'radialaxis', 'scene', 'separators', 'shapes',
'showlegend', 'sliders', 'smith', 'ternary', 'title', 'titlefont',
'updatemenus', 'width', 'xaxis', 'yaxis']
Run `<layout-object>.help('attribute')` on any of the above.
'<layout-object>' is the object at ['layout']
The animation runs without the slider dict added to layout and the slider is visible and operational, but does not change the graph. When I move the slider it produces the following error in the console:
Uncaught (in promise) undefined
Update:
I checked the graph you have, I am observing the below error sometimes.
Uncaught (in promise) undefined
This error might by due to plotly missing an click or other event, but this is internally within the plotly.js file, if you go to Plotly Slider animation link and to the slider animation section, click play and click on the slider while play is running we get this error, even when I click on pause I get this error. But the animation keeps on playing if I press the play again, hence there is no major impact! It's just that an event is not handled properly.
So as in the case of the graph you provided, I can get the animation working fine, eventhough I get the error (Uncaught (in promise) undefined) I am still able to play the animation!
You can use either iplot(fig, validate=False) or plot(fig) to show the graphs in Python with the animation!
Answer:
The error is because the layout object has a property called sliders not slider, so wherever you are using slider under layout, please change this, also this plot is very complicated and may have other errors also, please share the code, for debugging. But for now this will be my answer.
Before:
['layout']['slider']
After:
['layout']['sliders']
Please replace all the slider properties that are related to layout, these need to be changed to sliders.
References:
I have handled issues related to this particular slider animated Plotly graph. Please refer to them if need, they may help solve your issue!
Plotly Animated Bubble Chart No Data in the Plot
Plotly Error Invalid Figure or Data Argument
Plotly Icreate Animations Offline on Jupyter Notebook
You are probably hitting this error because of a typo in that notebook. It should be sliders instead of slider, see the docs.
The other error too, seems to be caused by this typo. It seems that this code is in an event handler that gets triggered whenever you move the slider.
So below line (and similar ones):
figure['layout']['slider']
should be corrected to:
figure['layout']['sliders']
Here is the code for that example:
import plotly.plotly as py
import plotly.graph_objs as go
from plotly.grid_objs import Grid, Column
from plotly.tools import FigureFactory as FF
import pandas as pd
import time
url = 'https://raw.githubusercontent.com/plotly/datasets/master/gapminderDataFiveYear.csv'
dataset = pd.read_csv(url)
table = FF.create_table(dataset.head(10))
py.iplot(table, filename='animations-gapminder-data-preview')
years_from_col = set(dataset['year'])
years_ints = sorted(list(years_from_col))
years = [str(year) for year in years_ints]
years.remove('1957')
# make list of continents
continents = []
for continent in dataset['continent']:
if continent not in continents:
continents.append(continent)
columns = []
# make grid
for year in years:
for continent in continents:
dataset_by_year = dataset[dataset['year'] == int(year)]
dataset_by_year_and_cont = dataset_by_year[dataset_by_year['continent'] == continent]
for col_name in dataset_by_year_and_cont:
# each column name is unique
column_name = '{year}_{continent}_{header}_gapminder_grid'.format(
year=year, continent=continent, header=col_name
)
a_column = Column(list(dataset_by_year_and_cont[col_name]), column_name)
columns.append(a_column)
# upload grid
grid = Grid(columns)
url = py.grid_ops.upload(grid, 'gapminder_grid'+str(time.time()), auto_open=False)
figure = {
'data': [],
'layout': {},
'frames': [],
'config': {'scrollzoom': True}
}
# fill in most of layout
figure['layout']['xaxis'] = {'range': [30, 85], 'title': 'Life Expectancy', 'gridcolor': '#FFFFFF'}
figure['layout']['yaxis'] = {'title': 'GDP per Capita', 'type': 'log', 'gridcolor': '#FFFFFF'}
figure['layout']['hovermode'] = 'closest'
figure['layout']['plot_bgcolor'] = 'rgb(223, 232, 243)'
figure['layout']['sliders'] = {
'args': [
'slider.value', {
'duration': 400,
'ease': 'cubic-in-out'
}
],
'initialValue': '1952',
'plotlycommand': 'animate',
'values': years,
'visible': True
}
figure['layout']['updatemenus'] = [
{
'buttons': [
{
'args': [None, {'frame': {'duration': 500, 'redraw': False},
'fromcurrent': True, 'transition': {'duration': 300, 'easing': 'quadratic-in-out'}}],
'label': 'Play',
'method': 'animate'
},
{
'args': [[None], {'frame': {'duration': 0, 'redraw': False}, 'mode': 'immediate',
'transition': {'duration': 0}}],
'label': 'Pause',
'method': 'animate'
}
],
'direction': 'left',
'pad': {'r': 10, 't': 87},
'showactive': False,
'type': 'buttons',
'x': 0.1,
'xanchor': 'right',
'y': 0,
'yanchor': 'top'
}
]
sliders_dict = {
'active': 0,
'yanchor': 'top',
'xanchor': 'left',
'currentvalue': {
'font': {'size': 20},
'prefix': 'Year:',
'visible': True,
'xanchor': 'right'
},
'transition': {'duration': 300, 'easing': 'cubic-in-out'},
'pad': {'b': 10, 't': 50},
'len': 0.9,
'x': 0.1,
'y': 0,
'steps': []
}
custom_colors = {
'Asia': 'rgb(171, 99, 250)',
'Europe': 'rgb(230, 99, 250)',
'Africa': 'rgb(99, 110, 250)',
'Americas': 'rgb(25, 211, 243)',
#'Oceania': 'rgb(9, 255, 255)'
'Oceania': 'rgb(50, 170, 255)'
}
col_name_template = '{year}_{continent}_{header}_gapminder_grid'
year = 1952
for continent in continents:
data_dict = {
'xsrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='lifeExp'
)),
'ysrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='gdpPercap'
)),
'mode': 'markers',
'textsrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='country'
)),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'sizesrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='pop'
)),
'color': custom_colors[continent]
},
'name': continent
}
figure['data'].append(data_dict)
for year in years:
frame = {'data': [], 'name': str(year)}
for continent in continents:
data_dict = {
'xsrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='lifeExp'
)),
'ysrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='gdpPercap'
)),
'mode': 'markers',
'textsrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='country'
)),
'marker': {
'sizemode': 'area',
'sizeref': 200000,
'sizesrc': grid.get_column_reference(col_name_template.format(
year=year, continent=continent, header='pop'
)),
'color': custom_colors[continent]
},
'name': continent
}
frame['data'].append(data_dict)
figure['frames'].append(frame)
slider_step = {'args': [
[year],
{'frame': {'duration': 300, 'redraw': False},
'mode': 'immediate',
'transition': {'duration': 300}}
],
'label': year,
'method': 'animate'}
sliders_dict['steps'].append(slider_step)
figure['layout']['sliders'] = [sliders_dict]
py.icreate_animations(figure, 'gapminder_example'+str(time.time()))
Note: Strange but the code executed successfully for me with the above mentioned typo as well!
Demo output.
You need plotly >= 2.0.0
try
pip install plotly --upgrade
As others have mentioned, the documentation is incorrect. But simply replacing all slider with sliders will still give an error. Therefore, here's a self contained example.
http://nbviewer.jupyter.org/gist/empet/365cf202391bf7a58021388fadd52004