I having trouble getting some air pollution data to show different colors in a chloropleth map using folium. Please let me know where my code may be throwing an error. I think it is the key_on parameter but need help.
This is how my map turns out.
enter image description here
What I would like is for the mean concentration of the air pollution data to show up on the map but the map is still greyed out.
Here are the files I used:
Geojson file - Used "download zip" in upper right of this website https://gist.github.com/miguelpaz/edbc79fc55447ae736704654b3b2ef90#file-uhf42-geojson
Data file - Exported data from here https://a816-dohbesp.nyc.gov/IndicatorPublic/VisualizationData.aspx?id=2023,719b87,122,Summarize
Here is my code:
import geopandas as gpd
import folium
#clean pollution data
pm_df1 = pd.read_csv('/work/Fine Particulate Matter (PM2.5).csv',header = 5, usecols = ['GeoTypeName', 'Borough','Geography', 'Geography ID','Mean (mcg per cubic meter)'], nrows = 140)
#limit dataframe to rows with neighborhood (UHF 42) that matches geojson file
pm_df2 = pm_df1[(pm_df1['GeoTypeName'] == 'Neighborhood (UHF 42)')]
pm_df2
#clean geojson file
uhf_df2 = gpd.read_file('/work/uhf42.geojson', driver='GeoJSON')
uhf_df2.head()
#drop row 1 that has no geography
uhf_df3 = uhf_df2.iloc[1:]
uhf_df3.head()
## create a map
pm_testmap = folium.Map(location=[40.65639,-73.97379], tiles = "cartodbpositron", zoom_start=10)
# generate choropleth map
pm_testmap.choropleth(
geo_data=uhf_df3,
data=pm_df2,
columns=['Geography', 'Mean (mcg per cubic meter)'],
key_on='feature.properties.uhf_neigh', #think this is where I mess up.
fill_color='BuPu',
fill_opacity=0.2,
line_opacity=0.7,
legend_name='Average dust concentration',
smooth_factor=0)
# display map
pm_testmap
The problem with key_on is right as you think.
Both data have the name of UHF written on them, but in a completely different form.
In order to link these two, the data must first be preprocessed.
I don't know your data.
It would be nice if you could df.head() the two data to show them, but I'll explain based on the data I checked through the link you provided.
In your geojson file, uhf_neigh simply says Northeast Bronx. However, your PM data appears to have the region listed as Bronx: Northeast Bronx. The following process seems to be necessary to unify your local name before plotting map.
uhf_df2['UHF_NEIGH'] = uhf_df2['BOROUGH']+ ': ' + uhf_df2['UHF_NEIGH']
I tried to run it with your data and code, but it was not even displaying the map. There should be no problem in your code because you have associated the place name in the data frame with the place name in geojson. I gave up on the string association and changed the association to a place name code association, and the map was displayed. The provided csv file failed to load, so I deleted the unnecessary lines and loaded it. Also, I read the file as a json file instead of geopandas.
import pandas as pd
import geopandas as gpd
import json
import folium
pm_df1 = pd.read_csv('./data/test_20211221.csv')
pm_df1 = pm_df1[['GeoTypeName', 'Borough', 'Geography', 'Geography ID', 'Mean (mcg per cubic meter)']]
pm_df2 = pm_df1[(pm_df1['GeoTypeName'] == 'Neighborhood (UHF 42)')]
with open('./data/uhf42.geojson') as f:
uhf_df3 = json.load(f)
pm_testmap = folium.Map(location=[40.65639,-73.97379], tiles = "cartodbpositron", zoom_start=10)
# generate choropleth map
pm_testmap.choropleth(
geo_data=uhf_df3,
data=pm_df2,
columns=['Geography ID', 'Mean (mcg per cubic meter)'],
key_on='feature.properties.uhfcode', #think this is where I mess up.
fill_color='BuPu',
fill_opacity=0.2,
line_opacity=0.7,
legend_name='Average dust concentration',
smooth_factor=0)
# display map
pm_testmap
Related
Good afternoon Stack!!
Pretty straight forward question here. I built a Choropleth to show the density of the soldiers I have where in my army reserve unit. The problem I have is that the map results come back in pretty much monochrome. its not the green scale I used in fill. Am I dumb and missed something??
For obvious reasons I'm not going to share what the map looks like now so any advice is warmly welcomed.
import folium
import pandas as pd
import json
from folium import plugins
##Create data frame
df = pd.read_csv('soldier data.csv')
numSoldiersSeries = df.groupby('Zip').count()
soldiersByZip = pd.DataFrame()
soldiersByZip['ZCTA5CE10'] = [str(i) for i in numSoldiersSeries.index]
soldiersByZip['numSoldiers'] = numSoldiersSeries.values
#Create Zipcode Overlay
with open('pa.geojson', 'r') as f:
pa = json.load(f)
##Create starting map
neMap = folium.Map(location=[40.767937,-73.982155], zoom_start=7)
##Create color overlay
folium.Choropleth(geo_data = pa, data=soldiersByZip,
columns=['ZCTA5CE10', 'numSoldiers'],
key_on='feature.properties.ZCTA5CE10',
fill_color='YlGn',
fill_opacity=.2,
line_opacity=1).add_to(neMap)
neMap.save('nemap.html')
I have a dataframe created on a csv file about Italian Covid-19 spread all over regions. I was trying to create a px.choropleth plot in which showing Total Positive values for every regions in Italy.
This the code tried:
italy_regions=[i for i in region['Region'].unique()]
fig = px.choropleth(italy_last, locations="Country",
locationmode=italy_regions,
color=np.log(italy_last["TotalPositive"]),
hover_name="Region", hover_data=['TotalPositive'],
color_continuous_scale="Sunsetdark",
title='Regions with Positive Cases')
fig.update(layout_coloraxis_showscale=False)
fig.show()
Now I report some info: 'Country' is the name given to my dataframe and is filled only with the same values: 'Italy'. If I only input 'location="Country"' the graph is fine and I can see Italy colored into the world map.
The problems start when I try to make pyplot color my regions. As I'm a newbye in pyplot express, I read some examples and I thought I had to create a list of italian regions names and then put into 'choropleth' as input for 'barmode'.
Clearly I'm wrong.
So, what is the procedure to follow to make it run (if any)?
In case of need, I can provide both the csv file that the jupyter file I'm working on.
You need to provide a geojson with Italian region borders as geojson parameter to plotly.express.choropleth, for instance this one
https://gist.githubusercontent.com/datajournalism-it/48e29e7c87dca7eb1d29/raw/2636aeef92ba0770a073424853f37690064eb0ea/regioni.geojson
If you use this one, you need to explicitly pass featureidkey='properties.NOME_REG' as a parameter of plotly.express.choropleth.
Working example:
import pandas as pd
import requests
import plotly.express as px
regions = ['Piemonte', 'Trentino-Alto Adige', 'Lombardia', 'Puglia', 'Basilicata',
'Friuli Venezia Giulia', 'Liguria', "Valle d'Aosta", 'Emilia-Romagna',
'Molise', 'Lazio', 'Veneto', 'Sardegna', 'Sicilia', 'Abruzzo',
'Calabria', 'Toscana', 'Umbria', 'Campania', 'Marche']
# Create a dataframe with the region names
df = pd.DataFrame(regions, columns=['NOME_REG'])
# For demonstration, create a column with the length of the region's name
df['name_length'] = df['NOME_REG'].str.len()
# Read the geojson data with Italy's regional borders from github
repo_url = 'https://gist.githubusercontent.com/datajournalism-it/48e29e7c87dca7eb1d29/raw/2636aeef92ba0770a073424853f37690064eb0ea/regioni.geojson'
italy_regions_geo = requests.get(repo_url).json()
# Choropleth representing the length of region names
fig = px.choropleth(data_frame=df,
geojson=italy_regions_geo,
locations='NOME_REG', # name of dataframe column
featureidkey='properties.NOME_REG', # path to field in GeoJSON feature object with which to match the values passed in to locations
color='name_length',
color_continuous_scale="Magma",
scope="europe",
)
fig.update_geos(showcountries=False, showcoastlines=False, showland=False, fitbounds="locations")
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})
fig.show()
Output image
The task is to make an adress popularity map for Moscow. Basically, it should look like this:
https://nbviewer.jupyter.org/github/python-visualization/folium/blob/master/examples/GeoJSON_and_choropleth.ipynb
For my map I use public geojson: http://gis-lab.info/qa/moscow-atd.html
The only data I have - points coordinates and there's no information about the district they belong to.
Question 1:
Do I have to manually calculate for each disctrict if the point belongs to it, or there is more effective way to do this?
Question 2:
If there is no way to do this easier, then, how can I get all the coordinates for each disctrict from the geojson file (link above)?
import pandas as pd
import numpy as np
import geopandas as gpd
from shapely.geometry import Point
Reading in the Moscow area shape file with geopandas
districts = gpd.read_file('mo-shape/mo.shp')
Construct a mock user dataset
moscow = [55.7, 37.6]
data = (
np.random.normal(size=(100, 2)) *
np.array([[.25, .25]]) +
np.array([moscow])
)
my_df = pd.DataFrame(data, columns=['lat', 'lon'])
my_df['pop'] = np.random.randint(500, 100000, size=len(data))
Create Point objects from the user data
geom = [Point(x, y) for x,y in zip(my_df['lon'], my_df['lat'])]
# and a geopandas dataframe using the same crs from the shape file
my_gdf = gpd.GeoDataFrame(my_df, geometry=geom)
my_gdf.crs = districts.crs
Then the join using default value of 'inner'
gpd.sjoin(districts, my_gdf, op='contains')
Thanks to #BobHaffner, I tried to solve the problem using geopandas.
Here are my steps:
I download a shape-files for Moscow using this link click
From a list of tuples containing x and y (latitude and logitude) coordinates I create list of Points (docs)
Assuming that in the dataframe from the first link I have polygons I can write a simple loop for checking if the Point is inside this polygon. For details read this.
I have this code:
import pandas as pd
import numpy as np
from geopandas import GeoDataFrame
import geopandas
from shapely.geometry import LineString, Point
import matplotlib.pyplot as plt
import contextily
''' Do Something'''
df = start_stop_df.drop('track', axis=1)
crs = {'init': 'epsg:4326'}
gdf = GeoDataFrame(df, crs=crs, geometry=geometry)
ax = gdf.plot()
contextily.add_basemap(ax)
ax.set_axis_off()
plt.show()
Basically, this generates a background map that is in Singapore. However, when I run it, I get the following error: HTTPError: Tile URL resulted in a 404 error. Double-check your tile url:http://tile.stamen.com/terrain/29/268436843/268435436.png
However, it still produces this image:
How can I change the Tile URL? I would still like to have the map of Singapore as the base layer.
EDIT:
Also tried including this argument to add_basemap:
url ='https://www.openstreetmap.org/#map=12/1.3332/103.7987'
Which produced this error:
OSError: cannot identify image file <_io.BytesIO object at 0x000001CC3CC4BC50>
First make sure that your GeoDataframe is in Web Mercator projection (epsg=3857). Once your Geodataframe is correctly georeferenced, you can achieve this by Geopandas reprojection:
df = df.to_crs(epsg=3857)
Once you have this done, you easily choose any of the supported map styles. A full list can be found in contextily.sources module, at the time of writing:
### Tile provider sources ###
ST_TONER = 'http://tile.stamen.com/toner/tileZ/tileX/tileY.png'
ST_TONER_HYBRID = 'http://tile.stamen.com/toner-hybrid/tileZ/tileX/tileY.png'
ST_TONER_LABELS = 'http://tile.stamen.com/toner-labels/tileZ/tileX/tileY.png'
ST_TONER_LINES = 'http://tile.stamen.com/toner-lines/tileZ/tileX/tileY.png'
ST_TONER_BACKGROUND = 'http://tile.stamen.com/toner-background/tileZ/tileX/tileY.png'
ST_TONER_LITE = 'http://tile.stamen.com/toner-lite/tileZ/tileX/tileY.png'
ST_TERRAIN = 'http://tile.stamen.com/terrain/tileZ/tileX/tileY.png'
ST_TERRAIN_LABELS = 'http://tile.stamen.com/terrain-labels/tileZ/tileX/tileY.png'
ST_TERRAIN_LINES = 'http://tile.stamen.com/terrain-lines/tileZ/tileX/tileY.png'
ST_TERRAIN_BACKGROUND = 'http://tile.stamen.com/terrain-background/tileZ/tileX/tileY.png'
ST_WATERCOLOR = 'http://tile.stamen.com/watercolor/tileZ/tileX/tileY.png'
# OpenStreetMap as an alternative
OSM_A = 'http://a.tile.openstreetmap.org/tileZ/tileX/tileY.png'
OSM_B = 'http://b.tile.openstreetmap.org/tileZ/tileX/tileY.png'
OSM_C = 'http://c.tile.openstreetmap.org/tileZ/tileX/tileY.png'
Keep in mind that you should not be adding actual x,y,z tile numbers in your tile URL (like you did in your "EDIT" example). ctx will take care of all this.
You can find a working copy-pastable example and further info at GeoPandas docs.
import contextily as ctx
# Dataframe you want to plot
gdf = GeoDataFrame(df, crs= {"init": "epsg:4326"}) # Create a georeferenced dataframe
gdf = gdf.to_crs(epsg=3857) # reproject it in Web mercator
ax = gdf.plot()
# choose any of the supported maps from ctx.sources
ctx.add_basemap(ax, url=ctx.sources.ST_TERRAIN)
ax.set_axis_off()
plt.show()
Contextily's default crs is epsg:3857. However, your data-frame is in different CRS. Use the following,refer the manual here:
ctx.add_basemap(ax, crs='epsg:4326', source=ctx.providers.Stamen.TonerLite)
Please, refer to this link for using different sources such as Stamen.Toner, Stamen.Terrain etc. (Stamen.Terrain is used as default).
Also, you can cast your data frame to EPSG:3857 by using df.to_crs(). In this case, you should skip crs argument inside ctx.add_basemap() function.
im too new to add a comment but I wanted to point out to those saying in the comments that they get a 404 error. Check you capitaliations, etc. Stamen's urls are specifc on this. For instance there is not an all caps call. It is only capitalize the first letter. For example:
ctx.add_basemap(ax=ax,url=ctx.providers.Stamen.Toner, zoom=10)
I have created a geoDataFrame using, and would like to create a Folium Map, plotting the population eat for each country. Do I have to create the Json file, or I can directly use the geoDataFrame file?
import folium
import fiona
import geopandas as gpd
world = fiona.open(gpd.datasets.get_path('naturalearth_lowres'))
world = gpd.GeoDataFrame.from_features([feature for feature in world])
world = world[(world.pop_est > 0) & (world.name != "Antarctica")]
I used folium.map and geojson function, but it failed to create correct JSON files.
Thanks for the help!
The m.cholopleth() code in #joris's answer is now deprecated. The following code produces the same result using the new folium.Chloropleth() function:
m = folium.Map()
folium.Choropleth(world, data=world,
key_on='feature.properties.name',
columns=['name', 'pop_est'],
fill_color='YlOrBr').add_to(m)
folium.LayerControl().add_to(m)
m
In recent releases of folium, you don't need to convert the GeoDataFrame to geojson, but you can pass it directly. Connecting the population column to color the polygons is still somewhat tricky to get correct:
m = folium.Map()
m.choropleth(world, data=world, key_on='feature.properties.name',
columns=['name', 'pop_est'], fill_color='YlOrBr')
m