How to use Spatial Smoothing on relativities at postcode level in Python - python

I'm after some help with how I can use Spatial smoothing on my data which I have attached.
What I'd like is for neighboring postcodes to not have such a large difference in relativities.
Just to provide little more about these relativities - they are used in calculating the premium charged to a customer for home insurance.
I also have a postcode file which contains for all postcodes in the UK their longitude and latitude coordinates.
Please see attached a small subsection of my data for your reference.

Related

Open source data of the world map that includes longitude and latitude information along with land and sea indicator

I've came across OpenStreetMap and it's too large with unnecessary data for my use case.
What I'm searching for is an exact replica of the world map tagged with longitude and latitude information along with an indicator showing whether it's a land or sea, with clear contours of land. I need to be able to draw objects on it and plot it using scripting language like Python. Longitude and latitude information are required for objects I'm plotting and clear contours of land to verify that my plot overlayed above my region of interest.
https://simplemaps.com/data/world-cities provides coordinates of countries and its cities but it'd be better to know how large each country is, hence the requirement of land contours. Any other representation of data is acceptable as long as it is able to indicate that there's land.
you might want to have a look at the NaturalEarth datasets...
https://www.naturalearthdata.com/downloads/
They provide land and ocean shapefiles (and much more) which should fit your purpose.

How to broaden coordinates from a location query?

I have an application where the user can enter a location query such as :
Corinth
The application will add a postfix ",Greece" and calculate the latitude and longitude coordinates for the city. So, the query for the coordinates will be Corinth, Greece.
After, that the application returns the weather forecast for that location using DarkSkyApi. I want to only return forecasts for my country, which is Greece.
The problem is that if a user types : London. Then, the location will return the coordinates of some obscure location in Greece (eg. a cafe) and give a forecast for that location. I think this creates confusion, because it gives the impression of a bug and that the app can predict forecasts for other countries as well.
On the one hand, I want it to be specific enough to work for cities (eg. Athens), villages & neighborhoods (eg. Acropolis). On the other, not for minor locations (eg. cafes).
How can I go about fixing that? If somebody gives a very minor location or foreign city, I would like it to default to the capital city 'Athens', or somehow "broaden" the coordinates to the nearest city. Is there a way, to limit the coordinates somehow or filter the locations ? Is there a way to determine whether the coordinates are for a minor or large location ?

Create Geographical Grid based on data using K-D Trees (python)

For my research, I need to divide the geographical area of a city (i.e.Chicago or New York) using a grid. Later, I have data points consisting of GPS longitude and latitude location that I want to associate to its corresponding cell in the grid.
The simplest way to do this is dividing the space into squared cells of same size. However, this will lead to cells with very few points in non-populated (rural areas) areas and cells with a high number of points (city centre). In order to have a more fair representation and the relation between the number of points and cell size, an adaptative grid that create cells of size based on data density would be a better option.
I came across this paper that utilise a K-D tree to do the space partition and retrieve the cells from the nodes. However, I cannot find any implementation (in python) that does that. Many of the implementations out there only index data points in the tree to perform Nearest Neighbour search, but they not provide code to extract the polygon-rectangles that k-d tree generates.
For example, given the following image:
My resulting grid will contain 5 cells (node1 to node5) where each cell contains the associated data points.
Any idea on how to do that?
Anyone knows any implementation?
Many thanks,
David

Retrieve polygon label from shape file based on latitude and longitude

I am new to working with geospatial data in Python, and I need some help with what I hope is a simple task.
I have a shape file of New York City Police Precincts, which can be found here, and I have a dataframe with two columns, ‘Latitude’ and ‘Longitude’, that are associated with NYC 311 call incident locations. Unfortunately, 311 data does not contain a Police Precinct column, so I am hoping to add that feature myself.
I believe I need to retrieve the polygon label (i.e. ‘5th Precinct’ - or however it phrased) for a given Lat,Long coordinate.
Any help would be greatly appreciated.

GeoDjango distance search

I want to use GeoDjango to do basic location searches. Specifically I want to give the search function a ZIP code/city/county and find all the ZIP codes/cities/counties within 5mi, 10mi, 20mi, etc. I found the following paragraph in the documentation:
Using a geographic coordinate system may introduce complications for the developer later on. For example, PostGIS does not have the capability to perform distance calculations between non-point geometries using geographic coordinate systems, e.g., constructing a query to find all points within 5 miles of a county boundary stored as WGS84. [6]
What does this exactly mean if I want to use PostGIS and to be able to do the searches described above across the USA? The docs suggest using a projected coordinate system to cover only a specific region. I need to cover the whole country so this I suppose is not an option.
Basically in the end I want to be able to find neighbouring ZIP codes/cities/counties given a starting location and distance. I don't really care how this is done on a technical level.
Also where would I find a database that contains the geographic boundaries of ZIP codes/cities/counties in the USA that I can import into a GeoDjango model?
UPDATE
I found a database of that contains the latitude and longitude coordinates of all ZIP codes in the USA here. My plan is to import these points into a GeoDjango model and use PostGis to construct queries that can find other points within x miles from a given point. This gets around the issue raised in the documentation because all the ZIP codes are treated as points instead of as polygons. This is fine for my use case because perfect accuracy is not something I care about.
The good: the data file is free
The bad: this data is from the 2000 census so it is quite dated
The somewhat hopeful: the United States Census Bureau conducts a census every 10 years and it is almost 2010
The conclusion: it's good enough for me
To get around the limitation in the quote, you can just take the centroid of the zipcode region provided by the user, and then from that point find all zipcode regions that intersect a 5, 10 or whatever mile circle emanating from that point. I'm not sure how that would be achieved in geodjango, but with postgis it's definitely possible.
The limitation you quoted basically says you can't write a query that says "give me all points that are within 5 miles on the inside of the border of Ohio."
In [1]: o = Place.objects.get(pk=2463583) # Oakland, CA
In [2]: sf = Place.objects.get(pk=2487956) # San Francisco, CA
In [3]: o.coords.transform(3410) # use the NSIDC EASE-Grid Global projection
In [4]: sf.coords.transform(3410) # use the NSIDC EASE-Grid Global projection
In [5]: o.coords.distance(sf.coords) # find the distance between Oakland and San Francisco (in meters)
Out[5]: 14401.942808571299

Categories