We Keep Coding

Find the nearest coordinates at a certain angle, from a set of coordinates

Out of a list of given coordinates, I need to identify the latitude and longitude that are closest to a particular coordinate point.But I must also check that it is the right direction along the same line (with a certain maximum deviation of, say 10 degrees).
I have tried fooling around with the haversine distance formula, but I don't know how I can alter the code to find the closest coordinate at a certain angle instead of an sphere.
def dist(lat1, long1, lat2, long2):
# convert decimal degrees to radians
lat1, long1, lat2, long2 = map(radians, [lat1, long1, lat2, long2])
# haversine formula
dlon = long2 - long1
dlat = lat2 - lat1
a = sin(dlat/2)**2 + cos(lat1) * cos(lat2) * sin(dlon/2)**2
print(a)
c = 2 * asin(sqrt(a))
print(c)
# Radius of earth in kilometers is 6371
km = 6371 * c
return km
The code above is an example of haversine distance formula. Is there way to alter this code so that it disregards coordinates that are not at the right angle to the given point or should I look for something else (if so, do you guys have any ideas?)?

downsampling gps data using haversine formula using python

I have a high frequency of gps data which i want to downsample to every 50 meters ie keep gps latitude and longitude every 50 meter and discard inbetween points. I found a python code on the internet which basically calculates the distance between two points. But i am not sure how to basically read from a csv the lat and long values and feed it into the function and calculate the distance. If the distance reaches 50 meter i simply save that gps coordinates. So far, i have the following python code
from math import radians, cos, sin, asin, sqrt
def haversine(lon1, lat1, lon2, lat2):
lon1, lat1, lon2, lat2 = map(radians, [lon1, lat1, lon2, lat2])
# haversine formula
dlon = lon2 - lon1
dlat = lat2 - lat1
a = sin(dlat/2)**2 + cos(lat1) * cos(lat2) * sin(dlon/2)**2
c = 2 * asin(sqrt(a))
r = 6371 # Radius of earth in kilometers. Use 3956 for miles
return c * r
x1 = 52.19421607
x2 = 52.20000327
y1 = -1.484984011
y2 = -1.48533465
result = haversine(x1,y1,x2,y2) #need to give input from a csv
#if result is greater than 50m , save the coordinates
print(result)
How can i solve the problem?Any direction would be appreciated.

Here is a outline and a working code example - where I made some assumptions about which to keep/drop. I assume the dataframe is sorted.
First calculate distance to next point, indeed use haversine for lat/long pairs. This part is not fast in my implementation - you can find faster.
Use cumsum() of distances, to create distance groups, where group 1 is all distances below 50, group 2 between 50 and 100, etc...
Within each group, keep for instance only the first()
Note that this is approximately each 50 units based on group, so be aware this is different than take a point and jump to next point which is closest to 50 units away and repeat. But for data reduction purposes it should be fine.
Generate some random data around London.
import numpy as np
import sklearn
import pandas as pd
LONDON = (51.509865, -0.118092)
random_gps = np.random.random( (10000,2) ) / 25
random_gps[:,0] += np.arange(random_gps.shape[0]) / 25
random_gps[:,0] += LONDON[0]
random_gps[:,1] += LONDON[1]
gps_data = pd.DataFrame( random_gps, columns=["lat","long"] )
Shift the data to get the lat/long of the next point
gps_data['next_lat'] = gps_data.lat.shift(1)
gps_data['next_long'] = gps_data.long.shift(1)
gps_data.head()
Define the distance metric. This part can be improved in terms of speed by using vector expressions with numpy, so if speed is important change this part.
from sklearn.neighbors import DistanceMetric
dist = DistanceMetric.get_metric('haversine')
EARTH_RADIUS = 6371.009
def haversine_distance(row):
point_a = np.array([[row.lat, row.long]])
point_b = np.array([[row.next_lat, row.next_long]])
return EARTH_RADIUS * dist.pairwise(np.radians(point_a), np.radians(point_b) )[0][0]
and apply our distance function (slow part, which can be improved)
gps_data["distance_to_next"] = gps_data.apply( haversine_distance, axis=1)
gps_data["distance_cumsum"] = gps_data.distance_to_next.cumsum()
Finally, create groups and drop. (!) The haversine is returning the distance in KM - so here i wrongly did an example of 50 km instead of meters.
gps_data["distance_group"] = gps_data.distance_cumsum // 50
filtered = gps_data.groupby(['distance_group']).first()

Why Manhattan Distance with haversine formula for geolocalizations is not accurate? [Python]

I want to compute the "MANHATTAN DISTANCE" also called "CITY BLOCK DISTANCE" among pairs of coordinates with LAT, LNG.
Following this post Manhattan Distance for two geolocations I had computed the distance using the haversine formula:
source = (45.070060, 7.663708)
target = (45.068250, 7.663492)
This is my computation:
from math import radians, sin, asin, sqrt, atan2
# convert decimal degrees to radians
lat1, lon1, lat2, lon2 = map(radians, [source[0], source[1], target[0], target[1]])
#haversine formula for delta_lat
dlat = lat2 - lat1
a = sin(dlat / 2) ** 2
c = 2 * atan2(sqrt(a), sqrt(1-a)))
r = 6371
lat_d = c * r
# haversine formula for delta_lon
dlon = lon2 - lon1
a = sin(dlon / 2) ** 2
c = 2 * atan2(sqrt(a), sqrt(1-a))
r = 6371
lon_d = c * r
print lat_d + lon_d
The problem is that my result is 225m, while Google Maps says 270m.
Trying again to compute the distance among
source = (45.070060, 7.663708)
target = (45.072800, 7.665540)
the result I obtained is 508m while Google Maps says 350m.
I will appreciate if someone can help me understanding what is wrong here and how to improve this solution which is far from being acceptable.
Thank you!

I probably got the point on myself, the answer is that if you look at these pictures, which are the same posted above in the original question, you can understand that the haversine method I have implemented, computes the distance as the RED LINE in the image. For this reason in the first case I obtain 225m instead of 270m (lower because the red line is the hypotenuse of the triangle) while in the second case I obtained 508m instead of 350 (higher because sum of legs of the triangle). Hence the way to solve this problem should be ROTATE THE CITY MAP COUNTERCLOCKWISE to align the BLUE DOTTED line with the Y-AXIS of the cartesian reference.
Any suggestion will be appreciated. Thank You

IDW interpolation of point data using python and gdal

I have a CSV file with the Lat, Long and Rainfall Information. I would like to interpolate those point and create tiff file. Can any one can suggest me the easiest way to do that.
I am trying to using gdal_grid. I am very new on using gdal in python.

This is actually several questions. Assuming you have some scattered data for lats and longs you'll to build all the location were you want to make estimation (all lats and longs for the pixels of you Tiff image).
Once you have that you can use any of the solutions around to do IWD over your data (using a recent example in another question):
class Estimation():
# IWD. Check: https://stackoverflow.com/questions/36031338/interpolate-z-values-in-a-3d-surface-starting-from-an-irregular-set-of-points/36037288#36037288
def __init__(self,lon,lat,values):
self.x = lat
self.y = lon
self.v = values
def estimate(self,x,y,using='ISD'):
"""
Estimate point at coordinate x,y based on the input data for this
class.
"""
if using == 'ISD':
return self._isd(x,y)
def _isd(self,x,y):
#d = np.sqrt((x-self.x)**2+(y-self.y)**2)
d = x.copy()
for i in range(d.shape[0]):
d[i] = haversine(self.x[i],self.y[i],x,y)
if d.min() > 0:
v = np.sum(self.v*(1/d**2)/np.sum(1/d**2))
return v
else:
return self.v[d.argmin()]
The code above is actually adapted to calculate distance with the Haversine formula (which gives great-circle distances between two points on a sphere from their longitudes and latitudes). Notice again you can find all sorts of solutions for the haversine distance like this one:
def haversine(lon1, lat1, lon2, lat2):
"""
Check: https://stackoverflow.com/questions/15736995/how-can-i-quickly-estimate-the-distance-between-two-latitude-longitude-points
Calculate the great circle distance between two points
on the earth (specified in decimal degrees)
"""
# convert decimal degrees to radians
lon1, lat1, lon2, lat2 = map(radians, [lon1, lat1, lon2, lat2])
# haversine formula
dlon = lon2 - lon1
dlat = lat2 - lat1
a = sin(dlat/2)**2 + cos(lat1) * cos(lat2) * sin(dlon/2)**2
c = 2 * asin(sqrt(a))
km = 6367 * c
return km
Finally once you have your array ready you should just build the Tiff using GDAL. For this check the following question for which I quote a part of it's solution:
driver = gdal.GetDriverByName('GTiff')
ds = driver.Create('output.tif',xsize, ysize, 1, gdal.GDT_Float32, )
# this assumes the projection is Geographic lat/lon WGS 84
srs = osr.SpatialReference()
srs.ImportFromEPSG(4326)
ds.SetProjection(srs.ExportToWkt())
gt = [ulx, xres, 0, uly, 0, yres ]
ds.SetGeoTransform(gt)
outband=ds.GetRasterBand(1)
outband.SetStatistics(np.min(mag_grid), np.max(mag_grid), np.average(mag_grid), np.std(mag_grid))
outband.WriteArray(mag_grid)

How to compare great circle distance with euclidean distance of two sphere points using python?

I am trying to check the error that is introduced when you compute the distance of two points on earth with the euclidean distance instead of using the great circle distance (gcd). I have two points that are defined by their lattitude and longtitude.
I used the python geopy framework for the great circle distance. Here the code for the gcd:
def measure(self, a, b):
a, b = Point(a), Point(b)
lat1, lng1 = radians(degrees=a.latitude), radians(degrees=a.longitude)
lat2, lng2 = radians(degrees=b.latitude), radians(degrees=b.longitude)
sin_lat1, cos_lat1 = sin(lat1), cos(lat1)
sin_lat2, cos_lat2 = sin(lat2), cos(lat2)
delta_lng = lng2 - lng1
cos_delta_lng, sin_delta_lng = cos(delta_lng), sin(delta_lng)
d = atan2(sqrt((cos_lat2 * sin_delta_lng) ** 2 +
(cos_lat1 * sin_lat2 -
sin_lat1 * cos_lat2 * cos_delta_lng) ** 2),
sin_lat1 * sin_lat2 + cos_lat1 * cos_lat2 * cos_delta_lng)
return self.RADIUS * d
So or two points:
p1=[39.8616,-75.0748], p2=[-7.30933,112.76]
the
gcd = 78.8433004543197 klm
using the great_circle(p1,p2).kilometers function from geopy
I then transformed these two points in cartesian coordinates using this formula:
def spherical_to_cartesian(r,la,lo):
x=r*np.sin(90-la)*np.cos(lo)
y=r*np.sin(90-la)*np.sin(lo)
z=r*np.cos(90-la)
return (x,y,z)
where r=6372.795, which results in the following cartesians coordinates
p1=[ -765.81579368, -256.69640558, 6321.40405587],
p2=[480.8302149,-168.64726394,-6352.39140142]
Then by typing: np.linalg.norm(p2-p1) i am getting 1103.4963114787836 as their euclidean norm which doesn't seem reasonable compared with ~78klm from the gcd. Am i inffering sth wrong?

Python includes two functions in the math package; radians converts degrees to radians, and degrees converts radians to degrees.
The method sin() returns the sine of x, in radians.
import math
def spherical_to_cartesian(r,la,lo):
rlo = math.radians(lo)
rla = math.radians(90-la)
x=r*np.sin(rla)*np.cos(rlo)
y=r*np.sin(rla)*np.sin(rlo)
z=r*np.cos(rla)
return (x,y,z)