We Keep Coding

How to find smallest cluster of location which are within a given distance

I have a set of co-ordinates in latitude and longitude format. I need to find the smallest cluster from these coordinates which are within say 50 mile distance to each other.
I am new to data science, how can I implement this in Python without using sklearn library.

Well, if you need to find distance and you're using XY coords, and you don't want to use extra libs, then maybe a place to start is to write a little funciton which finds the euclidean distance - you know a squared plus b squared equals c squared - Maybe something like this can get you started - this is a function which takes two tuples as coordinates, but it can also works for 3D coords if needed:
import math
def getEuclideanDistance(pointA, pointB):
xA,yA = pointA
# or for 3D coords, xA, yA, zA = pointA...
xB,yB = pointB
result = math.sqrt((xA-xB)**2 + (yA-yB)**2)
# or for distance in 3d:
# result = math.sqrt((xA-xB)**2 + (yA-yB)**2 + (zA-zB)**2)
return (result)
We're using the **2 operator to bring the result to the 2nd power, and math.sqrt to do the rest. I hope this puts you on the right track.

Know difference between two angles with only their sine and cosine?

Hi recently I was writing a program with a bunch dots that the player had to avoid. The dots changed direction, but can't turn too much in a given timeframe.
I had a method of doing this but was inefficient as I had to arcsine, knowing the cosed angle and sined angle, then sine and cosine that angle. I thought of just returning the cosine and sined angle, but theres one problem. Once I received the cosine and sine, I need too know if it is too different from my current state.
With the angle this would be easy as I would just have too see the difference, here's a model program,(the code I currently have uses the angle, and isn't very helpful). I have tried graphing sine and cosine and trying to observe any patterns, none obvious showed up.
import math
def sendTargRot():
#I actually use a fairly long method to find it's current rotation, but a random.random() is a fair replacement, so in my real thing there would be no angle calculation
pretendAngle = math.pi*random.random()-math.pi
pretendCosedX = math.cos(pretendAngle)
pretendSinedX = math.sin(pretendAngle)
def changeDotAngle():
targRot = sendTargRot
#make sure targRot is not too much from current rotation
#do stuff with the angle, change dot's rotation
If you want to just give me an algorithm without code that is just as acceptable!
EDIT: I can't really change sendTargRot so it gives a proper rotation, because that would require me knowing the current angle, and really it's just moving the problem.

To get angle between two vectors you can use atan2 function
angle = Math.atan2(a.x * b.y - a.y * b.x, a.x * b.x + a.y * b.y)
if you already have got cosines and sines (the same as coordinates on unit circle):
angle = Math.atan2(cos(a) * sin(b) - sin(a) * cos(b), cos(a) * cos(b) + sin(a) * sin(b))
This approach gives angle needed to rotate a until it coincides with b accounting for direction (in range -Pi..Pi)

How to compute distance of two geographical coordinates?

I read this question and implemented the accepted answer in Python (see below). It works in principle, but the results are consistently about 30% higher than expected (Czech Republic) - is that the expected accuracy of this algorithm?
To verify the algorithm, I used BoundingBox to get a bounding box with a known diagonal distance (building, two cities) and used the output coordinates as input for "my" algorithm.
Where is the problem?
my implementation?
the algorithm itself?
Python?
testing?
My implementation:
R= 6371 #km
dLat = math.radians(lat2-lat1)
dLon = math.radians(lon2-lon1)
lat1 = math.radians(lat1)
lat2 = math.radians(lat2)
a= math.sin(dLat/2)*math.sin(dLat/2) + math.sin(dLon/2) * math.sin(dLon/2) * math.cos(lat1) * math.cos(lat2)
c= 2 * math.atan2(math.sqrt(a), math.sqrt(1-a))
d = R * c;
return d

No, the algorithm is not supposed to have an error of that magnitude. The link specifies that you can expect around a 0.3% error.
I can not reproduce your results with your code, so I believe the error is with your testing.
Here's some testing data from a site with the distance between and coordinates of Prague and Brno in decimal degrees format:
lat_prague, long_prague = 50.0833, 14.4667
lat_brno, long_brno = 49.2000, 16.6333
expected_km = 184.21
Here are the testing results:
>>> def calc(lat1,lon1, lat2,lon2):
# ... your code ...
>>> calc(lat_prague,long_prague,lat_brno,long_brno)
184.34019283649852
>>> calc(lat_prague,long_prague,lat_brno,long_brno) / expected_km
1.0007067631317437
A wild guess: for locations in the Czech Republic, the error you're getting seems in the right order of magnitude for with mixing up latitude and longitude:
>>> calc(long_prague,lat_prague,long_brno,lat_brno)
258.8286271447481
>>> calc(long_prague,lat_prague,long_brno,lat_brno) / expected_km
1.405073704710646
This is apparently a known confusion. A coordinate specified as only a pair of numbers is ambiguous (for instance: both BoundingBox and the reference for the distance above use (long, lat), and the algorithm uses the ordering lat, long). When you come across the ambiguous format with an unfamiliar data source without a formal specification, you'll just have to sanity-check. Sites like Wikipedia will tell you unambiguously that Prague lies at "50°05′N 14°25′E" -- that is, very roughly, around 50 degrees latitude (north-south) and 14 degrees longitude (east-west).

pyEphem - Calculating Positions of non-Earthy Moons

I'm trying to get the Earth distance and the right ascension (relative to my observer point in Earth) of a satellite not orbiting the Earth, but pyEphem isn't returning the same properties as other solar bodies.
With Ganymede (the largest moon of Jupiter), for instance:
import math, ephem
Observer = ephem.city('London')
Observer.date = '2013-04-23'
Observer.pressure, Observer.elevation = 0, 100
moonGanymede = ephem.Ganymede(Observer)
print math.cos(moonGanymede.ra) # right ascension
print moonGanymede.earth_distance * ephem.meters_per_au # distance
I get this error:
AttributeError: 'Ganymede' object has no attribute 'earth_distance'
The ra attribute exists, but is it relative to my Observer or to Jupiter?
Seems to be relative to the Observer, since if I change the location, the value changes too.
I've read the documentation and I know that these properties are not defined for moons, but I have no idea how to compute those relative to the Earth given the additional defined properties of moon bodies:
On planetary moons, also sets:
Position of moon relative to planet (measured in planet radii)
x — offset +east or –west
y — offset +south or –north
z — offset +front or –behind
Doing:
print moonGanymede.x, moonGanymede.y, moonGanymede.z
Outputs:
-14.8928060532 1.52614057064 -0.37974858284
Since Jupiter has an average radius of 69173 kilometers, those values translate to:
moonGanymede.x = 1030200 kilometers (west)
moonGanymede.y = 105570 kilometers (south)
moonGanymede.z = 26268 kilometers (behind)
Given that I know the distance and right ascension of Jupiter relative to the Observer, how can I calculate the distance and right ascension of moonGanymede (also relative to the Observer)?
I'm using pyEphem 3.7.5.1 (with Python 2.7).

Just some thoughts; You probably need to do it two steps.
Get location of satellite relative to parent planet
Get location of planet relative to observer
Trigonometry calculation; get the location of satellite relative to observer.
You already did 1, and can easily do 2. Convert all values to x,y,z and add then back to angular. Or I'm sure you / ephym can do this for you directly.
HTH

I'm still trying to figure it out (if anyone spots something, please do tell), but it seems that if I do:
sqrt((-14.8928060532)^2 + (1.52614057064)^2 + (-0.37974858284)^2) = 14.9756130481
I'll always get a value that always falls within the min/max distance from orbit center (14.95 - 14.99).
Since that's specified in orbit center radii, I'll need to multiply it by 69173 * 1000 to get the SI unit:
14.9756130481 * 69173 * 1000 = 1.0359080813762213 * 10^9 meters
Since pyEphem deals in distances with AU:
print (1.0359080813762213 * 10**9) / ephem.meters_per_au # 0.00692461785302
At the same time, the Earth-Jupiter distance was 5.79160547256 AU.
Now, to get the distance, I should either add or subtract depending on the sign of the z coordinate:
5.79160547256 - 0.00692461785302 = 5.78468085470698 AU
Running the same code for today (now) returns 6.03799937821 which seems to very close to the value of 6.031 that WolframAlpha is returning at the present time, it doesn't match 100% but perhaps that could be accounted for by some different underlying ephemeris library or data source. Not sure...

Looks like right ascension, declination, azimuth, etc are computed correctly:
In [31]: g = ephem.Ganymede(Observer)
In [32]: j = ephem.Jupiter(Observer)
In [33]: g.ra, g.az, g.dec
Out[33]: (1.3024204969406128, 5.586287021636963, 0.38997682929039)
In [34]: j.ra, j.az, j.dec
Out[34]: (1.303646765055829, 5.5853118896484375, 0.39010250333236757)
Values for Ganimede and Jupiter are close enough, it looks like you get correct results for everything except distance to object.

How to calculate the area of a polygon on the earth's surface using python?

The title basically says it all. I need to calculate the area inside a polygon on the Earth's surface using Python. Calculating area enclosed by arbitrary polygon on Earth's surface says something about it, but remains vague on the technical details:
If you want to do this with a more
"GIS" flavor, then you need to select
an unit-of-measure for your area and
find an appropriate projection that
preserves area (not all do). Since you
are talking about calculating an
arbitrary polygon, I would use
something like a Lambert Azimuthal
Equal Area projection. Set the
origin/center of the projection to be
the center of your polygon, project
the polygon to the new coordinate
system, then calculate the area using
standard planar techniques.
So, how do I do this in Python?

Let's say you have a representation of the state of Colorado in GeoJSON format
{"type": "Polygon",
"coordinates": [[
[-102.05, 41.0],
[-102.05, 37.0],
[-109.05, 37.0],
[-109.05, 41.0]
]]}
All coordinates are longitude, latitude. You can use pyproj to project the coordinates and Shapely to find the area of any projected polygon:
co = {"type": "Polygon", "coordinates": [
[(-102.05, 41.0),
(-102.05, 37.0),
(-109.05, 37.0),
(-109.05, 41.0)]]}
lon, lat = zip(*co['coordinates'][0])
from pyproj import Proj
pa = Proj("+proj=aea +lat_1=37.0 +lat_2=41.0 +lat_0=39.0 +lon_0=-106.55")
That's an equal area projection centered on and bracketing the area of interest. Now make new projected GeoJSON representation, turn into a Shapely geometric object, and take the area:
x, y = pa(lon, lat)
cop = {"type": "Polygon", "coordinates": [zip(x, y)]}
from shapely.geometry import shape
shape(cop).area # 268952044107.43506
It's a very close approximation to the surveyed area. For more complex features, you'll need to sample along the edges, between the vertices, to get accurate values. All caveats above about datelines, etc, apply. If you're only interested in area, you can translate your feature away from the dateline before projecting.

The easiest way to do this (in my opinion), is to project things into (a very simple) equal-area projection and use one of the usual planar techniques for calculating area.
First off, I'm going to assume that a spherical earth is close enough for your purposes, if you're asking this question. If not, then you need to reproject your data using an appropriate ellipsoid, in which case you're going to want to use an actual projection library (everything uses proj4 behind the scenes, these days) such as the python bindings to GDAL/OGR or (the much more friendly) pyproj.
However, if you're okay with a spherical earth, it quite simple to do this without any specialized libraries.
The simplest equal-area projection to calculate is a sinusoidal projection. Basically, you just multiply the latitude by the length of one degree of latitude, and the longitude by the length of a degree of latitude and the cosine of the latitude.
def reproject(latitude, longitude):
"""Returns the x & y coordinates in meters using a sinusoidal projection"""
from math import pi, cos, radians
earth_radius = 6371009 # in meters
lat_dist = pi * earth_radius / 180.0
y = [lat * lat_dist for lat in latitude]
x = [long * lat_dist * cos(radians(lat))
for lat, long in zip(latitude, longitude)]
return x, y
Okay... Now all we have to do is to calculate the area of an arbitrary polygon in a plane.
There are a number of ways to do this. I'm going to use what is probably the most common one here.
def area_of_polygon(x, y):
"""Calculates the area of an arbitrary polygon given its verticies"""
area = 0.0
for i in range(-1, len(x)-1):
area += x[i] * (y[i+1] - y[i-1])
return abs(area) / 2.0
Hopefully that will point you in the right direction, anyway...

A bit late perhaps, but here is a different method, using Girard's theorem. It states that the area of a polygon of great circles is R**2 times the sum of the angles between the polygons minus (N-2)*pi where N is number of corners.
I thought this would be worth posting, since it doesn't rely on any other libraries than numpy, and it is a quite different method than the others. Of course, this only works on a sphere, so there will be some inaccuracy when applying it to the Earth.
First, I define a function to compute the bearing angle from point 1 along a great circle to point 2:
import numpy as np
from numpy import cos, sin, arctan2
d2r = np.pi/180
def greatCircleBearing(lon1, lat1, lon2, lat2):
dLong = lon1 - lon2
s = cos(d2r*lat2)*sin(d2r*dLong)
c = cos(d2r*lat1)*sin(d2r*lat2) - sin(lat1*d2r)*cos(d2r*lat2)*cos(d2r*dLong)
return np.arctan2(s, c)
Now I can use this to find the angles, and then the area (In the following, lons and lats should of course be specified, and they should be in the right order. Also, the radius of the sphere should be specified.)
N = len(lons)
angles = np.empty(N)
for i in range(N):
phiB1, phiA, phiB2 = np.roll(lats, i)[:3]
LB1, LA, LB2 = np.roll(lons, i)[:3]
# calculate angle with north (eastward)
beta1 = greatCircleBearing(LA, phiA, LB1, phiB1)
beta2 = greatCircleBearing(LA, phiA, LB2, phiB2)
# calculate angle between the polygons and add to angle array
angles[i] = np.arccos(cos(-beta1)*cos(-beta2) + sin(-beta1)*sin(-beta2))
area = (sum(angles) - (N-2)*np.pi)*R**2
With the Colorado coordinates given in another reply, and with Earth radius 6371 km, I get that the area is 268930758560.74808

Or simply use a library: https://github.com/scisco/area
from area import area
>>> obj = {'type':'Polygon','coordinates':[[[-180,-90],[-180,90],[180,90],[180,-90],[-180,-90]]]}
>>> area(obj)
511207893395811.06
...returns the area in square meters.

You can compute the area directly on the sphere, instead of using an equal-area projection.
Moreover, according to this discussion, it seems that Girard's theorem (sulkeh's answer) does not give accurate results in certain cases, for example "the area enclosed by a 30º lune from pole to pole and bounded by the prime meridian and 30ºE" (see here).
A more precise solution would be to perform line integral directly on the sphere. The comparison below shows this method is more precise.
Like all other answers, I should mention the caveat that we assume a spherical earth, but I assume that for non-critical purposes this is enough.
Python implementation
Here is a Python 3 implementation which uses line integral and Green's theorem:
def polygon_area(lats, lons, radius = 6378137):
"""
Computes area of spherical polygon, assuming spherical Earth.
Returns result in ratio of the sphere's area if the radius is specified.
Otherwise, in the units of provided radius.
lats and lons are in degrees.
"""
from numpy import arctan2, cos, sin, sqrt, pi, power, append, diff, deg2rad
lats = np.deg2rad(lats)
lons = np.deg2rad(lons)
# Line integral based on Green's Theorem, assumes spherical Earth
#close polygon
if lats[0]!=lats[-1]:
lats = append(lats, lats[0])
lons = append(lons, lons[0])
#colatitudes relative to (0,0)
a = sin(lats/2)**2 + cos(lats)* sin(lons/2)**2
colat = 2*arctan2( sqrt(a), sqrt(1-a) )
#azimuths relative to (0,0)
az = arctan2(cos(lats) * sin(lons), sin(lats)) % (2*pi)
# Calculate diffs
# daz = diff(az) % (2*pi)
daz = diff(az)
daz = (daz + pi) % (2 * pi) - pi
deltas=diff(colat)/2
colat=colat[0:-1]+deltas
# Perform integral
integrands = (1-cos(colat)) * daz
# Integrate
area = abs(sum(integrands))/(4*pi)
area = min(area,1-area)
if radius is not None: #return in units of radius
return area * 4*pi*radius**2
else: #return in ratio of sphere total area
return area
I wrote a somewhat more explicit version (and with many more references and TODOs...) in the sphericalgeometry package there.
Numerical Comparison
Colorado will be the reference, since all previous answers were evaluated on its area. Its precise total area is 104,093.67 square miles (from the US Census Bureau, p. 89, see also here), or 269601367661 square meters. I found no source for the actual methodology of the USCB, but I assume it is based on summing actual measurements on ground, or precise computations using WGS84/EGM2008.
Method | Author | Result | Variation from ground truth
--------------------------------------------------------------------------------
Albers Equal Area | sgillies | 268952044107 | -0.24%
Sinusoidal | J. Kington | 268885360163 | -0.26%
Girard's theorem | sulkeh | 268930758560 | -0.25%
Equal Area Cylindrical | Jason | 268993609651 | -0.22%
Line integral | Yellows | 269397764066 | **-0.07%**
Conclusion: using direct integral is more precise.
Performance
I have not benchmarked the different methods, and comparing pure Python code with compiled PROJ projections would not be meaningful. Intuitively less computations are needed. On the other hand, trigonometric functions may be computationally intensive.

Here is a solution that uses basemap, instead of pyproj and shapely, for the coordinate conversion. The idea is the same as suggested by #sgillies though. NOTE that I've added the 5th point so that the path is a closed loop.
import numpy
from mpl_toolkits.basemap import Basemap
coordinates=numpy.array([
[-102.05, 41.0],
[-102.05, 37.0],
[-109.05, 37.0],
[-109.05, 41.0],
[-102.05, 41.0]])
lats=coordinates[:,1]
lons=coordinates[:,0]
lat1=numpy.min(lats)
lat2=numpy.max(lats)
lon1=numpy.min(lons)
lon2=numpy.max(lons)
bmap=Basemap(projection='cea',llcrnrlat=lat1,llcrnrlon=lon1,urcrnrlat=lat2,urcrnrlon=lon2)
xs,ys=bmap(lons,lats)
area=numpy.abs(0.5*numpy.sum(ys[:-1]*numpy.diff(xs)-xs[:-1]*numpy.diff(ys)))
area=area/1e6
print area
The result is 268993.609651 in km^2.
UPDATE: Basemap has been deprecated, so you may want to consider alternative solutions first.

Because the earth is a closed surface a closed polygon drawn on its surface creates TWO polygonal areas. You also need to define which one is inside and which is outside!
Most times people will be dealing with small polygons, and so it's 'obvious' but once you have things the size of oceans or continents, you better make sure you get this the right way round.
Also, remember that lines can go from (-179,0) to (+179,0) in two different ways. One is very much longer than the other. Again, mostly you'll make the assumption that this is a line that goes from (-179,0) to (-180,0) which is (+180,0) and then to (+179,0), but one day... it won't.
Treating lat-long like a simple (x,y) coordinate system, or even neglecting the fact that any coordinate projection is going to have distortions and breaks, can make you fail big-time on spheres.

I know that answering 10 years later has some advantages, but to somebody that looks today at this question it seems fair to provide an updated answer.
pyproj directly calculates areas, without need of calling shapely:
# Modules:
from pyproj import Geod
import numpy as np
# Define WGS84 as CRS:
geod = Geod('+a=6378137 +f=0.0033528106647475126')
# Data for Colorado (no need to close the polygon):
coordinates = np.array([
[-102.05, 41.0],
[-102.05, 37.0],
[-109.05, 37.0],
[-109.05, 41.0]])
lats = coordinates[:,1]
lons = coordinates[:,0]
# Compute:
area, perim = geod.polygon_area_perimeter(lons, lats)
print(abs(area)) # Positive is counterclockwise, the data is clockwise.
The result is: 269154.54988400977 km2, or -0.17% of the reported correct value (269601.367661 km2).

According to Yellows' assertion, direct integral is more precise.
But Yellows use an earth radius = 6378 137m, which is the WGS-84 ellipsoid, semi-major axis, while Sulkeh use 6371 000 m.
Using a radius = 6378 137 m in the Sulkeh' method, gives 269533625893 square meters.
Assuming that the true value of Colorado area (from the US Census Bureau) is 269601367661 square meters then the variation from the ground truth of Sulkeh' method is : -0,025%, better than -0.07 with the Line integral method.
So Sulkeh' proposal seems to be the more precise so far.
In order to be able to make a numerical comparison of the solutions, with the assumption of a spherical Earth, all calculations must use the same terrestrial radius.

Here is a Python 3 implementation where the function would take a list of tuple-pairs of lats and longs and would return the area enclosed in the projected polygon.It uses pyproj to project the coordinates and then Shapely to find the area of any projected polygon
def calc_area(lis_lats_lons):
import numpy as np
from pyproj import Proj
from shapely.geometry import shape
lons, lats = zip(*lis_lats_lons)
ll = list(set(lats))[::-1]
var = []
for i in range(len(ll)):
var.append('lat_' + str(i+1))
st = ""
for v, l in zip(var,ll):
st = st + str(v) + "=" + str(l) +" "+ "+"
st = st +"lat_0="+ str(np.mean(ll)) + " "+ "+" + "lon_0" +"=" + str(np.mean(lons))
tx = "+proj=aea +" + st
pa = Proj(tx)
x, y = pa(lons, lats)
cop = {"type": "Polygon", "coordinates": [zip(x, y)]}
return shape(cop).area
For a sample set of lats/longs, it gives an area value close to the surveyed approximation value
calc_area(lis_lats_lons = [(-102.05, 41.0),
(-102.05, 37.0),
(-109.05, 37.0),
(-109.05, 41.0)])
Which outputs an area of 268952044107.4342 Sq. Mts.