We Keep Coding

How do I program a spatially dependend random number distribution?

I wrote a routine that distributes circles randomly (uniformly) with an arbitrary diameter in my study area.
def no_nearby_dots(new_dot, dots_sim, min_distance):
for dot in dots_sim:
if np.sqrt((dot[0] - new_dot[0]) ** 2 + (dot[1] - new_dot[1]) ** 2) <= min_distance:
return False
return True
while realizations < simulations:
dots_sim = []
new_dot = True
dots_sim.append((np.random.uniform(xmin, xmax), np.random.uniform(ymin, ymax)))
failed_attempts = 0
while new_dot:
xp = np.random.uniform(xmin, xmax)
yp = np.random.uniform(ymin, ymax)
if no_nearby_dots((xp, yp), dots_sim, diameter):
dots_sim.append((xp, yp))
failed_attempts = 0
else:
failed_attempts += 1
if len(dots_sim) == n_sim:
new_dot = False
if failed_attempts > 2000:
new_dot = False
print('ERROR...exit loop')
break
x_sim = [dot[0] for dot in dots_sim]
y_sim = [dot[1] for dot in dots_sim]
I want to introduce a second circle around the initial ones where the possibility of distributing points reduces exponentially towards the inner border -> I want to prevent a "hard" border, the points are allowed to occur anywhere on the plane but not closer than diameter, additionally they can only occur to a certain degree between diameter and diameter2.
Are there any ideas how to do that?

Here is an idea.
Choose a random radius between diameter/2 and diameter2/2, then generate a random point in the circle formed by that radius. There are many ways to choose a radius that meets your requirements. For example, the following chooses a radius such that radii very close to diameter2/2 are much more likely to be chosen:
radius = (diameter1/2) + ((diameter2/2) - (diameter1/2)) * random.random()**(1/20)
Note that 1/20 is the 20th root of a uniform (0, 1) random number. Try changing 1/20 to a different value and see what happens.
There are other ways to choose a radius this way, and they can all be described by a probability density function (for more information, see the following answer: Generate a random point within a circle (uniformly), which shows how a linear density function leads to a uniform distribution of points in a circle).

I solved it, and this is what I did:
while realizations < simulations:
dots_sim = []
new_dot = True
dots_sim.append((np.random.uniform(x_min, x_max), np.random.uniform(y_min, y_max)))
failed_attempts = 0
while new_dot:
x = np.random.uniform(x_min, x_max)
y = np.random.uniform(y_min, y_max)
diameter_simulation = np.random.uniform(min_diameter, max_diameter)
if no_nearby_dots((x, y), dots_sim, diameter_simulation):
dots_sim.append((x, y))
failed_attempts = 0
else:
failed_attempts += 1
if len(dots_sim) == len(x_coordinate):
new_dot = False
if failed_attempts > 1000:
new_dot = False
print('ERROR... -> no more space to place QDs! -> exit loop!')
break
What I did was creating diameters for my circles also using uniformly distributed numbers in an arbitrary interval, which smoothes my cumulative distribution function. This is the solution I needed, but it might not fit the initial question very well (or the question was formulated inaccurately in the first place :p)

Shortest path through ordered circular waypoints

I am trying to implement an algorithm which computes the shortest path and and its associated distance from a current position to a goal through an ordered list of waypoints in a 2d plane. A waypoint is defined by its center coordinates (x, y) and its radius r. The shortest path have to intersect each waypoint circumference at least once. This is different from other path optimization problems because I already know the order in which the waypoints have to be crossed.
In the simple case, consecutive waypoints are distinct and not aligned and this can be solved using consecutive angle bisections. The tricky cases are :
when three or more consecutive waypoints have the same center but different radii
when consecutive waypoints are aligned such that a straight line passes through all of them
Here is a stripped down version of my Python implementation, which does not handle aligned waypoints, and handles badly concentric consecutive waypoints. I adapted it because it normally uses latitudes and longitudes, not points in the euclidean space.
def optimize(position, waypoints):
# current position is on the shortest path, cumulative distance starts at zero
shortest_path = [position.center]
optimized_distance = 0
# if only one waypoint left, go in a straight line
if len(waypoints) == 1:
shortest_path.append(waypoints[-1].center)
optimized_distance += distance(position.center, waypoints[-1].center)
else:
# consider the last optimized point (one) and the next two waypoints (two, three)
for two, three in zip(waypoints[:], waypoints[1:]):
one = fast_waypoints[-1]
in_heading = get_heading(two.center, one.center)
in_distance = distance(one.center, two.center)
out_distance = distance(two.center, three.center)
# two next waypoints are concentric
if out_distance == 0:
next_target, nb_concentric = find_next_not_concentric(two, waypoints)
out_heading = get_heading(two.center, next_target.center)
angle = out_heading - in_heading
leg_distance = two.radius
leg_heading = in_heading + (0.5/nb_concentric) * angle
else:
out_heading = get_heading(two.center, three.center)
angle = out_heading - in_heading
leg_heading = in_heading + 0.5 * angle
leg_distance = (2 * in_distance * out_distance * math.cos(math.radians(angle * 0.5))) / (in_distance + out_distance)
best_leg_distance = min(leg_distance, two.radius)
next_best = get_offset(two.center, leg_heading, min_leg_distance)
shortest_path.append(next_best.center)
optimized_distance += distance(one.center, next_best.center)
return optimized_distance, shortest_path
I can see how to test for the different corner cases but I think this approach is bad, because there may be other corner cases I haven't thought of. Another approach would be to discretize the waypoints circumferences and apply a shortest path algorithm such as A*, but that would be highly inefficient.
So here is my question : Is there a more concise approach to this problem ?

For the record, I implemented a solution using Quasi-Newton methods, and described it in this short article. The main work is summarized below.
import numpy as np
from scipy.optimize import minimize
# objective function definition
def tasklen(θ, x, y, r):
x_proj = x + r*np.sin(θ)
y_proj = y + r*np.cos(θ)
dists = np.sqrt(np.power(np.diff(x_proj), 2) + np.power(np.diff(y_proj), 2))
return dists.sum()
# center coordinates and radii of turnpoints
X = np.array([0, 5, 0, 7, 12, 12]).astype(float)
Y = np.array([0, 0, 4, 7, 0, 5]).astype(float)
R = np.array([0, 2, 1, 2, 1, 0]).astype(float)
# first initialization vector is an array of zeros
init_vector = np.zeros(R.shape).astype(float)
# using scipy's solvers to minimize the objective function
result = minimize(tasklen, init_vector, args=(X, Y, R), tol=10e-5)

I would do it like this:
For each circle in order, pick any point on the circumference, and route the path through these points.
For each circle, move the point along the circumference in the direction that makes the total path length smaller.
Repeat 2. until no further improvement can be done.

Creating polygons/polylines from origin points using Python in ArcGIS?

I am still very new to Python. I am heading a project to map the building footprints within our county on the tax map.
I have found a previous question that may be very helpful for this project: https://gis.stackexchange.com/questions/6724/creating-line-of-varying-distance-from-origin-point-using-python-in-arcgis-deskt
Our Cama system generates views/table with the needed information. Below is an example:
PARID LLINE VECT X_COORD Y_COORD
1016649 0 R59D26L39U9L20U17 482547 1710874
180,59,270,26,0,39,90,9,0,20,90,17 (VECT column converted)
I have found some python examples to convert the VECT column, which are distance and direction calls to angles and distances separated by commas.
My question: Is there a way to implement a loop into the script below to utilize a table rather than static, user entered, numbers? This would be very valuable to the county as we have several thousand polygons to construct.
Below is the snippet to change the distances and angles to x, y points to be generated in ArcMap 10.2
#Using trig to deflect from a starting point
import arcpy
from math import radians, sin, cos
origin_x, origin_y = (400460.99, 135836.7)
distance = 800
angle = 15 # in degrees
# calculate offsets with light trig
(disp_x, disp_y) = (distance * sin(radians(angle)),\
distance * cos(radians(angle)))
(end_x, end_y) = (origin_x + disp_x, origin_y + disp_y)
output = "offset-line.shp"
arcpy.CreateFeatureClass_management("c:\workspace", output, "Polyline")
cur = arcpy.InsertCursor(output)
lineArray = arcpy.Array()
# start point
start = arcpy.Point()
(start.ID, start.X, start.Y) = (1, origin_x, origin_y)
lineArray.add(start)
# end point
end = arcpy.Point()
(end.ID, end.X, end.Y) = (2, end_x, end_y)
lineArray.add(end)
# write our fancy feature to the shapefile
feat = cur.newRow()
feat.shape = lineArray
cur.insertRow(feat)
# yes, this shouldn't really be necessary...
lineArray.removeAll()
del cur
Any suggestions would be greatly appreciated.
Thank you for your valuable time and knowledge.

You can create a dictionary of dictionaries from given table that would hold all the different values. Such as
d = {1:{"x":400460.99,"y":135836.7,"distance":800,"angle":15},
2:{"x":"etc","y":"etc","distance":"etc","angle":"etc"}}
for k in d.keys():
origin_x, d[k]["x"]
origin_y = d[k]["y"]
distance = d[k]["distance"]
angle = d[k]["angle"]
#rest of the code
#.....

How to solve...ValueError: cannot convert float NaN to integer

I'm running quite a complex code so I won't bother with details as I've had it working before but now im getting this error.
Particle is a 3D tuple filled with 0 or 255, and I am using the scipy centre of mass function and then trying to turn the value into its closest integer (as I'm dealing with arrays). The error is found with on the last line... can anyone explain why this might be??
2nd line fills Particle
3rd line deletes any surrounding particles with a different label (This is in a for loop for all labels)
Particle = []
Particle = big_labelled_stack[x_start+20:x_stop+20,y_start+20:y_stop+20,z_start+20:z_stop+20]
Particle = np.where(Particle == i ,255,0)
CoM = scipy.ndimage.measurements.center_of_mass(Particle)
CoM = [ (int(round(x)) for x in CoM ]
Thanks in advance. If you need more code just ask but I dont think it will help you and its very messy.
################## MORE CODE
border = 30
[labelled_stack,no_of_label] = label(labelled,structure_array,output_type)
# RE-LABEL particles now no. of seeds has been reduced! LAST LABELLING
#Increase size of stack by increasing borders and equal them to 0; to allow us to cut out particles into cube shape which else might lye outside the border
h,w,l = labelled.shape
big_labelled_stack = np.zeros(shape=(h+60,w+60,l+60),dtype=np.uint32)
# Creates an empty border around labelled_stack full of zeros of size border
if (no_of_label > 0): #Small sample may return no particles.. so this stage not neccesary
info = np.zeros(shape=(no_of_label,19)) #Creates array to store coordinates of particles
for i in np.arange(1,no_of_label,1):
coordinates = find_objects(labelled_stack == i)[0] #Find coordinates of label i.
x_start = int(coordinates[0].start)
x_stop = int(coordinates[0].stop)
y_start = int(coordinates[1].start)
y_stop = int(coordinates[1].stop)
z_start = int(coordinates[2].start)
z_stop = int(coordinates[2].stop)
dx = (x_stop - x_start)
dy = (y_stop - y_start)
dz = (z_stop - z_start)
Particle = np.zeros(shape=(dy,dx,dz),dtype = np.uint16)
Particle = big_labelled_stack[x_start+30:x_start+dx+30,y_start+30:y_start+dy+30,z_start+30:z_start+dz+30]
Particle = np.where(Particle == i ,255,0)
big_labelled_stack[border:h+border,border:w+border,border:l+border] = labelled_stack
big_labelled_stack = np.where(big_labelled_stack == i , 255,0)
CoM_big_stack = scipy.ndimage.measurements.center_of_mass(big_labelled_stack)
C = np.asarray(CoM_big_stack) - border
if dx > dy:
b = dx
else: #Finds the largest of delta_x,y,z and saves as b, so that we create 'Cubic_Particle' of size 2bx2bx2b (cubic box)
b = dy
if dz > b:
b = dz
CoM = scipy.ndimage.measurements.center_of_mass(Particle)
CoM = [ (int(round(x))) for x in CoM ]
Cubic_Particle = np.zeros(shape=(2*b,2*b,2*b))
Cubic_Particle[(b-CoM[0]):(b+dx-CoM[0]),(b-CoM[1]):(b+dy-CoM[1]),(b-CoM[2]):(b+dz-CoM[2])] = Particle
volume = Cubic_Particle.size # Gives volume of the box in voxels
info[i-1,:] = [C[0],C[1],C[2],i,C[0]-b,C[1]-b,C[2]-b,C[0]+b,C[1]+b,C[2]+b,volume,0,0,0,0,0,0,0,0] # Fills an array with label.No., size of box, and co-ords
else:
print('No particles found, try increasing the sample size')
info = []
Ok, so I have a stack full of labelled particles, there are two things I am trying to do, first find the centre of masses of each particle with respect ot the labelled_stack which is what CoM_big_labelled_stack (and C) does. and stores the co-ords in a list (tuple) called info. I am also trying to create a cubic box around the particle, with its centre of mass as the centre (which is relating to the CoM variable), so first I use the find objects function in scipy to find a particle, i then use these coordinates to create a non-cubic box around the particle, and find its centre of mass.I then find the longest dimension of the box and call it b, creating a cubic box of size 2b and filling it with particle in the right position.
Sorry this code is a mess, I am very new to Python

Find the area between two curves plotted in matplotlib (fill_between area)

I have a list of x and y values for two curves, both having weird shapes, and I don't have a function for any of them. I need to do two things:
Plot it and shade the area between the curves like the image below.
Find the total area of this shaded region between the curves.
I'm able to plot and shade the area between those curves with fill_between and fill_betweenx in matplotlib, but I have no idea on how to calculate the exact area between them, specially because I don't have a function for any of those curves.
Any ideas?
I looked everywhere and can't find a simple solution for this. I'm quite desperate, so any help is much appreciated.
Thank you very much!
EDIT: For future reference (in case anyone runs into the same problem), here is how I've solved this: connected the first and last node/point of each curve together, resulting in a big weird-shaped polygon, then used shapely to calculate the polygon's area automatically, which is the exact area between the curves, no matter which way they go or how nonlinear they are. Works like a charm! :)
Here is my code:
from shapely.geometry import Polygon
x_y_curve1 = [(0.121,0.232),(2.898,4.554),(7.865,9.987)] #these are your points for curve 1 (I just put some random numbers)
x_y_curve2 = [(1.221,1.232),(3.898,5.554),(8.865,7.987)] #these are your points for curve 2 (I just put some random numbers)
polygon_points = [] #creates a empty list where we will append the points to create the polygon
for xyvalue in x_y_curve1:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 1
for xyvalue in x_y_curve2[::-1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 2 in the reverse order (from last point to first point)
for xyvalue in x_y_curve1[0:1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append the first point in curve 1 again, to it "closes" the polygon
polygon = Polygon(polygon_points)
area = polygon.area
print(area)
EDIT 2: Thank you for the answers. Like Kyle explained, this only works for positive values. If your curves go below 0 (which is not my case, as showed in the example chart), then you would have to work with absolute numbers.

The area calculation is straightforward in blocks where the two curves don't intersect: thats the trapezium as has been pointed out above. If they intersect, then you create two triangles between x[i] and x[i+1], and you should add the area of the two. If you want to do it directly, you should handle the two cases separately. Here's a basic working example to solve your problem. First, I will start with some fake data:
#!/usr/bin/python
import numpy as np
# let us generate fake test data
x = np.arange(10)
y1 = np.random.rand(10) * 20
y2 = np.random.rand(10) * 20
Now, the main code. Based on your plot, looks like you have y1 and y2 defined at the same X points. Then we define,
z = y1-y2
dx = x[1:] - x[:-1]
cross_test = np.sign(z[:-1] * z[1:])
cross_test will be negative whenever the two graphs cross. At these points, we want to calculate the x coordinate of the crossover. For simplicity, I will calculate x coordinates of the intersection of all segments of y. For places where the two curves don't intersect, they will be useless values, and we won't use them anywhere. This just keeps the code easier to understand.
Suppose you have z1 and z2 at x1 and x2, then we are solving for x0 such that z = 0:
# (z2 - z1)/(x2 - x1) = (z0 - z1) / (x0 - x1) = -z1/(x0 - x1)
# x0 = x1 - (x2 - x1) / (z2 - z1) * z1
x_intersect = x[:-1] - dx / (z[1:] - z[:-1]) * z[:-1]
dx_intersect = - dx / (z[1:] - z[:-1]) * z[:-1]
Where the curves don't intersect, area is simply given by:
areas_pos = abs(z[:-1] + z[1:]) * 0.5 * dx # signs of both z are same
Where they intersect, we add areas of both triangles:
areas_neg = 0.5 * dx_intersect * abs(z[:-1]) + 0.5 * (dx - dx_intersect) * abs(z[1:])
Now, the area in each block x[i] to x[i+1] is to be selected, for which I use np.where:
areas = np.where(cross_test < 0, areas_neg, areas_pos)
total_area = np.sum(areas)
That is your desired answer. As has been pointed out above, this will get more complicated if the both the y graphs were defined at different x points. If you want to test this, you can simply plot it (in my test case, y range will be -20 to 20)
negatives = np.where(cross_test < 0)
positives = np.where(cross_test >= 0)
plot(x, y1)
plot(x, y2)
plot(x, z)
plt.vlines(x_intersect[negatives], -20, 20)

Define your two curves as functions f and g that are linear by segment, e.g. between x1 and x2, f(x) = f(x1) + ((x-x1)/(x2-x1))*(f(x2)-f(x1)).
Define h(x)=abs(g(x)-f(x)). Then use scipy.integrate.quad to integrate h.
That way you don't need to bother about the intersections. It will do the "trapeze summing" suggested by ch41rmn automatically.

Your set of data is quite "nice" in the sense that the two sets of data share the same set of x-coordinates. You can therefore calculate the area using a series of trapezoids.
e.g. define the two functions as f(x) and g(x), then, between any two consecutive points in x, you have four points of data:
(x1, f(x1))-->(x2, f(x2))
(x1, g(x1))-->(x2, g(x2))
Then, the area of the trapezoid is
A(x1-->x2) = ( f(x1)-g(x1) + f(x2)-g(x2) ) * (x2-x1)/2 (1)
A complication arises that equation (1) only works for simply-connected regions, i.e. there must not be a cross-over within this region:
|\ |\/|
|_| vs |/\|
The area of the two sides of the intersection must be evaluated separately. You will need to go through your data to find all points of intersections, then insert their coordinates into your list of coordinates. The correct order of x must be maintained. Then, you can loop through your list of simply connected regions and obtain a sum of the area of trapezoids.
EDIT:
For curiosity's sake, if the x-coordinates for the two lists are different, you can instead construct triangles. e.g.
.____.
| / \
| / \
| / \
|/ \
._________.
Overlap between triangles must be avoided, so you will again need to find points of intersections and insert them into your ordered list. The lengths of each side of the triangle can be calculated using Pythagoras' formula, and the area of the triangles can be calculated using Heron's formula.

The area_between_two_curves function in pypi library similaritymeasures (released in 2018) might give you what you need. I tried a trivial example on my side, comparing the area between a function and a constant value and got pretty close tie-back to Excel (within 2%). Not sure why it doesn't give me 100% tie-back, maybe I am doing something wrong. Worth considering though.

I had the same problem.The answer below is based on an attempt by the question author. However, shapely will not directly give the area of the polygon in purple. You need to edit the code to break it up into its component polygons and then get the area of each. After-which you simply add them up.
Area Between two lines
Consider the lines below:
Sample Two lines
If you run the code below you will get zero for area because it takes the clockwise and subtracts the anti clockwise area:
from shapely.geometry import Polygon
x_y_curve1 = [(1,1),(2,1),(3,3),(4,3)] #these are your points for curve 1
x_y_curve2 = [(1,3),(2,3),(3,1),(4,1)] #these are your points for curve 2
polygon_points = [] #creates a empty list where we will append the points to create the polygon
for xyvalue in x_y_curve1:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 1
for xyvalue in x_y_curve2[::-1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 2 in the reverse order (from last point to first point)
for xyvalue in x_y_curve1[0:1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append the first point in curve 1 again, to it "closes" the polygon
polygon = Polygon(polygon_points)
area = polygon.area
print(area)
The solution is therefore to split the polygon into smaller pieces based on where the lines intersect. Then use a for loop to add these up:
from shapely.geometry import Polygon
x_y_curve1 = [(1,1),(2,1),(3,3),(4,3)] #these are your points for curve 1
x_y_curve2 = [(1,3),(2,3),(3,1),(4,1)] #these are your points for curve 2
polygon_points = [] #creates a empty list where we will append the points to create the polygon
for xyvalue in x_y_curve1:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 1
for xyvalue in x_y_curve2[::-1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 2 in the reverse order (from last point to first point)
for xyvalue in x_y_curve1[0:1]:
polygon_points.append([xyvalue[0],xyvalue[1]]) #append the first point in curve 1 again, to it "closes" the polygon
polygon = Polygon(polygon_points)
area = polygon.area
x,y = polygon.exterior.xy
# original data
ls = LineString(np.c_[x, y])
# closed, non-simple
lr = LineString(ls.coords[:] + ls.coords[0:1])
lr.is_simple # False
mls = unary_union(lr)
mls.geom_type # MultiLineString'
Area_cal =[]
for polygon in polygonize(mls):
Area_cal.append(polygon.area)
Area_poly = (np.asarray(Area_cal).sum())
print(Area_poly)

A straightforward application of the area of a general polygon (see Shoelace formula) makes for a super-simple and fast, vectorized calculation:
def area(p):
# for p: 2D vertices of a polygon:
# area = 1/2 abs(sum(p0 ^ p1 + p1 ^ p2 + ... + pn-1 ^ p0))
# where ^ is the cross product
return np.abs(np.cross(p, np.roll(p, 1, axis=0)).sum()) / 2
Application to area between two curves. In this example, we don't even have matching x coordinates!
np.random.seed(0)
n0 = 10
n1 = 15
xy0 = np.c_[np.linspace(0, 10, n0), np.random.uniform(0, 10, n0)]
xy1 = np.c_[np.linspace(0, 10, n1), np.random.uniform(0, 10, n1)]
p = np.r_[xy0, xy1[::-1]]
>>> area(p)
4.9786...
Plot:
plt.plot(*xy0.T, 'b-')
plt.plot(*xy1.T, 'r-')
p = np.r_[xy0, xy1[::-1]]
plt.fill(*p.T, alpha=.2)
Speed
For both curves having 1 million points:
n = 1_000_000
xy0 = np.c_[np.linspace(0, 10, n), np.random.uniform(0, 10, n)]
xy1 = np.c_[np.linspace(0, 10, n), np.random.uniform(0, 10, n)]
%timeit area(np.r_[xy0, xy1[::-1]])
# 42.9 ms ± 140 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
Simple viz of polygon area calculation
# say:
p = np.array([[0, 3], [1, 0], [3, 3], [1, 3], [1, 2]])
p_closed = np.r_[p, p[:1]]
fig, axes = plt.subplots(ncols=2, figsize=(10, 5), subplot_kw=dict(box_aspect=1), sharex=True)
ax = axes[0]
ax.set_aspect('equal')
ax.plot(*p_closed.T, '.-')
ax.fill(*p_closed.T, alpha=0.6)
center = p.mean(0)
txtkwargs = dict(ha='center', va='center')
ax.text(*center, f'{area(p):.2f}', **txtkwargs)
ax = axes[1]
ax.set_aspect('equal')
for a, b in zip(p_closed, p_closed[1:]):
ar = 1/2 * np.cross(a, b)
pos = ar >= 0
tri = np.c_[(0,0), a, b, (0,0)].T
# shrink a bit to make individual triangles easier to visually identify
center = tri.mean(0)
tri = (tri - center)*0.95 + center
c = 'b' if pos else 'r'
ax.plot(*tri.T, 'k')
ax.fill(*tri.T, c, alpha=0.2, zorder=2 - pos)
t = ax.text(*center, f'{ar:.1f}', color=c, fontsize=8, **txtkwargs)
t.set_bbox(dict(facecolor='white', alpha=0.8, edgecolor='none'))
plt.tight_layout()