Find intersection of numpy float arrays - python

How can I find the intersection of two numpy float arrays?:
a = np.arange(2, 3, 0.1)
b = np.array([2.3, 2.4, 2.5])
out_data = np.intersect1d(a, b)
the result is
out_data -> ndarray: []

Because of the way floats work, in your example a[3] is not 2.3, but 2.3000000000000003. This is because 0.1 does not have an exact representation in IEEE double precision floats. The intersect1d method in numpy is really only well suited for integers. To solve this, you should implement your own method that takes in a tolerance to decide if two floats are sufficiently close.

Here's a vectorized approach using NumPy's broadcasting capability -
tol = 1e-5 # tolerance
out = b[(np.abs(a[:,None] - b) < tol).any(0)]
Sample run -
In [31]: a
Out[31]: array([ 2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9])
In [32]: b
Out[32]: array([ 2.3 , 2.4 , 2.5 , 2.25, 2.1 ])
In [33]: tol = 1e-5 # tolerance
In [34]: b[(np.abs(a[:,None] - b) < tol).any(0)]
Out[34]: array([ 2.3, 2.4, 2.5, 2.1])

Putting my comments in function form (assuming both lists are sorted, which you should do ahead of time):
import numpy as np
from itertools import islice
def findOverlap(self, a, b, rtol = 1e-05, atol = 1e-08):
ovr_a = []
ovr_b = []
start_b = 0
for i, ai in enumerate(a):
for j, bj in islice(enumerate(b), start_b, None):
if np.isclose(ai, bj, rtol=rtol, atol=atol, equal_nan=False):
ovr_a.append(i)
ovr_b.append(j)
elif bj > ai: # (more than tolerance)
break # all the rest will be farther away
else: # bj < ai (more than tolerance)
start_b += 1 # ignore further tests of this item
return (ovr_a, ovr_b)
EDIT: getting rid of equal_nan -- if you're going to sort you may as well ditch the nans
EDIT: using islice instead of array slice
EDIT: fixed bug

The following routine will return the indexes of common values within specified tolerance(s) relative to list a.
def findOverlap(self, a, b, rtol = 1e-05, atol = 1e-08, equal_nan = False):
overlap_indexes = []
for i, item_a in enumerate(a):
for item_b in b:
if np.isclose(item_a, item_b, rtol = rtol, atol = atol, equal_nan = equal_nan):
overlap_indexes.append(i)
return overlap_indexes
eg
a = np.arange(2, 3, 0.1).tolist()
b = np.array([2.3, 2.4, 2.5]).tolist()
self.findOverlap(a, b)
-> overlap_indexes:[3, 4, 5]

Related

Insert calculated values between consecutive values in array

Let's say I have a simple array, like this one:
import numpy as np
a = np.array([1,2,3])
Which returns me, obviously:
array([1, 2, 3])
I'm trying to add calculated values between consecutive values in this array. The calculation should return me n equally spaced values between it's bounds.
To express myself in numbers, let's say I want to add 1 value between each pair of consecutive values, so the function should return me a array like this:
array([1, 1.5, 2, 2.5, 3])
Another example, now with 2 values between each pair:
array([1, 1.33, 1.66, 2, 2.33, 2.66, 3])
I know the logic and I can create myself a function which will do the work, but I feel numpy has specific functions that would make my code so much cleaner!
If your array is
import numpy as np
n = 2
a = np.array([1,2,5])
new_size = a.size + (a.size - 1) * n
x = np.linspace(a.min(), a.max(), new_size)
xp = np.linspace(a.min(), a.max(), a.size)
fp = a
result = np.interp(x, xp, fp)
returns: array([1. , 1.33333333, 1.66666667, 2. , 2.66666667, 3.33333333, 4. ])
If your array is always evenly spaced, you can just use
new_size = a.size + (a.size - 1) * n
result = np.linspace(a.min(), a.max(), new_size)
Using linspace should do the trick:
a = np.array([1,2,3])
n = 1
temps = []
for i in range(1, len(a)):
temps.append(np.linspace(a[i-1], a[i], num=n+1, endpoint=False))
# Add last final ending point
temps.append(np.array([a[-1]]))
new_a = np.concatenate(temps)
print(new_a)
Try with np.arange:
a = np.array([1,2,3])
n = 2
print(np.arange(a.min(), a.max(), 1 / (n + 1)))
Output:
[1. 1.33333333 1.66666667 2. 2.33333333 2.66666667]

How to find lowest 3 values in a matrix

So i have this fucntion to find the lowest value in a matrix, and return its position in the matrix, ie its indices:
final_matrix=[[3.57 2.71 9.2 5.63]
[4.42 1.4 3.53 8.97]
[1.2 0.33 6.26 7.77]
[6.36 3.6 8.91 7.42]
[1.59 0.9 2.4 4.24]] # this changes in my code, im just giving a very simple version of it here
def lowest_values(final_matrix):
best_value=10000 #or any arbitrarily high number
for i in range(0,len(final_matrix[:,0])):
for j in range(0,len(final_matrix[0,:])):
if final_matrix[i,j]<best_value:
best_value=final_matrix[i,j]
lowest_val_i=i
lowest_val_j=j
return(lowest_val_i, lowest_val_j)
this returns me (1,2), which just by visual analysis is correct. i want to now find the lowest 3 values - hopefully, to build into this loop. But i really cannot think how! Or at least i dont know how to implement it. I was thinking of some if-else loop, that if the lowest value is already found, then 'void' this one and found 2nd lowest, and then same thing to find the third. But im not sure.
Please dont be too quick to shut this question down - im very new to programming, and very stuck!
The human Approach
I think my approach to this is different enough from different answers to share.
I am only doing 3 comparisons for every list element so it should be O(n). Also I'm not creating a entirely new list with (value, indices) tuple of all the elements.
matrix=[[3.57, 2.71, 9.2, 5.63],
[4.42, 1.4, 3.53, 8.97],
[1.2, 0.33, 6.26, 7.77],
[6.36, 3.6, 8.91, 7.42],
[1.59, 0.9, 2.4, 4.24]]
def compare_least_values(value, i, j):
global least
if value < least[2][0] :
if value < least[1][0] :
if value < least[0][0] :
least.insert(0, (value,(i,j)))
else:
least.insert(1, (value,(i,j)))
else:
least.insert(2, (value,(i,j)))
def lowest_three_values(matrix):
global least
least = [(10000, (None, None)), (10000, (None, None)), (10000, (None, None))]
for i, row in enumerate(matrix):
for j, value in enumerate(row):
compare_least_values(value, i, j)
return least[:3]
print(lowest_three_values(matrix))
Output:
[(0.33, (2, 1)), (0.9, (4, 1)), (1.2, (2, 0))]
The practical approach (Numpy)
If you're familiar with numpy than this is the way to go. Even if you're not it can be use as a copy-paste snippet.
import numpy as np
matrix=[[3.57, 2.71, 9.2, 5.63],
[4.42, 1.4, 3.53, 8.97],
[1.2, 0.33, 6.26, 7.77],
[6.36, 3.6, 8.91, 7.42],
[1.59, 0.9, 2.4, 4.24]]
matrix = np.array(matrix)
indices_1d = np.argpartition(matrix, 3, axis=None)[:3]
indices_2d = np.unravel_index(indices_1d, matrix.shape)
least_three = matrix[indices_2d]
print('least three values : ', least_three)
print('indices : ', *zip(*indices_2d) )
Output:
least three values : [0.33 0.9 1.2 ]
indices : (2, 1) (4, 1) (2, 0)
See this Stackoverflow query for detailed answer on this.
I didn't understand that return thing (1,2). Lowest value of matrix is 0.33. and that's position is (2,1).
So my solution for your code;
all_items = []
# i appended matrix's all items to one list
for row in final_matrix:
for i in row:
all_items.append(i)
# and i sorted that list as from min to max
all_items.sort()
# then i took first 3 values
lowest_3 = all_items[0:3]
positions = []
# and i append their positions that's in the matrix into positions
for i in lowest_3:
for row in range(len(final_matrix)):
if(i in final_matrix[row]):
positions.append([row, final_matrix[row].index(i)])
break
# lowest_3 = [0.33, 0.9, 1.2]
# positions = [[2, 1], [4, 1], [2, 0]]
this is my solution:
final_matrix=[[3.57, 2.71, 9.2, 5.63],
[4.42, 1.4, 3.53, 8.97],
[1.2, 0.33, 6.26, 7.77],
[6.36, 3.6, 8.91, 7.42],
[1.59, 0.9, 2.4, 4.24]]
min_values = []
for i in range(3):
mini = final_matrix[0][0]
for row in final_matrix:
for n in row:
if n < mini:
mini = n
n_index = row.index(n)
row_index = final_matrix.index(row)
min_values.append(mini)
del final_matrix[row_index][n_index]
print("Finals {}".format(min_values))
Let me explains you:
The first loop through how many min values you want (change it, you will see what I mean)
the second and the last one, loop through the matrix to take the minimun value
The line del final_matrix[row_index][n_index] will destroy the minimal number IN the original matrix
So if you want the keep the original matrix you have to create a new one and copy the original in => use deepcopy() from the copy module
You are looking for the N smallest items from a list of items (or a "matrix" in this case) and when N is small you can do better than sorting the items of your list by creating a heap queue, which takes linear time and then by popping the N smallest elements which is an O(log N) operation for each pop. The heap queue is an important data structure, which you should study.
import heapq
final_matrix=[[3.57, 2.71, 9.2, 5.63],
[4.42, 1.4, 3.53, 8.97],
[1.2, 0.33, 6.26, 7.77],
[6.36, 3.6, 8.91, 7.42],
[1.59, 0.9, 2.4, 4.24]]
def lowest_values(final_matrix):
# create a flat list, l, from the matrix
# each element is a tuple: (value, x-coordinate, y-coordinate)
l = [(final_matrix[x][y], x, y)
for x in range(len(final_matrix))
for y in range(len(final_matrix[0]))
]
heapq.heapify(l) # O(N) time
for _ in range(3):
# pop next smallest tuple:
value, x, y = heapq.heappop(l) # O(log N) time
print(f'value={value}, x={x}, y={y}')
lowest_values(final_matrix)
Prints:
value=0.33, x=2, y=1
value=0.9, x=4, y=1
value=1.2, x=2, y=0
Note
The above code could have been simplified to the following, which is probably even slightly more efficient if all you want are the 3 smallest items and then you have no further need for the heap queue structure. But I wanted to show the two basic operations of creating a heap queue from a list and then successively producing the smallest items from that heap queue:
import heapq
final_matrix=[[3.57, 2.71, 9.2, 5.63],
[4.42, 1.4, 3.53, 8.97],
[1.2, 0.33, 6.26, 7.77],
[6.36, 3.6, 8.91, 7.42],
[1.59, 0.9, 2.4, 4.24]]
def lowest_values(final_matrix):
# create a flat list, l, from the matrix
# each element is a tuple: (value, x-coordinate, y-coordinate)
l = [(final_matrix[x][y], x, y)
for x in range(len(final_matrix))
for y in range(len(final_matrix[0]))
]
for value, x, y in heapq.nsmallest(3, l):
print(f'value={value}, x={x}, y={y}')
lowest_values(final_matrix)

list of floats in python [duplicate]

How do I iterate between 0 and 1 by a step of 0.1?
This says that the step argument cannot be zero:
for i in range(0, 1, 0.1):
print(i)
Rather than using a decimal step directly, it's much safer to express this in terms of how many points you want. Otherwise, floating-point rounding error is likely to give you a wrong result.
Use the linspace function from the NumPy library (which isn't part of the standard library but is relatively easy to obtain). linspace takes a number of points to return, and also lets you specify whether or not to include the right endpoint:
>>> np.linspace(0,1,11)
array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ])
>>> np.linspace(0,1,10,endpoint=False)
array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9])
If you really want to use a floating-point step value, use numpy.arange:
>>> import numpy as np
>>> np.arange(0.0, 1.0, 0.1)
array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9])
Floating-point rounding error will cause problems, though. Here's a simple case where rounding error causes arange to produce a length-4 array when it should only produce 3 numbers:
>>> numpy.arange(1, 1.3, 0.1)
array([1. , 1.1, 1.2, 1.3])
range() can only do integers, not floating point.
Use a list comprehension instead to obtain a list of steps:
[x * 0.1 for x in range(0, 10)]
More generally, a generator comprehension minimizes memory allocations:
xs = (x * 0.1 for x in range(0, 10))
for x in xs:
print(x)
Building on 'xrange([start], stop[, step])', you can define a generator that accepts and produces any type you choose (stick to types supporting + and <):
>>> def drange(start, stop, step):
... r = start
... while r < stop:
... yield r
... r += step
...
>>> i0=drange(0.0, 1.0, 0.1)
>>> ["%g" % x for x in i0]
['0', '0.1', '0.2', '0.3', '0.4', '0.5', '0.6', '0.7', '0.8', '0.9', '1']
>>>
Increase the magnitude of i for the loop and then reduce it when you need it.
for i * 100 in range(0, 100, 10):
print i / 100.0
EDIT: I honestly cannot remember why I thought that would work syntactically
for i in range(0, 11, 1):
print i / 10.0
That should have the desired output.
NumPy is a bit overkill, I think.
[p/10 for p in range(0, 10)]
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
Generally speaking, to do a step-by-1/x up to y you would do
x=100
y=2
[p/x for p in range(0, int(x*y))]
[0.0, 0.01, 0.02, 0.03, ..., 1.97, 1.98, 1.99]
(1/x produced less rounding noise when I tested).
scipy has a built in function arange which generalizes Python's range() constructor to satisfy your requirement of float handling.
from scipy import arange
Similar to R's seq function, this one returns a sequence in any order given the correct step value. The last value is equal to the stop value.
def seq(start, stop, step=1):
n = int(round((stop - start)/float(step)))
if n > 1:
return([start + step*i for i in range(n+1)])
elif n == 1:
return([start])
else:
return([])
Results
seq(1, 5, 0.5)
[1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]
seq(10, 0, -1)
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
seq(10, 0, -2)
[10, 8, 6, 4, 2, 0]
seq(1, 1)
[ 1 ]
The range() built-in function returns a sequence of integer values, I'm afraid, so you can't use it to do a decimal step.
I'd say just use a while loop:
i = 0.0
while i <= 1.0:
print i
i += 0.1
If you're curious, Python is converting your 0.1 to 0, which is why it's telling you the argument can't be zero.
Here's a solution using itertools:
import itertools
def seq(start, end, step):
if step == 0:
raise ValueError("step must not be 0")
sample_count = int(abs(end - start) / step)
return itertools.islice(itertools.count(start, step), sample_count)
Usage Example:
for i in seq(0, 1, 0.1):
print(i)
[x * 0.1 for x in range(0, 10)]
in Python 2.7x gives you the result of:
[0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9]
but if you use:
[ round(x * 0.1, 1) for x in range(0, 10)]
gives you the desired:
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
import numpy as np
for i in np.arange(0, 1, 0.1):
print i
Best Solution: no rounding error
>>> step = .1
>>> N = 10 # number of data points
>>> [ x / pow(step, -1) for x in range(0, N + 1) ]
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
Or, for a set range instead of set data points (e.g. continuous function), use:
>>> step = .1
>>> rnge = 1 # NOTE range = 1, i.e. span of data points
>>> N = int(rnge / step
>>> [ x / pow(step,-1) for x in range(0, N + 1) ]
[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
To implement a function: replace x / pow(step, -1) with f( x / pow(step, -1) ), and define f.
For example:
>>> import math
>>> def f(x):
return math.sin(x)
>>> step = .1
>>> rnge = 1 # NOTE range = 1, i.e. span of data points
>>> N = int(rnge / step)
>>> [ f( x / pow(step,-1) ) for x in range(0, N + 1) ]
[0.0, 0.09983341664682815, 0.19866933079506122, 0.29552020666133955, 0.3894183423086505,
0.479425538604203, 0.5646424733950354, 0.644217687237691, 0.7173560908995228,
0.7833269096274834, 0.8414709848078965]
And if you do this often, you might want to save the generated list r
r=map(lambda x: x/10.0,range(0,10))
for i in r:
print i
more_itertools is a third-party library that implements a numeric_range tool:
import more_itertools as mit
for x in mit.numeric_range(0, 1, 0.1):
print("{:.1f}".format(x))
Output
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
This tool also works for Decimal and Fraction.
My versions use the original range function to create multiplicative indices for the shift. This allows same syntax to the original range function.
I have made two versions, one using float, and one using Decimal, because I found that in some cases I wanted to avoid the roundoff drift introduced by the floating point arithmetic.
It is consistent with empty set results as in range/xrange.
Passing only a single numeric value to either function will return the standard range output to the integer ceiling value of the input parameter (so if you gave it 5.5, it would return range(6).)
Edit: the code below is now available as package on pypi: Franges
## frange.py
from math import ceil
# find best range function available to version (2.7.x / 3.x.x)
try:
_xrange = xrange
except NameError:
_xrange = range
def frange(start, stop = None, step = 1):
"""frange generates a set of floating point values over the
range [start, stop) with step size step
frange([start,] stop [, step ])"""
if stop is None:
for x in _xrange(int(ceil(start))):
yield x
else:
# create a generator expression for the index values
indices = (i for i in _xrange(0, int((stop-start)/step)))
# yield results
for i in indices:
yield start + step*i
## drange.py
import decimal
from math import ceil
# find best range function available to version (2.7.x / 3.x.x)
try:
_xrange = xrange
except NameError:
_xrange = range
def drange(start, stop = None, step = 1, precision = None):
"""drange generates a set of Decimal values over the
range [start, stop) with step size step
drange([start,] stop, [step [,precision]])"""
if stop is None:
for x in _xrange(int(ceil(start))):
yield x
else:
# find precision
if precision is not None:
decimal.getcontext().prec = precision
# convert values to decimals
start = decimal.Decimal(start)
stop = decimal.Decimal(stop)
step = decimal.Decimal(step)
# create a generator expression for the index values
indices = (
i for i in _xrange(
0,
((stop-start)/step).to_integral_value()
)
)
# yield results
for i in indices:
yield float(start + step*i)
## testranges.py
import frange
import drange
list(frange.frange(0, 2, 0.5)) # [0.0, 0.5, 1.0, 1.5]
list(drange.drange(0, 2, 0.5, precision = 6)) # [0.0, 0.5, 1.0, 1.5]
list(frange.frange(3)) # [0, 1, 2]
list(frange.frange(3.5)) # [0, 1, 2, 3]
list(frange.frange(0,10, -1)) # []
Lots of the solutions here still had floating point errors in Python 3.6 and didnt do exactly what I personally needed.
Function below takes integers or floats, doesnt require imports and doesnt return floating point errors.
def frange(x, y, step):
if int(x + y + step) == (x + y + step):
r = list(range(int(x), int(y), int(step)))
else:
f = 10 ** (len(str(step)) - str(step).find('.') - 1)
rf = list(range(int(x * f), int(y * f), int(step * f)))
r = [i / f for i in rf]
return r
Suprised no-one has yet mentioned the recommended solution in the Python 3 docs:
See also:
The linspace recipe shows how to implement a lazy version of range that suitable for floating point applications.
Once defined, the recipe is easy to use and does not require numpy or any other external libraries, but functions like numpy.linspace(). Note that rather than a step argument, the third num argument specifies the number of desired values, for example:
print(linspace(0, 10, 5))
# linspace(0, 10, 5)
print(list(linspace(0, 10, 5)))
# [0.0, 2.5, 5.0, 7.5, 10]
I quote a modified version of the full Python 3 recipe from Andrew Barnert below:
import collections.abc
import numbers
class linspace(collections.abc.Sequence):
"""linspace(start, stop, num) -> linspace object
Return a virtual sequence of num numbers from start to stop (inclusive).
If you need a half-open range, use linspace(start, stop, num+1)[:-1].
"""
def __init__(self, start, stop, num):
if not isinstance(num, numbers.Integral) or num <= 1:
raise ValueError('num must be an integer > 1')
self.start, self.stop, self.num = start, stop, num
self.step = (stop-start)/(num-1)
def __len__(self):
return self.num
def __getitem__(self, i):
if isinstance(i, slice):
return [self[x] for x in range(*i.indices(len(self)))]
if i < 0:
i = self.num + i
if i >= self.num:
raise IndexError('linspace object index out of range')
if i == self.num-1:
return self.stop
return self.start + i*self.step
def __repr__(self):
return '{}({}, {}, {})'.format(type(self).__name__,
self.start, self.stop, self.num)
def __eq__(self, other):
if not isinstance(other, linspace):
return False
return ((self.start, self.stop, self.num) ==
(other.start, other.stop, other.num))
def __ne__(self, other):
return not self==other
def __hash__(self):
return hash((type(self), self.start, self.stop, self.num))
This is my solution to get ranges with float steps.
Using this function it's not necessary to import numpy, nor install it.
I'm pretty sure that it could be improved and optimized. Feel free to do it and post it here.
from __future__ import division
from math import log
def xfrange(start, stop, step):
old_start = start #backup this value
digits = int(round(log(10000, 10)))+1 #get number of digits
magnitude = 10**digits
stop = int(magnitude * stop) #convert from
step = int(magnitude * step) #0.1 to 10 (e.g.)
if start == 0:
start = 10**(digits-1)
else:
start = 10**(digits)*start
data = [] #create array
#calc number of iterations
end_loop = int((stop-start)//step)
if old_start == 0:
end_loop += 1
acc = start
for i in xrange(0, end_loop):
data.append(acc/magnitude)
acc += step
return data
print xfrange(1, 2.1, 0.1)
print xfrange(0, 1.1, 0.1)
print xfrange(-1, 0.1, 0.1)
The output is:
[1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0]
[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1]
[-1.0, -0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, -0.2, -0.1, 0.0]
For completeness of boutique, a functional solution:
def frange(a,b,s):
return [] if s > 0 and a > b or s < 0 and a < b or s==0 else [a]+frange(a+s,b,s)
You can use this function:
def frange(start,end,step):
return map(lambda x: x*step, range(int(start*1./step),int(end*1./step)))
It can be done using Numpy library. arange() function allows steps in float. But, it returns a numpy array which can be converted to list using tolist() for our convenience.
for i in np.arange(0, 1, 0.1).tolist():
print i
start and stop are inclusive rather than one or the other (usually stop is excluded) and without imports, and using generators
def rangef(start, stop, step, fround=5):
"""
Yields sequence of numbers from start (inclusive) to stop (inclusive)
by step (increment) with rounding set to n digits.
:param start: start of sequence
:param stop: end of sequence
:param step: int or float increment (e.g. 1 or 0.001)
:param fround: float rounding, n decimal places
:return:
"""
try:
i = 0
while stop >= start and step > 0:
if i==0:
yield start
elif start >= stop:
yield stop
elif start < stop:
if start == 0:
yield 0
if start != 0:
yield start
i += 1
start += step
start = round(start, fround)
else:
pass
except TypeError as e:
yield "type-error({})".format(e)
else:
pass
# passing
print(list(rangef(-100.0,10.0,1)))
print(list(rangef(-100,0,0.5)))
print(list(rangef(-1,1,0.2)))
print(list(rangef(-1,1,0.1)))
print(list(rangef(-1,1,0.05)))
print(list(rangef(-1,1,0.02)))
print(list(rangef(-1,1,0.01)))
print(list(rangef(-1,1,0.005)))
# failing: type-error:
print(list(rangef("1","10","1")))
print(list(rangef(1,10,"1")))
Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64
bit (AMD64)]
I know I'm late to the party here, but here's a trivial generator solution that's working in 3.6:
def floatRange(*args):
start, step = 0, 1
if len(args) == 1:
stop = args[0]
elif len(args) == 2:
start, stop = args[0], args[1]
elif len(args) == 3:
start, stop, step = args[0], args[1], args[2]
else:
raise TypeError("floatRange accepts 1, 2, or 3 arguments. ({0} given)".format(len(args)))
for num in start, step, stop:
if not isinstance(num, (int, float)):
raise TypeError("floatRange only accepts float and integer arguments. ({0} : {1} given)".format(type(num), str(num)))
for x in range(int((stop-start)/step)):
yield start + (x * step)
return
then you can call it just like the original range()... there's no error handling, but let me know if there is an error that can be reasonably caught, and I'll update. or you can update it. this is StackOverflow.
To counter the float precision issues, you could use the Decimal module.
This demands an extra effort of converting to Decimal from int or float while writing the code, but you can instead pass str and modify the function if that sort of convenience is indeed necessary.
from decimal import Decimal
def decimal_range(*args):
zero, one = Decimal('0'), Decimal('1')
if len(args) == 1:
start, stop, step = zero, args[0], one
elif len(args) == 2:
start, stop, step = args + (one,)
elif len(args) == 3:
start, stop, step = args
else:
raise ValueError('Expected 1 or 2 arguments, got %s' % len(args))
if not all([type(arg) == Decimal for arg in (start, stop, step)]):
raise ValueError('Arguments must be passed as <type: Decimal>')
# neglect bad cases
if (start == stop) or (start > stop and step >= zero) or \
(start < stop and step <= zero):
return []
current = start
while abs(current) < abs(stop):
yield current
current += step
Sample outputs -
from decimal import Decimal as D
list(decimal_range(D('2')))
# [Decimal('0'), Decimal('1')]
list(decimal_range(D('2'), D('4.5')))
# [Decimal('2'), Decimal('3'), Decimal('4')]
list(decimal_range(D('2'), D('4.5'), D('0.5')))
# [Decimal('2'), Decimal('2.5'), Decimal('3.0'), Decimal('3.5'), Decimal('4.0')]
list(decimal_range(D('2'), D('4.5'), D('-0.5')))
# []
list(decimal_range(D('2'), D('-4.5'), D('-0.5')))
# [Decimal('2'),
# Decimal('1.5'),
# Decimal('1.0'),
# Decimal('0.5'),
# Decimal('0.0'),
# Decimal('-0.5'),
# Decimal('-1.0'),
# Decimal('-1.5'),
# Decimal('-2.0'),
# Decimal('-2.5'),
# Decimal('-3.0'),
# Decimal('-3.5'),
# Decimal('-4.0')]
Add auto-correction for the possibility of an incorrect sign on step:
def frange(start,step,stop):
step *= 2*((stop>start)^(step<0))-1
return [start+i*step for i in range(int((stop-start)/step))]
My solution:
def seq(start, stop, step=1, digit=0):
x = float(start)
v = []
while x <= stop:
v.append(round(x,digit))
x += step
return v
Here is my solution which works fine with float_range(-1, 0, 0.01) and works without floating point representation errors. It is not very fast, but works fine:
from decimal import Decimal
def get_multiplier(_from, _to, step):
digits = []
for number in [_from, _to, step]:
pre = Decimal(str(number)) % 1
digit = len(str(pre)) - 2
digits.append(digit)
max_digits = max(digits)
return float(10 ** (max_digits))
def float_range(_from, _to, step, include=False):
"""Generates a range list of floating point values over the Range [start, stop]
with step size step
include=True - allows to include right value to if possible
!! Works fine with floating point representation !!
"""
mult = get_multiplier(_from, _to, step)
# print mult
int_from = int(round(_from * mult))
int_to = int(round(_to * mult))
int_step = int(round(step * mult))
# print int_from,int_to,int_step
if include:
result = range(int_from, int_to + int_step, int_step)
result = [r for r in result if r <= int_to]
else:
result = range(int_from, int_to, int_step)
# print result
float_result = [r / mult for r in result]
return float_result
print float_range(-1, 0, 0.01,include=False)
assert float_range(1.01, 2.06, 5.05 % 1, True) ==\
[1.01, 1.06, 1.11, 1.16, 1.21, 1.26, 1.31, 1.36, 1.41, 1.46, 1.51, 1.56, 1.61, 1.66, 1.71, 1.76, 1.81, 1.86, 1.91, 1.96, 2.01, 2.06]
assert float_range(1.01, 2.06, 5.05 % 1, False)==\
[1.01, 1.06, 1.11, 1.16, 1.21, 1.26, 1.31, 1.36, 1.41, 1.46, 1.51, 1.56, 1.61, 1.66, 1.71, 1.76, 1.81, 1.86, 1.91, 1.96, 2.01]
I am only a beginner, but I had the same problem, when simulating some calculations. Here is how I attempted to work this out, which seems to be working with decimal steps.
I am also quite lazy and so I found it hard to write my own range function.
Basically what I did is changed my xrange(0.0, 1.0, 0.01) to xrange(0, 100, 1) and used the division by 100.0 inside the loop.
I was also concerned, if there will be rounding mistakes. So I decided to test, whether there are any. Now I heard, that if for example 0.01 from a calculation isn't exactly the float 0.01 comparing them should return False (if I am wrong, please let me know).
So I decided to test if my solution will work for my range by running a short test:
for d100 in xrange(0, 100, 1):
d = d100 / 100.0
fl = float("0.00"[:4 - len(str(d100))] + str(d100))
print d, "=", fl , d == fl
And it printed True for each.
Now, if I'm getting it totally wrong, please let me know.
The trick to avoid round-off problem is to use a separate number to move through the range, that starts and half the step ahead of start.
# floating point range
def frange(a, b, stp=1.0):
i = a+stp/2.0
while i<b:
yield a
a += stp
i += stp
Alternatively, numpy.arange can be used.
My answer is similar to others using map(), without need of NumPy, and without using lambda (though you could). To get a list of float values from 0.0 to t_max in steps of dt:
def xdt(n):
return dt*float(n)
tlist = map(xdt, range(int(t_max/dt)+1))

Pythonic way to remove elements from Numpy array closer than threshold

What is the best way to remove the minimal number of elements from a sorted Numpy array so that the minimal distance among the remaining is always bigger than a certain threshold?
For example, if the threshold is 1, the following sequence [0.1, 0.5, 1.1, 2.5, 3.] will become [0.1, 1.1, 2.5]. The 0.5 is removed because it is too close to 0.1 but then 1.1 is preserved because it is far enough from 0.1.
My current code:
import numpy as np
MIN_DISTANCE = 1
a = np.array([0.1, 0.5, 1.1, 2.5, 3.])
for i in range(len(a)-1):
if(a[i+1] - a[i] < MIN_DISTANCE):
a[i+1] = a[i]
a = np.unique(a)
a
array([0.1, 1.1, 2.5])
Is there a more efficient way to do so?
Note that my question is similar to Remove values from numpy array closer to each other but not exactly the same.
You could use numpy.ufunc.accumulate to iterate thru adjacent pairs of the array instead of the for loop.
The numpy.add.accumulate example or itertools.accumulate probably shows best what it's doing.
Along with numpy.frompyfunc your condition can be applied as ufunc (universal functions ).
Code: (with an extended array to cross check some additional cases, but works with your array as well)
import numpy as np
MIN_DISTANCE = 1
a = np.array([0.1, 0.5, 0.6, 0.7, 1.1, 2.5, 3., 4., 6., 6.1])
print("original: \n" + str(a))
def my_py_function(arr1, arr2):
if(arr2 - arr1 < MIN_DISTANCE):
arr2 = arr1
return arr2
my_np_function = np.frompyfunc(my_py_function, 2, 1)
my_np_function.accumulate(a, dtype=np.object, out=a).astype(float)
print("complete: \n" + str(a))
a = np.unique(a)
print("unique: \n" + str(a))
Result:
original:
[0.1 0.5 0.6 0.7 1.1 2.5 3. 4. 6. 6.1]
complete:
[0.1 0.1 0.1 0.1 1.1 2.5 2.5 4. 6. 6. ]
unique:
[0.1 1.1 2.5 4. 6. ]
Concerning execution time timeit shows a turnaround at array length of about 20.
Your code is much faster (relative) for your array length of 5
whereas for array length >>20 the accumulate option speeds up considerably (~35% in time for array length 300)

python: dividing a number by a numpy array

I have tried the top two solutions here and the bottom solution since it dealt with numpy but nothing worked.
I wanted to take 80.0 divided each element of my array name that a new array dx.
import numpy
L = 80.0
N = []
for n in range(-4, 10):
N.append(str(2 ** N))
N = np.array([N])
This is the set up. So what I have tried is:
dx = L / N
dx = map(lambda x: L / x, N)
dx = np.array([dx])
Lastly, keeping N as a list and keeping N as a numpy array and doing
dx = [x / N for x in N]
dx = np.array([dx])
Unfortunately, I haven't been able to find a post that helped or anything in the documentation. What can I do to achieve the desired result?
Your code contains several bugs and you have a lot of unnecessary casts, but however: why don't you try this with numpy directly?
Something like this:
import numpy as np
L = 80.0
N = 2 ** np.arange(-4, 10, dtype=np.float64)
dx = L / N
gives you the expected result
array([ 1.28000000e+03, 6.40000000e+02, 3.20000000e+02,
1.60000000e+02, 8.00000000e+01, 4.00000000e+01,
2.00000000e+01, 1.00000000e+01, 5.00000000e+00,
2.50000000e+00, 1.25000000e+00, 6.25000000e-01,
3.12500000e-01, 1.56250000e-01])
Btw. you can also implicitly force the dtype to be float when using dots:
N = 2 ** np.arange(-4., 10.)
You could do it in a single line using List comprehensions.
In [8]: N=[80.0/(2**n) for n in range(-4,10)]
In [10]: print N
[1280.0, 640.0, 320.0, 160.0, 80.0, 40.0, 20.0, 10.0, 5.0, 2.5, 1.25, 0.625, 0.3125, 0.15625]
You can avoid using Numpy for such tasks.
The for loop equivalent of this would be (without preallocating N):
In [11]: N=[]
In [12]: for n in range(-4,10):
....: N.append(80.0/(2**n))
....:
In [13]: print N
[1280.0, 640.0, 320.0, 160.0, 80.0, 40.0, 20.0, 10.0, 5.0, 2.5, 1.25, 0.625, 0.3125, 0.15625]

Categories