Special tensor contraction in Python - python
I need to perform a special type of tensor contraction. I want something of this kind:
A_{bg} = Sum_{a,a',a''} ( B_{a} C_{a'b} D_{a''g} )
where all the indices can have values 0,1 and the sum over a, a' and a'' is carried for all cases where a+a'+a'' = 1 or a+a'+a'' = 2. So it is like the reverse of the Einstein summation convention: I want to sum only when one of the three indices is different to the others.
Moreover, I want some flexibility with the number of indices that are not being summed: in the example the resulting tensor has 2 indices, and the sum is over products of elements of 3 tensors, one with one index, the other two with two indices. These numbers of indices are going to vary, so in general I would like to be able to write something like this:
A_{...} = Sum_{a,a',a''} ( B_{a...} C_{a...} D_{a''...} )
I want to point that the number of indices is not fixed, but it is controlled: I can know and specify how many indices every tensor has in each step.
I tried np.einsum(), but then apparently I am forced to sum over repeated indices in the standard Einstein convention, and I don't know how to implement the condition I exposed here.
And I cannot write everything with various for because, as I said, the number of indices of the tensors involved is not fixed.
Anyone has an idea?
From comments:
I would write what I put here in programming language like this:
tensa = np.zeros((2,2))
for be in range(2):
for ga in range(2):
for al in range(2):
for alp in range(2):
for alpp in range(res(al,alp),prod(al,alp)):
tensa[be,ga] += tensb[al] * tensc[alp,be] * tensd[alpp,ga]
where res and prod are two functions that ensure that al+alp+alpp = 1 or 2. The problem with this is that I need to specify all the indices involved, and I cannot do that in the general calculation for all the lattice.
First, lets write your example out in Python loops, to have a baseline for comparisons. If I understood you correctly, this is what you want to do:
b, g = 4, 5
B = np.random.rand(2)
C = np.random.rand(2, b)
D = np.random.rand(2, g)
out = np.zeros((b, g))
for j in (0, 1):
for k in (0, 1):
for l in (0, 1):
if j + k + l in (1, 2):
out += B[j] * C[k, :, None] * D[l, None, :]
When I run this, I get this output:
>>> out
array([[ 1.27679643, 2.26125361, 1.32775173, 1.5517918 , 0.47083151],
[ 0.84302586, 1.57516142, 1.1335904 , 1.14702252, 0.34226837],
[ 0.70592576, 1.34187278, 1.02080112, 0.99458563, 0.29535054],
[ 1.66907981, 3.07143067, 2.09677013, 2.20062463, 0.65961165]])
You can't get at this directly with np.einsum, but you can run it twice and get your result as the difference of these two:
>>> np.einsum('i,jk,lm->km', B, C, D) - np.einsum('i,ik,im->km', B, C, D)
array([[ 1.27679643, 2.26125361, 1.32775173, 1.5517918 , 0.47083151],
[ 0.84302586, 1.57516142, 1.1335904 , 1.14702252, 0.34226837],
[ 0.70592576, 1.34187278, 1.02080112, 0.99458563, 0.29535054],
[ 1.66907981, 3.07143067, 2.09677013, 2.20062463, 0.65961165]])
The first call to np.einsum is adding everything up, regardless of what the indices add up to. The second only adds up those where all three indices are the same. So obviously your result is the difference of the two.
Ideally, you could now go on to write something like:
>>>(np.einsum('i...,j...,k...->...', B, C, D) -
... np.einsum('i...,i...,i...->...', B, C, D))
and get your result regardless of the dimensions of your C and D arrays. If you try the first, you will get the following error message:
ValueError: operands could not be broadcast together with remapped shapes
[original->remapped]: (2)->(2,newaxis,newaxis) (2,4)->(4,newaxis,2,newaxis)
(2,5)->(5,newaxis,newaxis,2)
The problem is that, since you are not specifying what you want to do with the b and g dimensions of your tensors, it tries to broadcast them together, and since they are different, it fails. You can get it to work by adding extra dimensions of size 1:
>>> (np.einsum('i...,j...,k...->...', B, C, D[:, None]) -
... np.einsum('i...,i...,i...->...', B, C, D[:, None]))
array([[ 1.27679643, 2.26125361, 1.32775173, 1.5517918 , 0.47083151],
[ 0.84302586, 1.57516142, 1.1335904 , 1.14702252, 0.34226837],
[ 0.70592576, 1.34187278, 1.02080112, 0.99458563, 0.29535054],
[ 1.66907981, 3.07143067, 2.09677013, 2.20062463, 0.65961165]])
If you wanted all the axes of B to be placed before all the axes of C, and these before all the axes of D, the following seems to work, at least as far as creating an output of the right shape, although you may want to double check that the result is really what you want:
>>> B = np.random.rand(2, 3)
>>> C = np.random.rand(2, 4, 5)
>>> D = np.random.rand(2, 6)
>>> C_idx = (slice(None),) + (None,) * (B.ndim - 1)
>>> D_idx = C_idx + (None,) * (C.ndim - 1)
>>> (np.einsum('i...,j...,k...->...', B, C[C_idx], D[D_idx]) -
... np.einsum('i...,i...,i...->...', B, C[C_idx], D[D_idx])).shape
(3L, 4L, 5L, 6L)
EDIT From the comments, if instead of just the first axis of each tensor having to be reduced over, it is the first two, then the above could be written as:
>>> B = np.random.rand(2, 2, 3)
>>> C = np.random.rand(2, 2, 4, 5)
>>> D = np.random.rand(2, 2, 6)
>>> C_idx = (slice(None),) * 2 + (None,) * (B.ndim - 2)
>>> D_idx = C_idx + (None,) * (C.ndim - 2)
>>> (np.einsum('ij...,kl...,mn...->...', B, C[C_idx], D[D_idx]) -
... np.einsum('ij...,ij...,ij...->...', B, C[C_idx], D[D_idx])).shape
(3L, 4L, 5L, 6L)
More generally, if reducing over d indices, C_idx and D_idx would look like:
>>> C_idx = (slice(None),) * d + (None,) * (B.ndim - d)
>>> D_idx = C_idx + (None,) * (C.ndim - d)
and the calls to np.einsum would need to have d letters in the indexing, unique in the first call, repeating in the second.
EDIT 2 So what exactly goes on with C_idx and D_idx? Take the last example, with B, C and D with shapes (2, 2, 3), (2, 2, 4, 5) and (2, 2, 6). C_idx is made up of two empty slices, plus as many Nones as the number of dimensions of B minus 2, so when we take C[C_idx] the result has shape (2, 2, 1, 4, 5). Similarly D_idx is C_idx plus as many Nones as the number of dimensions of C minus 2, so the result of D[D_idx] has shape (2, 2, 1, 1, 1, 6). These three arrays don't braodcast together, but np.einsum adds additional dimensions of size 1, i.e. the "remapped" shapes of the error above, so the resulting arrays turn out to have extra trailing ones, and the shapes amtch as follows:
(2, 2, 3, 1, 1, 1)
(2, 2, 1, 4, 5, 1)
(2, 2, 1, 1, 1, 6)
The first two axes are reduced, so the disappear from the output, and in the other cases, broadcasting applies, where a dimension of size 1 is replicated to match a larger one, so the output is (3, 4, 5, 6) as we wanted.
#hpaulj proposes a method using "Levi-Civita like" tensors, that should in theory be faster, see my comments to the original question. Here's some code for comparison:
b, g = 5000, 2000
B = np.random.rand(2)
C = np.random.rand(2, b)
D = np.random.rand(2, g)
def calc1(b, c, d):
return (np.einsum('i,jm,kn->mn', b, c, d) -
np.einsum('i,im,in->mn', b, c, d))
def calc2(b, c, d):
return np.einsum('ijk,i,jm,kn->mn', calc2.e, b, c, d)
calc2.e = np.ones((2,2,2))
calc2.e[0, 0, 0] = 0
calc2.e[1, 1, 1] = 0
But when running it:
%timeit calc1(B, C, D)
1 loops, best of 3: 361 ms per loop
%timeit calc2(B, C, D)
1 loops, best of 3: 643 ms per loop
np.allclose(calc1(B, C, D), calc2(B, C, D))
Out[48]: True
A surprising result, which I can't explain...
Related
How to get two synchronised generators from a function
i have a nested tuple like this one :
this_one = (w,h,(1,2,4,8,16),(0,2,3,4,6),("0"),("0"),(0,1))
It will be used to feed:
itertools.product(*this_one)
w and h have to be both generators.
Generated values from h depends on generated values from w as in this function:
def This_Function(maxvalue):
for i in range(maxvalue):
w_must_generate = i
for j in range(i+1):
h_must_generate = j
Iv tried using a yield in This_function(maxvalue):
def This_Function(maxvalue):
for i in range(maxvalue):
for j in range(i+1):
yield i,j
Returning a single generator object which gives:
which_gives = itertools.product(*this_one)
for result in which_gives:
print(result)
...
((23,22),(16),(6),(0),(0),(0))
((23,22),(16),(6),(0),(0),(1))
((23,23),(1),(0),(0),(0),(0))
...
A single tuple at result[0] holding 2 values.
And this is not what i want .
The result i want has to be like the following:
the_following = itertools.product(*this_one)
for result in the_following:
print(result)
...
((23),(22),(16),(6),(0),(0),(0))
((23),(22),(16),(6),(0),(0),(1))
((23),(23),(1),(0),(0),(0),(0))
...
Without having to do something like :
the_following = itertools.product(*this_one)
for result in the_following:
something_like = tuple(result[0][0],result[0][1],result[1],...,result[5])
print(something_like)
...
((23),(22),(16),(6),(0),(0),(0))
((23),(22),(16),(6),(0),(0),(1))
((23),(23),(1),(0),(0),(0),(0))
...
Any ideas ?
That's fundamentally not how product works. From the documentation: Before product() runs, it completely consumes the input iterables, keeping pools of values in memory to generate the products. Accordingly, it is only useful with finite inputs. Thus, having the second one be dependent won't get you the result you want, assuming that is for h values to always be w+1..max in the resulting output sequence. I'm not sure why you couldn't just run it with your This_function approach and destructure it with a generator expression: result = ((a, b, c, d, e, f, g) for ((a, b), c, d, e, f, g) in itertools.product(*this_one))
Since the values are all entirely determined, this works and is fairly legible:
from itertools import product
maxvalue = 5
w = (i for i in range(maxvalue) for j in range(i + 1))
h = (j for i in range(maxvalue) for j in range(i + 1))
this_one = (w, h, (1, 2, 4, 8, 16), (0, 2, 3, 4, 6), ("0"), ("0"), (0, 1))
result = product(*this_one)
The values yielded from w are basically 0, 1, 1, 2, 2, 2, 3, 3, 3, 3, ... and the values yielded from h are 0, 0, 1, 0, 1, 2, 0, 1, 2, 3, .... Plus, the code pretty much looks like your description of the desired behaviour, which is generally a good thing.
Both w and h are generators, no space is wasted and the solution is fairly optimal. I just can't tell if result is what you actually need, but that depends on what you need and expect product() to do here.
How to correctly substitude indexed variable in sympy?
I try to use IndexBase for summation, but dont understand, how to do substitution of sequences:
A = sympy.IndexedBase('A')
i = sympy.Symbol('i', integer=True)
N = sympy.Symbol('N', integer=True)
S = sympy.Sum(A[i], (i, 0, N))
Trace = sympy.Sum(A[i, i], (i, 0, N))
S.subs([(A, range(3)), (N, 2)]).doit() # python3 range
# result: 3
S.subs([(A, [8, 16, 32]), (N, 2)]).doit()
# result: A[0] + A[1] + A[2]
S.subs([(A, numpy.arange(3)), (N, 2)]).doit()
# result: A[0] + A[1] + A[2]
Trace.subs([(A, numpy.diag([2, 4, 8])), (N, 2)]).doit()
# result: A[0, 0] + A[1, 1] + A[2, 2]
The only case which works is substitution of range. Could you explain, how to substitude it in general case?
Usually one substitutes for Indexed objects A[i] rather than for IndexedBase A. This works if the sum is written out explicitly by doit prior to substitution.
S.subs(N, 2).doit().subs([(A[i], i**2) for i in range(3)]) # 5
or
values = [8, 16, 32]
S.subs(N, 2).doit().subs([(A[i], values[i]) for i in range(3)]) # 56
Similarly, Trace.subs(N, 2).doit().subs([(A[i, i], values[i]) for i in range(3)]) returns 56.
The substitution by Python range works because it's sympified by subs into SymPy's Range object, which is something that can be a part of a SymPy expression.
>>> S.subs([(A, range(3)), (N, 2)])
Sum(Range(0, 3, 1)[i], (i, 0, 2))
It looks like one should be able to similarly substitute with SymPy's SeqFormula objects:
>>> n = sympy.symbols('n')
>>> S.subs([(A, sympy.sequence(n**2, (n, 0, 3))), (N, 3)])
Sum(SeqFormula(n**2, (n, 0, 3))[i], (i, 0, 3))
But subsequent doit fails here with SympifyError: None which looks like a bug.
Merge two lists of intervals with priority for one list
I'm currently stuck with an algorithm problem in which I want to optimize the complexity. I have two lists of intervals S = [[s1, s2], [s3, s4], ..., [sn-1, sn]] and W = [[w1, w2], [w3, w4], ..., [wm-1, wm]] that I want to merge respecting ordinal order, and intervals of S have priority over those of W. (S for strong, W for weak) For example, that priority imply : S = [[5,8]] and W = [[1, 5], [7, 10]] will give : res = [[1, 4, W], [5, 8, S], [9, 10, W]]. Here intervals from W are cropped in priority for intervals of S S = [[5, 8]] and W = [[2, 10]] will give : res = [[2, 4, W], [5, 8, S], [9, 10, W]]. Here the interval of W is split into two parts because S has priority. While merging those lists, I need to keep track of the strong of weak nature of those intervals by writing a third element beside each interval, that we can call the symbol. that's why the result is something like : [[1, 4, W], [5, 8, S], [9, 10, W]]. Finally, as the union of all interval does not cover all integers in a certain range, we have a third symbol, let's say B for blank which fill missing interval : [[1, 2, W], [5, 8, S], [9, 10, W], [16, 20, S]] will be filled in to become : [1, 2, W], [3, 4, B], [5, 8, S], [9, 10, W], [11, 15, B], [16, 20, S]] My first attempt was very naive and lazy (because I first wanted it to work) : If the greatest integer covered by these two lists of intervals is M, then I created a list of size M filled with B symbols : res = [B]*M = [B, B, B ..., B] Then I first take interval from W one by one and rewrite elements from res of index in this interval to change its symbol to W. Next, I do the same with intervals of S, and the priority is respected because I overwrite with the symbol S in the last step. It gives something like : [B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B, B] [B, B, B, W, W, W, W, B, W, W, W, W, B, W, W, B, B] [B, B, S, S, W, W, W, B, S, S, W, W, B, S, W, B, B] Finally, I go through the big list one last time to factorize and recreate intervals with its corresponding symbols. Previous example gives : [[1, 2, B], [3, 4, S], [5, 7, W], [8, 8, B], [9, 10, S], [11, 12, W], [13, 13, B], [14, 14, S], [15, 15, W], [16, 17, B]] Unfortunately but predictably, this algorithm is not usable in practice : M is around 1000000 in my application and this algorithm is O(n2) if I'm not mistaken. So, I would like some advice and directions to solve this algorithmic complexity problem. I'm sure that this problem looks alike a well-known algorithmic problem but I don't know where to go. My few ideas to improve that for now can be used to optimize the algorithm, but are quite complex to implement so I think there is better ideas. but here they are : Do the same kind of overwrite process to respect priority : in list W, insert intervals of S with overwriting when necessary to respect priority. Then fill in this list to insert missing interval with B symbol. But we would have an heavy use of if to compare intervals because of the great amount of cases. Construct a new list while browsing S and W step by step. In this idea we would have one cursor by list to go from interval to interval until the end of one of the two lists. Again we use a lot of if and cases we insert intervals in the new list with respect to priority. But it raises the same complex problem with the great amount of cases. I hope I made myself clear, if not I can explain in other way. Please teach me with experience and cleverness :) Thanks EDIT: here is my "naive" algorithm code: def f(W, S, size): #We first write one symbol per sample int_result = ['B'] * size for interval in W: for i in range(interval[0], interval[1]+1): int_result[i] = 'W' for interval in S: for i in range(interval[0], interval[1]+1): int_result[i] = 'S' #we then factorize: we store one symbol for an interval of the same symbol. symbols_intervals = [] sym = int_result[0] start = 0 for j in range(len(int_result)): if int_result[j] != sym: symbols_intervals.append([start, j-1, sym]) sym = all_symbols[j] start = j if j == len(int_result)-1: symbols_intervals.append([start, j-1, sym]) return symbols_intervals
Your naive method sounds very reasonable; I think the time complexity of it is O(NM), where N is the number of intervals you're trying to resolve, and M is the the range over which you're trying to resolve them. The difficulty you might have is that you also have space complexity of O(M), which might use up a fair bit of memory.
Here's a method for merging without building a "master list", which may be faster; because it treats intervals as objects, complexity is no longer tied to M.
I'll represent an interval (or list of intervals) as a set of tuples (a,b,p), each of which indicates the time points from a to b, inclusively, with the integer priority p (W can be 1, and S can be 2). In each interval, it must be the case that a < b. Higher priorities are preferred.
We need a predicate to define the overlap between two intervals:
def has_overlap(i1, i2):
'''Returns True if the intervals overlap each other.'''
(a1, b1, p1) = i1
(a2, b2, p2) = i2
A = (a1 - a2)
B = (b2 - a1)
C = (b2 - b1)
D = (b1 - a2)
return max(A * B, D * C, -A * D, B * -C) >= 0
When we find overlaps, we need to resolve them. This method takes care of that, respecting priority:
def join_intervals(i1, i2):
'''
Joins two intervals, fusing them if they are of the same priority,
and trimming the lower priority one if not.
Invariant: the interval(s) returned by this function will not
overlap each other.
>>> join_intervals((1,5,2), (4,8,2))
{(1, 8, 2)}
>>> join_intervals((1,5,2), (4,8,1))
{(1, 5, 2), (6, 8, 1)}
>>> join_intervals((1,3,2), (4,8,2))
{(1, 3, 2), (4, 8, 2)}
'''
if has_overlap(i1, i2):
(a1, b1, p1) = i1
(a2, b2, p2) = i2
if p1 == p2:
# UNION
return set([(min(a1, a2), max(b1, b2), p1)])
# DIFFERENCE
if p2 < p1:
(a1, b1, p1) = i2
(a2, b2, p2) = i1
retval = set([(a2, b2, p2)])
if a1 < a2 - 1:
retval.add((a1, a2 - 1, p1))
if b1 > b2 + 1:
retval.add((b2 + 1, b1, p1))
return retval
else:
return set([i1, i2])
Finally, merge_intervals takes an iterable of intervals and joins them together until there are no more overlaps:
import itertools
def merge_intervals(intervals):
'''Resolve overlaps in an iterable of interval tuples.'''
# invariant: retval contains no mutually overlapping intervals
retval = set()
for i in intervals:
# filter out the set of intervals in retval that overlap the
# new interval to add O(N)
overlaps = set([i2 for i2 in retval if has_overlap(i, i2)])
retval -= overlaps
overlaps.add(i)
# members of overlaps can potentially all overlap each other;
# loop until all overlaps are resolved O(N^3)
while True:
# find elements of overlaps which overlap each other O(N^2)
found = False
for i1, i2 in itertools.combinations(overlaps, 2):
if has_overlap(i1, i2):
found = True
break
if not found:
break
overlaps.remove(i1)
overlaps.remove(i2)
overlaps.update(join_intervals(i1, i2))
retval.update(overlaps)
return retval
I think this has worst-case time complexity of O(N^4), although the average case should be fast. In any case, you may want to time this solution against your simpler method, to see what works better for your problem.
As far as I can see, my merge_intervals works for your examples:
# example 1
assert (merge_intervals({(5, 8, 2), (1, 5, 1), (7, 10, 1)}) ==
{(1, 4, 1), (5, 8, 2), (9, 10, 1)})
# example 2
assert (merge_intervals({(5, 8, 2), (2, 10, 1)}) ==
{(2, 4, 1), (5, 8, 2), (9, 10, 1)})
To cover the case with blank (B) intervals, simply add another interval tuple which covers the whole range with priority 0: (1, M, 0):
# example 3 (with B)
assert (merge_intervals([(1, 2, 1), (5, 8, 2), (9, 10, 1),
(16, 20, 2), (1, 20, 0)]) ==
{(1, 2, 1), (3, 4, 0), (5, 8, 2),
(9, 10, 1), (11, 15, 0), (16, 20, 2)})
The below solution has O(n + m) complexity, where n and m are the lengths of S and W lists. It assumes that S and W are internally sorted. def combine(S, W): s, w = 0, 0 # indices of S and W common = [] while s < len(S) or w < len(W): # only weak intervals remain, so append them to common if s == len(S): common.append((W[w][0], W[w][1], 'W')) w += 1 # only strong intervals remain, so append them to common elif w == len(W): common.append((S[s][0], S[s][1], 'S')) s += 1 # assume that the strong interval starts first elif S[s][0] <= W[w][0]: W[w][0] = max(W[w][0], S[s][1]+1) if W[w][0] > W[w][1]: # drop the weak interval w += 1 common.append((S[s][0], S[s][1], 'S')) s += 1 # assume that the weak interval starts first elif S[s][0] > W[w][0]: # end point of weak interval before the start of the strong if W[w][1] < S[s][0]: common.append(W[w][0], W[w][1], 'W') w += 1 # end point of the weak interval between a strong interval elif S[s][0] <= W[w][1] <= S[s][1]: W[w][1] = S[s][0] - 1 common.append((W[w][0], W[w][1], 'W')) w += 1 # end point of the weak interval after the end point of the strong elif W[w][1] > S[s][1]: common.append((W[w][0], S[s][0]-1, 'W')) W[w][0] = S[s][1] + 1 return common print combine(S=[[5,8]], W=[[1, 5],[7, 10]]) print combine(S=[[5,8]], W=[[2,10]])
numpy / python - indexing by arrays with duplicates
I'm trying to make a 3D histogram. Initially h = zeros((6,6,8)). I'll explain my problem with an example. Suppose I have 3 lists of coordinates for h, each list for one dimension: x = array([2,1,0,1,2,2]) y = array([1,3,0,3,2,1]) z = array([6,2,0,2,5,6]) (the coordinates (x[0],y[0],z[0]) and (x[6],y[6],z[6]) are duplicates, and (x[1],y[1],z[1]) and (x[3],y[3],z[3]) also are) and also a list of corresponding quantities to accumulate into h: q = array([1,2,5,9,8,7]) I tried and h[x,y,z] += q does not work because only q[5] = 7 is added to h[2,1,6] and q[0] = 1 is not. How can I work around this? Thank you.
IIUC, you want np.add.at. To quote the docs: "For addition ufunc, this method is equivalent to a[indices] += b, except that results are accumulated for elements that are indexed more than once." For example: >>> np.add.at(h, [x,y,z], q) >>> for i, val in np.ndenumerate(h): ... if val: print(i, val) ... ((0, 0, 0), 5.0) ((1, 3, 2), 11.0) ((2, 1, 6), 8.0) ((2, 2, 5), 8.0)
creating a non-literal python tuple programmatically
I want to create a tuple of length m, with a 1 in each position except for one n in position k. e.g.: m=5, n=7, k=3 should yield (1,1,1,7,1) (length 5 with a 7 in position 3) How can I do this?
>>> m, n, k = 5, 7, 3 >>> tuple(n if i == k else 1 for i in range(m)) (1, 1, 1, 7, 1)
First construct a list, then turn it into a tuple if you really need to: def make_tuple(m, n, k): a = [1] * m a[k] = n return tuple(a) Example: >>> make_tuple(m=5, n=7, k=3) (1, 1, 1, 7, 1)
(1,) * k + (n,) + (1,) * (m-k-1)