We Keep Coding

Issues implementing the "Wave Collapse Function" algorithm in Python

In a nutshell:
My implementation of the Wave Collapse Function algorithm in Python 2.7 is flawed but I'm unable to identify where the problem is located. I would need help to find out what I'm possibly missing or doing wrong.
What is the Wave Collapse Function algorithm ?
It is an algorithm written in 2016 by Maxim Gumin that can generate procedural patterns from a sample image. You can see it in action here (2D overlapping model) and here (3D tile model).
Goal of this implementation:
To boil down the algorithm (2D overlapping model) to its essence and avoid the redondancies and clumsiness of the original C# script (surprisingly long and difficult to read). This is an attempt to make a shorter, clearer and pythonic version of this algorithm.
Characteristics of this implementation:
I'm using Processing (Python mode), a software for visual design that makes image manipulation easier (no PIL, no Matplotlib, ...). The main drawbacks are that I'm limited to Python 2.7 and can NOT import numpy.
Unlike the original version, this implementation:
is not object oriented (in its current state), making it easier to understand / closer to pseudo-code
is using 1D arrays instead of 2D arrays
is using array slicing for matrix manipulation
The Algorithm (as I understand it)
1/ Read the input bitmap, store every NxN patterns and count their occurences.
(optional: Augment pattern data with rotations and reflections.)
For example, when N = 3:
2/ Precompute and store every possible adjacency relations between patterns.
In the example below, patterns 207, 242, 182 and 125 can overlap the right side of pattern 246
3/ Create an array with the dimensions of the output (called W for wave). Each element of this array is an array holding the state (True of False) of each pattern.
For example, let's say we count 326 unique patterns in input and we want our output to be of dimensions 20 by 20 (400 cells). Then the "Wave" array will contain 400 (20x20) arrays, each of them containing 326 boolan values.
At start, all booleans are set to True because every pattern is allowed at any position of the Wave.
W = [[True for pattern in xrange(len(patterns))] for cell in xrange(20*20)]
4/ Create another array with the dimensions of the output (called H). Each element of this array is a float holding the "entropy" value of its corresponding cell in output.
Entropy here refers to Shannon Entropy and is computed based on the number of valid patterns at a specific location in the Wave. The more a cell has valid patterns (set to True in the Wave), the higher its entropy is.
For example, to compute the entropy of cell 22 we look at its corresponding index in the wave (W[22]) and count the number of booleans set to True. With that count we can now compute the entropy with the Shannon formula. The result of this calculation will be then stored in H at the same index H[22]
At start, all cells have the same entropy value (same float at every position in H) since all patterns are set to True, for each cell.
H = [entropyValue for cell in xrange(20*20)]
These 4 steps are introductory steps, they are necessary to initalize the algorithm. Now starts the core of the algorithm:
5/ Observation:
Find the index of the cell with the minimum nonzero entropy (Note that at the very first iteration all entropies are equal so we need to pick the index of a cell randomly.)
Then, look at the still valid patterns at the corresponding index in the Wave and select one of them randomly, weighted by the frequency that pattern appears in the input image (weighted choice).
For example if the lowest value in H is at index 22 (H[22]), we look at all the patterns set to True at W[22] and pick one randomly based on the number of times it appears in the input. (Remember at step 1 we've counted the number of occurences for each pattern). This insures that patterns appear with a similar distribution in the output as are found in the input.
6/ Collapse:
We now assign the index of the selected pattern to the cell with the minimum entropy. Meaning that every pattern at the corresponding location in the Wave are set to False except for the one that has been chosen.
For example if pattern 246 in W[22] was set to True and has been selected, then all other patterns are set to False. Cell 22 is assigned pattern 246.
In output cell 22 will be filled with the first color (top left corner) of pattern 246. (blue in this example)
7/ Propagation:
Because of adjacency constraints, that pattern selection has consequences on the neighboring cells in the Wave. The arrays of booleans corresponding to the cells on the left and right, on top of and above the recently collapsed cell need to be updated accordingly.
For example if cell 22 has been collapsed and assigned with pattern 246, then W[21] (left), W[23] (right), W[2] (up) and W[42] (down) have to be modified so as they only keep to True the patterns that are adjacent to pattern 246.
For example, looking back at the picture of step 2, we can see that only patterns 207, 242, 182 and 125 can be placed on the right of pattern 246. That means that W[23] (right of cell 22) needs to keep patterns 207, 242, 182 and 125 as True and set all other patterns in the array as False. If these patterns are not valid anymore (already set to False because of a previous constraint) then the algorithm is facing a contradiction.
8/ Updating entropies
Because a cell has been collapsed (one pattern selected, set to True) and its surrounding cells updated accordingly (setting non adjacent patterns to False) the entropy of all these cells have changed and needs to be computed again. (Remember that the entropy of a cell is correlated to the number of valid pattern it holds in the Wave.)
In the example, the entropy of cell 22 is now 0, (H[22] = 0, because only pattern 246 is set to True at W[22]) and the entropy of its neighboring cells have decreased (patterns that were not adjacent to pattern 246 have been set to False).
By now the algorithm arrives at the end of the first iteration and will loop over steps 5 (find cell with minimum non zero entropy) to 8 (update entropies) until all cells are collapsed.
My script
You'll need Processing with Python mode installed to run this script.
It contains around 80 lines of code (short compared to the ~1000 lines of the original script) that are fully annotated so it can be rapidly understood. You'll also need to download the input image and change the path on line 16 accordingly.
from collections import Counter
from itertools import chain, izip
import math
d = 20 # dimensions of output (array of dxd cells)
N = 3 # dimensions of a pattern (NxN matrix)
Output = [120 for i in xrange(d*d)] # array holding the color value for each cell in the output (at start each cell is grey = 120)
def setup():
size(800, 800, P2D)
textSize(11)
global W, H, A, freqs, patterns, directions, xs, ys, npat
img = loadImage('Flowers.png') # path to the input image
iw, ih = img.width, img.height # dimensions of input image
xs, ys = width//d, height//d # dimensions of cells (squares) in output
kernel = [[i + n*iw for i in xrange(N)] for n in xrange(N)] # NxN matrix to read every patterns contained in input image
directions = [(-1, 0), (1, 0), (0, -1), (0, 1)] # (x, y) tuples to access the 4 neighboring cells of a collapsed cell
all = [] # array list to store all the patterns found in input
# Stores the different patterns found in input
for y in xrange(ih):
for x in xrange(iw):
''' The one-liner below (cmat) creates a NxN matrix with (x, y) being its top left corner.
This matrix will wrap around the edges of the input image.
The whole snippet reads every NxN part of the input image and store the associated colors.
Each NxN part is called a 'pattern' (of colors). Each pattern can be rotated or flipped (not mandatory). '''
cmat = [[img.pixels[((x+n)%iw)+(((a[0]+iw*y)/iw)%ih)*iw] for n in a] for a in kernel]
# Storing rotated patterns (90°, 180°, 270°, 360°)
for r in xrange(4):
cmat = zip(*cmat[::-1]) # +90° rotation
all.append(cmat)
# Storing reflected patterns (vertical/horizontal flip)
all.append(cmat[::-1])
all.append([a[::-1] for a in cmat])
# Flatten pattern matrices + count occurences
''' Once every pattern has been stored,
- we flatten them (convert to 1D) for convenience
- count the number of occurences for each one of them (one pattern can be found multiple times in input)
- select unique patterns only
- store them from less common to most common (needed for weighted choice)'''
all = [tuple(chain.from_iterable(p)) for p in all] # flattern pattern matrices (NxN --> [])
c = Counter(all)
freqs = sorted(c.values()) # number of occurences for each unique pattern, in sorted order
npat = len(freqs) # number of unique patterns
total = sum(freqs) # sum of frequencies of unique patterns
patterns = [p[0] for p in c.most_common()[:-npat-1:-1]] # list of unique patterns sorted from less common to most common
# Computes entropy
''' The entropy of a cell is correlated to the number of possible patterns that cell holds.
The more a cell has valid patterns (set to 'True'), the higher its entropy is.
At start, every pattern is set to 'True' for each cell. So each cell holds the same high entropy value'''
ent = math.log(total) - sum(map(lambda x: x * math.log(x), freqs)) / total
# Initializes the 'wave' (W), entropy (H) and adjacencies (A) array lists
W = [[True for _ in xrange(npat)] for i in xrange(d*d)] # every pattern is set to 'True' at start, for each cell
H = [ent for i in xrange(d*d)] # same entropy for each cell at start (every pattern is valid)
A = [[set() for dir in xrange(len(directions))] for i in xrange(npat)] #see below for explanation
# Compute patterns compatibilities (check if some patterns are adjacent, if so -> store them based on their location)
''' EXAMPLE:
If pattern index 42 can placed to the right of pattern index 120,
we will store this adjacency rule as follow:
A[120][1].add(42)
Here '1' stands for 'right' or 'East'/'E'
0 = left or West/W
1 = right or East/E
2 = up or North/N
3 = down or South/S '''
# Comparing patterns to each other
for i1 in xrange(npat):
for i2 in xrange(npat):
for dir in (0, 2):
if compatible(patterns[i1], patterns[i2], dir):
A[i1][dir].add(i2)
A[i2][dir+1].add(i1)
def compatible(p1, p2, dir):
'''NOTE:
what is refered as 'columns' and 'rows' here below is not really columns and rows
since we are dealing with 1D patterns. Remember here N = 3'''
# If the first two columns of pattern 1 == the last two columns of pattern 2
# --> pattern 2 can be placed to the left (0) of pattern 1
if dir == 0:
return [n for i, n in enumerate(p1) if i%N!=2] == [n for i, n in enumerate(p2) if i%N!=0]
# If the first two rows of pattern 1 == the last two rows of pattern 2
# --> pattern 2 can be placed on top (2) of pattern 1
if dir == 2:
return p1[:6] == p2[-6:]
def draw(): # Equivalent of a 'while' loop in Processing (all the code below will be looped over and over until all cells are collapsed)
global H, W, grid
### OBSERVATION
# Find cell with minimum non-zero entropy (not collapsed yet)
'''Randomly select 1 cell at the first iteration (when all entropies are equal),
otherwise select cell with minimum non-zero entropy'''
emin = int(random(d*d)) if frameCount <= 1 else H.index(min(H))
# Stoping mechanism
''' When 'H' array is full of 'collapsed' cells --> stop iteration '''
if H[emin] == 'CONT' or H[emin] == 'collapsed':
print 'stopped'
noLoop()
return
### COLLAPSE
# Weighted choice of a pattern
''' Among the patterns available in the selected cell (the one with min entropy),
select one pattern randomly, weighted by the frequency that pattern appears in the input image.
With Python 2.7 no possibility to use random.choice(x, weight) so we have to hard code the weighted choice '''
lfreqs = [b * freqs[i] for i, b in enumerate(W[emin])] # frequencies of the patterns available in the selected cell
weights = [float(f) / sum(lfreqs) for f in lfreqs] # normalizing these frequencies
cumsum = [sum(weights[:i]) for i in xrange(1, len(weights)+1)] # cumulative sums of normalized frequencies
r = random(1)
idP = sum([cs < r for cs in cumsum]) # index of selected pattern
# Set all patterns to False except for the one that has been chosen
W[emin] = [0 if i != idP else 1 for i, b in enumerate(W[emin])]
# Marking selected cell as 'collapsed' in H (array of entropies)
H[emin] = 'collapsed'
# Storing first color (top left corner) of the selected pattern at the location of the collapsed cell
Output[emin] = patterns[idP][0]
### PROPAGATION
# For each neighbor (left, right, up, down) of the recently collapsed cell
for dir, t in enumerate(directions):
x = (emin%d + t[0])%d
y = (emin/d + t[1])%d
idN = x + y * d #index of neighbor
# If that neighbor hasn't been collapsed yet
if H[idN] != 'collapsed':
# Check indices of all available patterns in that neighboring cell
available = [i for i, b in enumerate(W[idN]) if b]
# Among these indices, select indices of patterns that can be adjacent to the collapsed cell at this location
intersection = A[idP][dir] & set(available)
# If the neighboring cell contains indices of patterns that can be adjacent to the collapsed cell
if intersection:
# Remove indices of all other patterns that cannot be adjacent to the collapsed cell
W[idN] = [True if i in list(intersection) else False for i in xrange(npat)]
### Update entropy of that neighboring cell accordingly (less patterns = lower entropy)
# If only 1 pattern available left, no need to compute entropy because entropy is necessarily 0
if len(intersection) == 1:
H[idN] = '0' # Putting a str at this location in 'H' (array of entropies) so that it doesn't return 0 (float) when looking for minimum entropy (min(H)) at next iteration
# If more than 1 pattern available left --> compute/update entropy + add noise (to prevent cells to share the same minimum entropy value)
else:
lfreqs = [b * f for b, f in izip(W[idN], freqs) if b]
ent = math.log(sum(lfreqs)) - sum(map(lambda x: x * math.log(x), lfreqs)) / sum(lfreqs)
H[idN] = ent + random(.001)
# If no index of adjacent pattern in the list of pattern indices of the neighboring cell
# --> mark cell as a 'contradiction'
else:
H[idN] = 'CONT'
# Draw output
''' dxd grid of cells (squares) filled with their corresponding color.
That color is the first (top-left) color of the pattern assigned to that cell '''
for i, c in enumerate(Output):
x, y = i%d, i/d
fill(c)
rect(x * xs, y * ys, xs, ys)
# Displaying corresponding entropy value
fill(0)
text(H[i], x * xs + xs/2 - 12, y * ys + ys/2)
Problem
Despite all my efforts to carefully put into code all the steps described above, this implementation returns very odd and disappointing results:
Example of a 20x20 output
Both the pattern distribution and the adjacency constraints seem to be respected (same amount of blue, green, yellow and brown colors as in input and same kind of patterns: horizontal ground , green stems).
However these patterns:
are often disconnected
are often incomplete (lack of "heads" composed of 4-yellow petals)
run into way too many contradictory states (grey cells marked as "CONT")
On that last point, I should clarify that contradictory states are normal but should happen very rarely (as stated in the middle of page 6 of this paper and in this article)
Hours of debugging convinced me that introductory steps (1 to 5) are correct (counting and storing patterns, adjacency and entropy computations, arrays initialization). This has led me to think that something must be off with the core part of the algorithm (steps 6 to 8). Either I am implementing one of these steps incorrectly or I am missing a key element of the logic.
Any help regarding that matter would thus be immensely appreciated !
Also, any answer that is based on the script provided (using Processing or not) is welcomed.
Useful additionnal ressources:
This detailed article from Stephen Sherratt and this explanatory paper from Karth & Smith.
Also, for comparison I would suggest to check this other Python implementation (contains a backtracking mechanism that isn't mandatory) .
Note: I did my best to make this question as clear as possible (comprehensive explanation with GIFs and illustrations, fully annotated code with useful links and ressources) but if for some reasons you decide to vote it down, please leave a brief comment to explain why you're doing so.

The hypothesis suggested by #mbrig and #Leon that the propagation step iterates over a whole stack of cells (instead of being limited to a set of 4 direct neighbors) was correct. The following is an attempt to provide further details while answering my own questions.
The problem occured at step 7, while propagating. The original algorithm does update the 4 direct neighbors of a specific cell BUT:
the index of that specific cell is in turns replaced by the indices of the previously updated neighbors.
this cascading process is triggered every time a cell is collapsed
and last as long as the adjacent patterns of a specific cell are available in 1 of its neighboring cell
In other words, and as mentionned in the comments, this is a recursive type of propagation that updates not only the neighbors of the collapsed cell, but also the neighbors of the neighbors... and so on as long as adjacencies are possible.
Detailed Algorithm
Once a cell is collapsed, its index is put in a stack. That stack is meant later to temporarily store indices of neighoring cells
stack = set([emin]) #emin = index of cell with minimum entropy that has been collapsed
The propagation will last as long as that stack is filled with indices:
while stack:
First thing we do is pop() the last index contained in the stack (the only one for now) and get the indices of its 4 neighboring cells (E, W, N, S). We have to keep them withing bounds and make sure they wrap around.
while stack:
idC = stack.pop() # index of current cell
for dir, t in enumerate(mat):
x = (idC%w + t[0])%w
y = (idC/w + t[1])%h
idN = x + y * w # index of neighboring cell
Before going any further, we make sure the neighboring cell is not collapsed yet (we don't want to update a cell that has only 1 pattern available):
if H[idN] != 'c':
Then we check all the patterns that could be placed at that location. ex: if the neighboring cell is on the left of the current cell (east side), we look at all the patterns that can be placed on the left of each pattern contained in the current cell.
possible = set([n for idP in W[idC] for n in A[idP][dir]])
We also look at the patterns that are available in the neighboring cell:
available = W[idN]
Now we make sure that the neighboring cell really have to be updated. If all its available patterns are already in the list of all the possible patterns —> there’s no need to update it (the algorithm skip this neighbor and goes on to the next) :
if not available.issubset(possible):
However, if it is not a subset of the possible list —> we look at the intersection of the two sets (all the patterns that can be placed at that location and that, "luckily", are available at that same location):
intersection = possible & available
If they don't intersect (patterns that could have been placed there but are not available) it means we ran into a "contradiction". We have to stop the whole WFC algorithm.
if not intersection:
print 'contradiction'
noLoop()
If, on the contrary, they do intersect --> we update the neighboring cell with that refined list of pattern's indices:
W[idN] = intersection
Because that neighboring cell has been updated, its entropy must be updated as well:
lfreqs = [freqs[i] for i in W[idN]]
H[idN] = (log(sum(lfreqs)) - sum(map(lambda x: x * log(x), lfreqs)) / sum(lfreqs)) - random(.001)
Finally, and most importantly, we add the index of that neighboring cell to the stack so it becomes the next current cell in turns (the one whose neighbors will be updated during the next while loop):
stack.add(idN)
Full updated script
from collections import Counter
from itertools import chain
from random import choice
w, h = 40, 25
N = 3
def setup():
size(w*20, h*20, P2D)
background('#FFFFFF')
frameRate(1000)
noStroke()
global W, A, H, patterns, freqs, npat, mat, xs, ys
img = loadImage('Flowers.png')
iw, ih = img.width, img.height
xs, ys = width//w, height//h
kernel = [[i + n*iw for i in xrange(N)] for n in xrange(N)]
mat = ((-1, 0), (1, 0), (0, -1), (0, 1))
all = []
for y in xrange(ih):
for x in xrange(iw):
cmat = [[img.pixels[((x+n)%iw)+(((a[0]+iw*y)/iw)%ih)*iw] for n in a] for a in kernel]
for r in xrange(4):
cmat = zip(*cmat[::-1])
all.append(cmat)
all.append(cmat[::-1])
all.append([a[::-1] for a in cmat])
all = [tuple(chain.from_iterable(p)) for p in all]
c = Counter(all)
patterns = c.keys()
freqs = c.values()
npat = len(freqs)
W = [set(range(npat)) for i in xrange(w*h)]
A = [[set() for dir in xrange(len(mat))] for i in xrange(npat)]
H = [100 for i in xrange(w*h)]
for i1 in xrange(npat):
for i2 in xrange(npat):
if [n for i, n in enumerate(patterns[i1]) if i%N!=(N-1)] == [n for i, n in enumerate(patterns[i2]) if i%N!=0]:
A[i1][0].add(i2)
A[i2][1].add(i1)
if patterns[i1][:(N*N)-N] == patterns[i2][N:]:
A[i1][2].add(i2)
A[i2][3].add(i1)
def draw():
global H, W
emin = int(random(w*h)) if frameCount <= 1 else H.index(min(H))
if H[emin] == 'c':
print 'finished'
noLoop()
id = choice([idP for idP in W[emin] for i in xrange(freqs[idP])])
W[emin] = [id]
H[emin] = 'c'
stack = set([emin])
while stack:
idC = stack.pop()
for dir, t in enumerate(mat):
x = (idC%w + t[0])%w
y = (idC/w + t[1])%h
idN = x + y * w
if H[idN] != 'c':
possible = set([n for idP in W[idC] for n in A[idP][dir]])
if not W[idN].issubset(possible):
intersection = possible & W[idN]
if not intersection:
print 'contradiction'
noLoop()
return
W[idN] = intersection
lfreqs = [freqs[i] for i in W[idN]]
H[idN] = (log(sum(lfreqs)) - sum(map(lambda x: x * log(x), lfreqs)) / sum(lfreqs)) - random(.001)
stack.add(idN)
fill(patterns[id][0])
rect((emin%w) * xs, (emin/w) * ys, xs, ys)
Overall improvements
In addition to these fixes I also did some minor code optimization to speed-up both the observation and propagation steps, and shorten the weighted choice computation.
The "Wave" is now composed of Python sets of indices whose size decrease as cells are "collapsed" (replacing large fixed size lists of booleans).
Entropies are stored in a defaultdict whose keys are progressively deleted.
The starting entropy value is replaced by a random integer (first entropy calculation not needed since equiprobable high level of uncertainty at start)
Cells are diplayed once (avoiding storing them in a array and redrawing at each frame)
The weighted choice is now a one-liner (avoiding several dispensable lines of list comprehension)

While looking at the live demo linked in one of your examples, and based on a quick review of the original algorithm code, I believe your error lies in the "Propagation" step.
The propagation is not just updating the neighbouring 4 cells to the collapsed cell. You must also update all of those cells neighbours, and then the neighbours to those cells, etc, recursively. Well, to be specific, as soon as you update a single neighbouring cell, you then update its neighbour (before getting to the other neighbours of the first cell), i.e. depth-first, not breadth-first updates. At least, that's what I gather from the live demo.
The actual C# code implementation of the original algorithm is quite complicated and I don't fully understand it, but the key points appear to be creation of the "propagator" object here, as well as the Propagate function itself, here.

Python Hellinger formula explanation

I was looking up some formulas for Hellinger's distance between distributions, and I found one (in Python) that I've never seen similar format for. I am confused how it works.
def hellinger(p,q):
"""Hellinger distance between distributions"""
return sum([(sqrt(t[0])-sqrt(t[1]))*(sqrt(t[0])-sqrt(t[1]))\
for t in zip(p,q)])/sqrt(2.)
I've never seen this kind of... format before. They are dividing by a for statement? I mean.. how does this even work?

I have a faible for distance measures, hence I made a notebook with some implementations of Hellinger distance.
Regarding your question, the construct is called a list comrehension and the backslash is just for line continuation.
Here is a possible listing without list comprehension:
def hellinger_explicit(p, q):
"""Hellinger distance between two discrete distributions.
Same as original version but without list comprehension
"""
list_of_squares = []
for p_i, q_i in zip(p, q):
# caluclate the square of the difference of ith distr elements
s = (math.sqrt(p_i) - math.sqrt(q_i)) ** 2
# append
list_of_squares.append(s)
# calculate sum of squares
sosq = sum(list_of_squares)
return sosq / math.sqrt(2)

python numpy.convolve to solve convolution integral with limits from 0 to t instead -t to t

I have a convolution integral of the type:
To solve this integral numerically, I would like to use numpy.convolve(). Now, as you can see in the online help, the convolution is formally done from -infinity to +infinity meaning that the arrays are moved along each other completely for evaluation - which is not what I need. I obviously need to be sure to pick the correct part of the convolution - can you confirm that this is the right way to do it or alternatively tell me how to do it right and (maybe even more important) why?
res = np.convolve(J_t, dF, mode="full")[:len(dF)]
J_t is an analytical function and I can evaluate as many points as I need, dF are derivatives of measurement data. for this attempt I choose len(J_t) = len(dF) because from my understanding I do not need more.
Thank you for your thoughts, as always, I appreciate your help!
Background information (for those who might be interested)
These type of integrals can be used to evaluate viscoelastic behaviour of bodies (or the response of an electric circuit during change of voltage, if you feel more familiar on this topic). For viscoelasticity, J(t) is the creep compliance function and F(t) can be the deviatoric strains over time, then this integral would yield the deviatoric stresses.
If you now e.g. have a J(t) of the form:
J_t = lambda p, t: p[0] + p[1]*N.exp(-t/p[2])
with p = [J_elastic, J_viscous, tau] this would be the "famous" standard linear solid. The integral limits are the start of the measurement t_0 = 0 and the moment of interest, t.

To get it right, I have chosen the following two functions:
a(t) = t
b(t) = t**2
It is easy to do the math and find that their "convolution" as defined in your case, takes
on the values:
c(t) = t**4 / 12
So lets try them out:
>>> delta = 0.001
>>> t = np.arange(1000) * delta
>>> a = t
>>> b = t**2
>>> c = np.convolve(a, b) * delta
>>> d = t**4 / 12
>>> plt.plot(np.arange(len(c)) * delta, c)
[<matplotlib.lines.Line2D object at 0x00000000025C37B8>]
>>> plt.plot(t[::50], d[::50], 'o')
[<matplotlib.lines.Line2D object at 0x000000000637AB38>]
>>> plt.show()
So by doing the above, if both your a and b have n elements, you get the right convolution values in the first n elements of c.
Not sure if the following explanation will make any sense, but here it goes... If you think of convolution as mirroring one of the functions along the y-axis, then sliding it along the x axis and computing the integral of the product at each point, it is easy to see how, since outside of the area of definition numpy takes them as if padded with zeros, you are effectively setting an integration interval from 0 to t, since the first function is zero below zero, and the second is zero above t, since it originally was zero below zero, but has been mirrored and moved t to the right.

I was tackling this same problem and solved it using a highly inefficient but functionally correct algorithm:
def Jfunk(inz,t):
c0 = inz[0]
c1 = inz[1]
c2 = inz[2]
J = c0 - c1*np.exp(-t/c2)
return J
def SLS_funk(inz, t, dl_dt):
boltz_int = np.empty(shape=(0,))
for i,v in enumerate(t, start=1):
t_int = t[0:i]
Jarg = v - t[0:i]
J_int = Jfunk(inz,Jarg)
dl_dt_int = dl_dt[0:i]
inter_grand = np.multiply(J_int, dl_dt_int)
boltz_int = np.append(boltz_int, simps (inter_grand, x=t_int) )
return boltz_int
Thanks to this question and its answers, I was able to implement a much better solution based on the numpy convolution function suggested above. In case the OP was curious I did a time comparison of the two methods.
For an SLS (three parameter J function) with 20,000 time points:
Using Numpy convolution: ~0.1 seconds
Using Brute Force method: ~7.2 seconds

If if helps to get a feeling for the alignment, try convolving a pair of impulses. With matplotlib (using ipython --pylab):
In [1]: a = numpy.zeros(20)
In [2]: b = numpy.zeros(20)
In [3]: a[0] = 1
In [4]: b[0] = 1
In [5]: c = numpy.convolve(a, b, mode='full')
In [6]: plot(c)
You can see from the resultant plot that the first sample in c corresponds to the first position of overlap. In this case, only the first samples of a and b overlap. All the rest are floating in undefined space. numpy.convolve effectively replaces this undefined space with zeros, which you can see if you set a second non-zero value:
In [9]: b[1] = 1
In [10]: plot(numpy.convolve(a, b, mode='full'))
In this case, the first value of the plot is 1, as before (showing that the second value of b is not contributing at all).

I have been struggling with similar question for past 2 days.
The OP may have moved on, but I am still presenting my analysis here.
Following two sources helped me:
Discussion on stackoverflow
These notes
I will consider time-series data defined on the same time series starting from time .
Let the two series be A and B.
Their (continuous) convolution is
Substituting with in the above equation we get what np.convolve(A,B) returns:
What you want is
Again making the same substitution, we get
which is same as above because A for negative indices is extrapolated to zero and for i > (j + m) B[j - i + m] is zero.
If you look at the notes cited above, you can figure out that corresponds to time for our time series.
The next value in the list will correspond to and so on.
Therefore, the correct answer will be
is equal to np.convolve(A,B)[0:M], where M = len(A) = len(B).
Here keep in mind that M*dt = T, where T is the last element of time array.
Disclaimer: I am not a programmer, mathematician or an engineer. I had to use convolution somewhere and have derived these conclusions from my own struggle with the problem. I will be happy to cite any book which has this analysis if someone can point it out.

Interpolation of sin(x) using Python

I am working on a homework problem for which I am supposed to make a function that interpolates sin(x) for n+1 interpolation points and compares the interpolation to the actual values of sin at those points. The problem statement asks for a function Lagrangian(x,points) that accomplishes this, although my current attempt at executing it does not use 'x' and 'points' in the loops, so I think I will have to try again (especially since my code doesn't work as is!) However, why I can't I access the items in the x_n array with an index, like x_n[k]? Additionally, is there a way to only access the 'x' values in the points array and loop over those for L_x? Finally, I think my 'error' definition is wrong, since it should also be an array of values. Is it necessary to make another for loop to compare each value in the 'error' array to 'max_error'? This is my code right now (we are executing in a GUI our professor made, so I think some of the commands are unique to that such as messages.write()):
def problem_6_run(problem_6_n, problem_6_m, plot, messages, **kwargs):
n = problem_6_n.value
m = problem_6_m.value
messages.write('\n=== PROBLEM 6 ==========================\n')
x_n = np.linspace(0,2*math.pi,n+1)
y_n = np.sin(x_n)
points = np.column_stack((x_n,y_n))
i = 0
k = 1
L_x = 1.0
def Lagrange(x, points):
for i in n+1:
for k in n+1:
return L_x = (x- x_n[k] / x_n[i] - x_n[k])
return Lagrange = y_n[i] * L_x
error = np.sin(x) - Lagrange
max_error = 0
if error > max_error:
max_error = error
print.messages('Maximum error = &g' % max_error)
plot.draw_lines(n+1,np.sin(x))
plot.draw_points(m,Lagrange)
plots.draw_points(m,error)
Edited:
Yes, the different things ThiefMaster mentioned are part of my (non CS) professor's environment; and yes, voithos, I'm using numpy and at this point have definitely had more practice with Matlab than Python (I guess that's obvious!). n and m are values entered by the user in the GUI; n+1 is the number of interpolation points and m is the number of points you plot against later.
Pseudocode:
Given n and m
Generate x_n a list of n evenly spaced points from 0 to 2*pi
Generate y_n a corresponding list of points for sin(x_n)
Define points, a 2D array consisting of these ordered pairs
Define Lagrange, a function of x and points
for each value in the range n+1 (this is where I would like to use points but don't know how to access those values appropriately)
evaluate y_n * (x - x_n[later index] / x_n[earlier index] - x_n[later index])
Calculate max error
Calculate error interpolation Lagrange - sin(x)
plot sin(x); plot Lagrange; plot error
Does that make sense?

Some suggestions:
You can access items in x_n via x_n[k] (to answer your question).
Your loops for i in n+1: and for k in n+1: only execute once each, one with i=n+1 and one with k=n+1. You need to use for i in range(n+1) (or xrange) to get the whole list of values [0,1,2,...,n].
in error = np.sin(x) - Lagrange: You haven't defined x anywhere, so this will probably result in an error. Did you mean for this to be within the Lagrange function? Also, you're subtracting a function (Lagrange) from a number np.sin(x), which isn't going to end well.
When you use the return statement in your def Lagrange you are exiting your function. So your loop will never loop more than once because you're returning out of the function. I think you might actually want to store those values instead of returning them.
Can you write some pseudocode to show what you'd like to do? e.g.:
Given a set of points `xs` and "interpolated" points `ys`:
For each point (x,y) in (xs,ys):
Calculate `sin(x)`
Calculate `sin(x)-y` being the difference between the function and y
.... etc etc
This will make the actual code easier for you to write, and easier for us to help you with (especially if you intellectually understand what you're trying to do, and the only problem is with converting that into python).
So : try fix up some of these points in your code, and try write some pseudocode to say what you want to do, and we'll keep helping you :)

How do I check if cartesian coordinates make up a rectangle efficiently?

The situation is as follows:
There are N arrays.
In each array (0..N-1) there are (x,y) tuples (cartesian coordinates) stored
The length of each array can be different
I want to extract the subset of coordinate combinations which make up a complete
retangle of size N. In other words; all the cartesian coordinates are adjacent to each other.
Example:
findRectangles({
{*(1,1), (3,5), (6,9)},
{(9,4), *(2,2), (5,5)},
{(5,1)},
{*(1,2), (3,6)},
{*(2,1), (3,3)}
})
yields the following:
[(1,1),(1,2),(2,1),(2,2)],
...,
...(other solutions)...
No two points can come from the same set.
I first just calculated the cartesian product, but this quickly becomes infeasible (my use-case at the moment has 18 arrays of points with each array roughly containing 10 different coordinates).

You can use hashing to great effect:
hash each point (keeping track of which list it is in)
for each pair of points (a,b) and (c,d):
if (a,d) exists in another list, and (c,b) exists in yet another list:
yield rectangle(...)
When I say exists, I mean do something like:
hashesToPoints = {}
for p in points:
hashesToPoints.setdefault(hash(p),set()).add(p)
for p1 in points:
for p2 in points:
p3,p4 = mixCoordinates(p1,p2)
if p3 in hashesToPoints[hash(p3)] and {{p3 doesn't share a bin with p1,p2}}:
if p4 in hashesToPoints[hash(p4)] and {{p4 doesn't share a bin with p1,p2,p3}}:
yield Rectangle(p1,p2)
This is O(#bins^2 * items_per_bin^2)~30000, which is downright speedy in your case of 18 arrays and 10 items_per_bin -- much better than the outer product approach which is... much worse with O(items_per_bin^#bins)~3trillion. =)
minor sidenote:
You can reduce both the base and exponent in your computation by making multiple passes of "pruning". e.g.
remove each point that is not corectilinear with another point in the X or Y direction
then maybe remove each point that is not corectilinear with 2 other points, in both X and Y direction
You can do this by sorting according to the X-coordinate, repeat for the Y-coordinate, in O(P log(P)) time in terms of number of points. You may be able to do this at the same time as the hashing too. If a bad guy is arranging your input, he can make this optimization not work at all. But depending on your distribution you may see significant speedup.

Let XY be your set of arrays. Construct two new sets X and Y, where X equals XY with all arrays sorted to x-coordinate and Y equals XY with all arrays sorted to y-coordinate.
For each point (x0,y0) in any of the arrays in X: find every point (x0,y1) with the same x-coordinate and a different y-coordinate in the remaining arrays from X
For each such pair of points (if it exists): search Y for points (x1,y0) and (x1,y1)
Let C be the size of the largest array. Then sorting all sets takes time O(N*C*log(C)). In step 1, finding a single matching point takes time O(N*log(C)) since all arrays in X are sorted. Finding all such points is in O(C*N), since there are at most C*N points overall. Step 2 takes time O(N*log(C)) since Y is sorted.
Hence, the overall asymptotic runtime is in O(C * N^2 * log(C)^2).
For C==10 and N==18, you'll get roughly 10.000 operations. Multiply that by 2, since I dropped that factor due to Big-O-notation.
The solution has the further benefit of being extremely simple to implement. All you need is arrays, sorting and binary search, the first two of which very likely being built into the language already, and binary search being extremely simple.
Also note that this is the runtime in the worst case where all rectangles start at the same x-coordinate. In the average case, you'll probably do much better than this.