I am trying to implement relaxation iterative solver for a project. The function we create should intake two inputs: Matrix A, and Vector B, and should return iterative vectors X that Approximate solution Ax = b.
Pseudo Code from the book is here:
enter image description here
I am new to Python so I am struggling quite a bit with implementing this method. Here is my code:
def SOR_1(A,b):
k=1
n = len(A)
xo = np.zeros_like(b)
x = np.zeros_like(b)
omega = 1.25
while (k <= N):
for i in range(n-1):
x[i] = (1.0-omega)*xo[i] + (1.0/A[i][i])[omega(-np.sum(A[i][j]*x[j]))
-np.sum(A[i][j]*xo[j] + b[i])]
if ( np.linalg.norm(x - xo) < 1e-9):
print (x)
k = k + 1.0
for i in range(n-1):
xo[i] = x[i]
return x
My question is how do I implement the for loop and generating the arrays correctly based off of the Pseudo Code.
Welcome to Python!
Variables in Python are case sensitive so n is defined but N is not defined. If they are supposed to be different variables, I don't see what your value is for N.
You are off to a good start but the following line is still psuedocode for the most part:
x[i] = (1.0-omega)*xo[i] + (1.0/A[i][i])[omega(-np.sum(A[i][j]*x[j]))
-np.sum(A[i][j]*xo[j] + b[i])]
In the textbook's pseudocode square brackets are being used as a grouping symbol but in Python, they are reserved for creating and accessing lists (which is what python calls arrays). Also, there is no implicit multiplication in Python so you have to write things like (1 + 2)*(3*(4+5)) rather than (1 + 2)[3(4+5)]
The other major issue is that you don't define j. You probably need a for loop that would either look like:
for j in range(1, i):
or if you want to do it inline
sum(A[i][j]*x[j] for j in range(1, i))
Note that range has two arguments, where to start and what value to stop before so range(1, i) is equivalent to the summation from 1 to i - 1
I think you are struggling with that line because there's far too much going on in that line. See if you can figure out parts of it using separate variables or offload some of the work to separate functions.
something like: x[i] =a + b * c * d() - e() but give a,b c, d and e meaningful names. You'd then have to correctly set each variable and define each function but at least you are trying to solve separate problems rather than one huge complex one.
Also, make sure you have your tabs correct. k = k + 1.0 should not be inside that for loop, just inside the while loop.
Coding is an iterative process. First get the while loop working. Don't try to do anything in it (except print out the variable so you can see that it is working). Next get the for loop working inside the while loop (again, just printing the variables). Next get (1.0-omega)*xo[i] working. Along the way, you'll discover and solve issues such as (1.0-omega)*xo[i] will evaluate to 0 because xo is a NumPy list initiated with all zeros.
You'd start with something like:
k = 1
N = 3
n = 3
xo = [1, 2, 3]
while (k <= N):
for i in range(n):
print(k, i)
omega = 1.25
print((1.0-omega)*xo[i])
k += 1
And slowly work more and more of the relaxation solver in until you have everything working.
My code is as follows.
I tried coding out for each case first, so given n = 4, my code looks like this:
a = overlay_frac(0,blank_bb,scale(1/4,rune))
b = overlay_frac(1/4,blank_bb,scale(1/2,rune))
c = overlay_frac(1/2,blank_bb,scale(3/4,rune))
d = overlay_frac(3/4,blank_bb,scale(1,rune))
show (overlay(a,(overlay(b,(overlay(c,d))))))
My understanding is that the recursion pattern is:
a = overlay_frac((1/n)-(1/n),blank_bb,scale(1/n,rune))
b = overlay_frac((2/n)-(1/n),blank_bb,scale(2/n,rune))
c = overlay_frac((3/n)-(1/n),blank_bb,scale(3/n,rune))
d = overlay_frac((4/n)-(1/n),blank_bb,sale(4/n,rune))
Hence, the recursion pattern that I came up with is:
def tree(n,rune):
if n==1:
return rune
else:
for i in range(n+1):
return overlay(overlay_frac(1-(1/n),blank_bb,scale(i/n,rune)),tree(n-1,rune))
When I hardcode this, everything turns out just fine, but I suspect I'm not doing the recursion properly. Where have I gone wrong?
You are in fact trying to do an iteration within a recursive call. In stead of using loop, you can use an inner function to memorize your status. The coefficient you defined is actually changed with both n and i, but for a given n it changed with i only. The status you need to memorize with inner function is then i, which is the same as you looping through i.
You can still achieve your goal by doing so
def f(i, n):
return overlay_frac((i/n)-(1/n),blank_bb,scale(i/n,rune))
# for each iteration, you check if i is equal to n
# if yes, return the result (base case)
# otherwise, you apply next coefficient to the previous result
# you start with i = 0 and increase by one every iteration until i reach to n (base case)
# notice how similar this recursive call looks like a loop
# the only difference is the status are updated within the function call itself
# therefore you will not have the problem of earlier return
def recursion(n):
def iteration(i, out):
if i == n:
return out
else:
return iteration(i+1, overlay(f(n-1, n), out))
return iteration(0, f(n, n))
Here, n is assumed to be the times of overlay you want to apply. When n = 0, no function applied on the last coefficient f(n, n). When n = 1, the output would be overlay applied once on coefficient with i = n - 1 and coefficient with i = n.
This way avoids the earlier return inside your loop.
In fact you can omit the inner function by adding additional argument to your outer function. Then you need to assign the default initial i. The inner function is not really necessary here. The key is to use the function argument to memorize the status (variable i in this case).
def f(i, n):
return overlay_frac((i/n)-(1/n),blank_bb,scale(i/n,rune))
def recursion(n, i=0):
if i == n:
return f(n, n)
else:
return overlay(f(n-1, n), recursion(n, i+1))
Your first two code blocks don't correspond to the same operations. This would be equivalent to your first block (in Python 3).
def overlayer(n, rune):
def layer(k):
# Scale decreases linearly with k
return overlay_frac((1 - (k+1)/n), blank_bb, scale(1-k/n, rune))
result = layer(0)
for i in range(1, n):
# Overlay on top of previous layers
result = overlay(layer(i), result)
return result
show(overlayer(4, rune))
Let's look at your equations again:
a = overlay_frac(0,blank_bb,scale(1/4,rune))
b = overlay_frac(1/4,blank_bb,scale(1/2,rune))
c = overlay_frac(1/2,blank_bb,scale(3/4,rune))
d = overlay_frac(3/4,blank_bb,scale(1,rune))
show (overlay(a,(overlay(b,(overlay(c,d))))))
What you wrote as "recursion" is not a recursion formula. If you compare your formulas for the recursion with the ones you gave us, you can infer n=4 which makes no sense. For a recursion pattern you need to describe your inner variables as a manifestation of the same expression with only a different parameter. That is, you should replace:
f_n = overlay_frac((1/4)*(n-1),blank_bb,sale(n/4,rune))
such that f_1=a, f_2=b etc...
Then your recursion fomula that you want to calculate translates to:
show (overlay(f_1,(overlay(f_2,(overlay(f_3,f_4))))))
You can write the function f_n as f(n) (and maybe other paramters) in your code and then do
def recurse(n):
if n == 4:
return f(4)
else:
return overlay(f(n),recurse(n+1))
then call:
show( recurse (1))
You need to assert that n<5and integer, otherwise you'll end up in an infinity loop.
There may still be some mistake, but it should be along those lines. Once you've actually written it like this however, it (maybe) doesn't really make sense to do a recursion anyways. If you only want to do it for n_max=4, that is. Just call the function in one line by replacing a,b,c,d with f_1,f_2,f_3,f_4
What costs more in python, making a list with comprehension or with a standalone function?
It appears I failed to find previous posts asking the same question. While other answers go into detail about bytes and internal workings of python, and that Is indeed helpful, I felt like the visual graphs help to show that there is a continuous trend.
I don't yet have a good enough understanding of the low level workings of python, so those answers are a bit foreign for me to try and comprehend.
I am currently an undergrad in CS, and I am continually amazed with how powerful python is. I recently did a small experiment to test the cost of forming lists with comprehension versus a standalone function. For example:
def make_list_of_size(n):
retList = []
for i in range(n):
retList.append(0)
return retList
creates a list of size n containing zeros.
It is well known that this function is O(n). I wanted to explore the growth of the following:
def comprehension(n):
return [0 for i in range(n)]
Which makes the same list.
let us explore!
This is the code I used for timing, and note the order of function calls (which way did I make the list first). I made the list with a standalone function first, and then with comprehension. I have yet to learn how to turn off garbage collection for this experiment, so, there is some inherent measurement error, brought about when garbage collection kicks in.
'''
file: listComp.py
purpose: to test the cost of making a list with comprehension
versus a standalone function
'''
import time as T
def get_overhead(n):
tic = T.time()
for i in range(n):
pass
toc = T.time()
return toc - tic
def make_list_of_size(n):
aList = [] #<-- O(1)
for i in range(n): #<-- O(n)
aList.append(n) #<-- O(1)
return aList #<-- O(1)
def comprehension(n):
return [n for i in range(n)] #<-- O(?)
def do_test(size_i,size_f,niter,file):
delta = 100
size = size_i
while size <= size_f:
overhead = get_overhead(niter)
reg_tic = T.time()
for i in range(niter):
reg_list = make_list_of_size(size)
reg_toc = T.time()
comp_tic = T.time()
for i in range(niter):
comp_list = comprehension(size)
comp_toc = T.time()
#--------------------
reg_cost_per_iter = (reg_toc - reg_tic - overhead)/niter
comp_cost_pet_iter = (comp_toc - comp_tic - overhead)/niter
file.write(str(size)+","+str(reg_cost_per_iter)+
","+str(comp_cost_pet_iter)+"\n")
print("SIZE: "+str(size)+ " REG_COST = "+str(reg_cost_per_iter)+
" COMP_COST = "+str(comp_cost_pet_iter))
if size == 10*delta:
delta *= 10
size += delta
def main():
fname = input()
file = open(fname,'w')
do_test(100,1000000,2500,file)
file.close()
main()
I did three tests. Two of them were up to list size 100000, the third was up to 1*10^6
See Plots:
Overlay with NO ZOOM
I found these results to be intriguing. Although both methods have a big-O notation of O(n), the cost, with respect to time, is less for comprehension for making the same list.
I have more information to share, including the same test done with the list made with comprehension first, and then with the standalone function.
I have yet to run a test without garbage collection.
This simple piece of code works perfectly fine. What I'm about to ask is completely unnecessary; however, I'm trying to learn more about the functional programming approach to doing things.
p=[0, 1, 0, 0, 0]
pHit = 0.6
pMiss = 0.2
pExact = 0.8
pOvershoot = 0.1
pUndershoot = 0.1
def move(p, U):
q = []
# A functional approach could be used here as well, but focusing on the outer loop at the moment.
for i in range(len(p)):
s = pExact * p[(i-U) % len(p)]
s = s + pOvershoot * p[(i-U-1) % len(p)]
s = s + pUndershoot * p[(i-U+1) % len(p)]
q.append(s)
return q
#Instead of this for loop here, is there a more functional approach to doing the same thing?
for i in range(0,1000):
p = move(p,1)
print move(p,1)
This person asked a similar question, but the difference is that he/she is applying the recursive function to the actual object being iterated over.
Using recursion with map in python
My case seems different because I am not iterating over the object (the list "p") that I am applying the recursive function to. The "for loop" handles this pretty well because I want to do the recursive operation range(0,1000) times, but I've seen this issue come up a few times now, and I'm very interested in seeing the functional programming solution to this problem.
I've tried to use reduce() a few times, but I find it difficult to pass the output of the X iteration to the X+1 iteration.
To replace that loop at the bottom, you could do something like:
reduce(lambda q,_: move(q, 1), range(1000), p)
Notice how the values of the range are never even used, so they're indicated as being irrelevant using a _.
The reduction automatically passes the result of move to the next iteration.
I have an undirected graph with 1034 vertices and 53498 edges. I'm computing the preferential attachment index for the vertices. The Preferential Attachment similarity between two vertices is defined as the multiplication of the degree of the first vertex times the degree of the second vertex. I noticed that my computations are very slow. It took 2.7 minutes to compute that for the mentioned graph. I'm not sure if it's my algorithm that is slow or something else is wrong. I would be very thankful if someone could have a little look into my code.
Edit: I just realized that S is a 1034_by_1034 matrix. Looking at the nested for-loops it seems that it's a O(n^2) algorithm! I guess that is why it's slow. Don't you agree?
def pa(graph):
"""
Calculates Preferential Attachment index.
Returns S the similarity matrix.
"""
A = gts.adjacency(graph)
S = np.zeros(A.shape)
for i in xrange(S.shape[0]):
for j in xrange(S.shape[0]):
i_degree = graph.vertex(i).out_degree()
j_degree = graph.vertex(j).out_degree()
factor = i_degree * j_degree
S[i,j] = factor
return S
With all i know about it, these are the speedups i can suggest:
zeroth speedup: the i_degree is not depending on j, so move it up one level
def pa(graph):
A = gts.adjacency(graph)
S = np.zeros(A.shape)
for i in xrange(S.shape[0]):
i_degree = graph.vertex(i).out_degree() # easy to see that this can be put here instead, since it does not depend on j
for j in xrange(S.shape[0]):
j_degree = graph.vertex(j).out_degree()
factor = i_degree * j_degree
S[i,j] = factor
return S
first speedup: calling out_degree() only N times, instead of 2N^2.
def pa2(graph):
A = gts.adjacency(graph)
i_degree = numpy.zeros(A.shape[0])
for i in xrange(A.shape[0]):
i_degree[i] = graph.vertex(i).out_degree()
S = np.zeros(A.shape)
for i in xrange(S.shape[0]):
for j in xrange(S.shape[0]):
S[i,j] = i_degree[i]*i_degree[j]
return S
Second speedup: numpy instead of python for-loop
def pa3(graph):
A = gts.adjacency(graph)
i_degree = numpy.zeros(A.shape[0])
for i in xrange(A.shape[0]):
i_degree[i] = graph.vertex(i).out_degree()
S = i_degree[:,None]*i_degree[None,:]
return S
This abuses the symmetry of your problem.
Note: The [None,:] does the same as using [numpy.newaxis,:]. If you wanted to keep your code, you could also use an #memoize decorator on that out_degree() method, but it is better to use that only on stuff that is recursive, and this is not one of those cases.