We Keep Coding

Sequentially take a window of rows from array (python)

I have an array of size nxm and want to take the first 10 rows and perform calculations, then take the next 10 rows perform calculations, etc. But this is hard coded, how can I make a loop?
Code Attempted:
import numpy as np
total = []
x= np.random.random((100,4))
a = np.average(x[:10])
total.append(a)
a = np.average(x[10:20])
total.append(a)
a = np.average(x[20:30])
....
Goal:
for *this array*:
# to something
# append value
# go back and get next 10 values

It looks like you want the following.
import numpy as np
x = np.random.random((100,4))
L = 10
k = 100//L
total = [np.average(x[L*i:L*(i+1)]) for i in range(k)]
If you'd rather implement this using a loop rather than list comprehension,
import numpy as np
x = np.random.random((100,4))
L = 10
k = 100//L
total = []
for i in range(k):
total.append(np.average(x[L*i:L*(i+1)]))
As an alternative, here's an approach using a 3-dimensional reshape.
x= np.random.random((100,4))
L = 10 #window length
n = x.shape[1] #number of columns
total = a.reshape(-1,10,n).mean(axis = (1,2))

import numpy as np
x = np.random.random((100,4))
a = 10
b = 100//a
c = 4
You want the array of average numbers of the first 10 * 4 part, the second 10 * 4 part,..., right?
reshape function can be really useful here.
x_splited = x.reshape((-1, a*c))
total = x_splited.mean(axis=1)
This is the answer you need. The reshape function let the first a*c elements in the original matrix become the first row of the new matrix. Then, mean(axis=1) help you get the average of the first row.
Also, you could try something like this:
x_splited = x.reshape((-1, a, c))
You can do something more complicated than this question with it.
Just a tip: in python, it is prefered to avoid using loop because it is slow.
Second tip: if you are still not proficient in using loop in Python, you are encouraged to spend some time to practice it.

When creating a function with summation in it, how do you add terms to the summation? (Python)

I need to find a way to add new terms to the function:
f(x) = 4sin(x)/pi
such that the new function looks like this:
f(x) = 4sin(x)/pi + 4sin(3x)/3pi + 4sin(5x)/5pi + 4sin(7x)/7pi ... up to 201.
I was thinking I could create the function with a for loop inside that would iterate using a list n but there must be a way to do it. I am using the math and numpy libraries. This is what I have so far:
# original function f(x)
def f(x):
return (4/math.pi)*math.sin(x)
# create list (n) composed of odd numbers up to 201
n = list(range(1, 202, 2))
# new function with added terms
def newf(x):
for i in n:
sum (4/n[i]*math.pi)*math.sin(n[i]*x)
I am still relatively new to Python and cannot find a similar example to troubleshoot with. Am I thinking through this correctly? I'm unsure why this code would not work.
I do not know how to do math notation in Stack Overflow, so I apologize for that!

You are close to the solution! The reason is not working is because inside your loop nothing is retaining the computation of the previous iterations.
And you can simply use i without accessing n[i], which eventually will throw you an IndexError: list index out of range
The code should provide you what you need:
# original function f(x)
import math
def f(x):
return (4/math.pi)*math.sin(x)
# create list (n) composed of odd numbers up to 201
n = list(range(1, 202, 2))
def new_f(x):
return sum([(4/i*math.pi)*math.sin(i*x) for i in n])

You could use list comprehension like so:
def f(x):
return sum([4 * math.sin(i * x) / (i * math.pi) for i in range(1,202,2)])

I think you can do it in numpy. Create a function like this:
def func(x,i):
return 4*np.sin(i*x)/(i*np.pi)
Then create a numpy array:
l = np.arange(1,202,2)
Then just apply for the disered x:
res = func(x,l).sum()

Do you want the result o just the expression?
Here is a way to get the result:
import math
def f(x, i):
value = 0.0
for i in range(1, i*2+1, 2):
value += (4/i*math.pi)*math.sin(x*i)
return value
print(f(math.pi, 201))
With i you choose how many terms do you want

Speed of different data structures in Python

I have a piece of code I need to run, however, it is taking so long. I estimate it will take a minimum of 2 hours to run and at most 100 hours. As you can see, I would like to speed this up.
My code is as follows:
#B1
file = open("/home/1015097/Downloads/B1/tour5.txt", "r").readlines()
# This is a very large file. You can find it on http://codemasters.eng.unimelb.edu.au/bit_problems/B1.zip
# It is named tour5.txt, you can also find the question I am trying to solve there.
racers = [int(file[0].split()[0]) / float(x) for x in file[0].split()[1::]]
print("hi")
def av(race):
race = race.split()
j = 0
while j != len([float(race[0]) / float(x) for x in race[1::]]):
racers[j] = [float(race[0]) / float(x) for x in race[1::]][j] + racers[j]
j += 1
print(j)
print("yay...")
for i in range(1, len(file)):
print("yay")
av(file[i])
print("yaaay")
a = min(racers)
del(racers[racers.index(min(racers))])
b = min(racers)
c = b-a
h = int(c)
c-=h
m = int(c * 60)
c-=m/60
s = round(c * 60 * 60)
print(str(h) + "h" + str(m) + "m" + str(s) + "s")
We are currently coming first in Australia for the Code Bits contest and would not like to drop our perfect score. The random print statements were so that we could tell if the code was actually running, they are essentially checkpoints. the number that is printed out is the racer number, there are at least 3000 racers, we do not know exactly.

I would start with changing:
while j != len([float(race[0]) / float(x) for x in race[1::]]):
racers[j] = [float(race[0]) / float(x) for x in race[1::]][j] + racers[j]
into
while j != len(race) - 1:
racers[j] += float(race[0]) / float(race[j])

Avoid loops like the plague. Vectorize everything. Use numpy, etc. If you have to, even look into Cython. But most importantly, vectorize.
The av() function is probably the part where your code is taking the most time (btw, it would be a good idea to profile the code at various points, figure out the most taxing process and focus on vectorizing it). Also, try to minimize the number of initializations. If you can, create an object only once and for all.
Below is how I would change up the function.
import numpy as np
racers = np.array(racers)
def av(race, racers):
race = race.split()
race_float = np.array(len([float(race[0]) / float(x) for x in race[1::]]))
racers += race_float
return racers
Also, please refrain from:
Using print for debugging. You have a built-in logging module. Use it.
Using globals. Just pass them into functions as arguments and return the new object instead of directly modifying a global object.

I think you should be looking at numpy arrays instead of lists. Thereby you can avoid the loops and obtain close to c -speed. This problem is easily applicable to that. And by the way why not store anything in float64 or float32 so there is no conversion of datatypes. Code example not fully portable. It is a schoolwork and I should not do it for you:
import numpy as np
racer=np.array(racer) # Will work for 1 d lists and for 2-d lists with same lenght
racer_time=racer/time # diving a vector by a scalar is easy

Append keeps on appending the same item, does not append the right ones, Python

This is what I have imported:
import random
import matplotlib.pyplot as plt
from math import log, e, ceil, floor
import numpy as np
from numpy import arange,array
import pdb
from random import randint
Here I define the function matrix(p,m)
def matrix(p,m): # A matrix with zeros everywhere, except in every entry in the middle of the row
v = [0]*m
v[(m+1)/2 - 1] = 1
vv = array([v,]*p)
return vv
ct = np.zeros(5) # Here, I choose 5 cause I wanted to work with an example, but should be p in general
Here I define MHops which basically takes the dimensions of the matrix, the matrix and the vector ct and gives me a new matrix mm and a new vector ct
def MHops(p,m,mm,ct):
k = 0
while k < p : # This 'spans' the rows
i = 0
while i < m : # This 'spans' the columns
if mm[k][i] == 0 :
i+=1
else:
R = random.random()
t = -log(1-R,e) # Calculate time of the hopping
ct[k] = ct[k] + t
r = random.random()
if 0 <= r < 0.5 : # particle hops right
if 0 <= i < m-1:
mm[k][i] = 0
mm[k][i+1] = 1
break
else:
break # Because it is at the boundary
else: # particle hops left
if 0 < i <=m-1:
mm[k][i] = 0
mm[k][i-1] = 1
break
else: # Because it is at the boundary
break
break
k+=1
return (mm,ct) # Gives me the new matrix showing the new position of the particles and a new vector of times, showing the times taken by each particle to hop
Now what I wanna do is iterating this process, but I wanna be able to visualize every step in a list. In short what I am doing is:
1. creating a matrix representing a lattice, where 0 means there is no particle in that slot and 1 means there is a particle there.
2. create a function MHops which simulate a random walk of one step and gives me the new matrix and a vector ct which shows the times at which the particles move.
Now I want to have a vector or an array where I have 2*n objects, i.e. the matrix mm and the vector ct for n iterations. I want the in a array, list or something like this cause I need to use them later on.
Here starts my problem:
I create an empty list, I use append to append items at every iteration of the while loop. However the result that I get is a list d with n equal objects coming from the last iteration!
Hence my function for the iteration is the following:
def rep_MHops(n,p,m,mm,ct):
mat = mm
cct = ct
d = []
i = 0
while i < n :
y = MHops(p,m,mat,cct) # Calculate the hop, so y is a tuple y = (mm,ct)
mat = y[0] # I reset mat and cct so that for the next iteration, I go further
cct = y[1]
d.append(mat)
d.append(cct)
i+=1
return d
z = rep_MHops(3,5,5,matrix(5,5),ct) #If you check this, it doesn't work
print z
However it doesn't work, I don't understand why. What I am doing is using MHops, then I want to set the new matrix and the new vector as those in the output of MHops and doing this again. However if you run this code, you will see that v works, i.e. the vector of the times increases and the matrix of the lattice change, however when I append this to d, d is basically a list of n equal objects, where the object are the last iteration.
What is my mistake?
Furthermore if you have any coding advice for this code, they would be more than welcome, I am not sure this is an efficient way.
Just to let you understand better, I would like to use the final vector d in another function where first of all I pick a random time T, then I would basically check every odd entry (every ct) and hence check every entry of every ct and see if these numbers are less than or equal to T. If this happens, then the movement of the particle happened, otherwise it didn't.
From this then I will try to visualize with matpotlibt the result with an histogram or something similar.
Is there anyone who knows how to run this kind of simulation in matlab? Do you think it would be easier?

You're passing and storing by references not copies, so on the next iteration of your loop MHops alters your previously stored version in d. Use import copy; d.append(copy.deepcopy(mat)) to instead store a copy which won't be altered later.
Why?
Python is passing the list by reference, and every loop you're storing a reference to the same matrix object in d.
I had a look through python docs, and the only mention I can find is
"how do i write a function with output parameters (call by reference)".
Here's a simpler example of your code:
def rep_MHops(mat_init):
mat = mat_init
d = []
for i in range(5):
mat = MHops(mat)
d.append(mat)
return d
def MHops(mat):
mat[0] += 1
return mat
mat_init = [10]
z = rep_MHops(mat_init)
print(z)
When run gives:
[[15], [15], [15], [15], [15]]
Python only passes mutable objects (such as lists) by reference. An integer isn't a mutable object, here's a slightly modified version of the above example which operates on a single integer:
def rep_MHops_simple(mat_init):
mat = mat_init
d = []
for i in range(5):
mat = MHops_simple(mat)
d.append(mat)
return d
def MHops_simple(mat):
mat += 1
return mat
z = rep_MHops_simple(mat_init=10)
print(z)
When run gives:
[11, 12, 13, 14, 15]
which is the behaviour you were expecting.
This SO answer How do I pass a variable by reference? explains it very well.

Equation to give a number of outputs based on change of one paramater

Here is my equation:
import math
import pandas as pd
import numpy as np
ha = 8.14
k = 0.0187
Do = 0.1738
Di = 0.0138
L = 3
F = 20
Ta = 293
Ts = 113
pi = 3.14159265
Q = (pi*(Ta-Ts))/(((1/ha*Do))+(1/(2*k))*math.log(Do/Di)) * L
h = (Q*3600)/F
Basically, I want the outputs for when F = np.arange(20,100,10)
is this possible?
I tried
a = np.arange(20,100,10)
F = 20 + i
for i in range(a):
print h
not sure why this doesn't work?
any ideas?

When you use numpy you should try to not use explicit loops, as numpy's main virtue is to execute implicitly the loops with the speed of a compiled language.
To come to our problem, it is simply
F = np.arange(20,100,10)
h = Q*3600/F
print h
where I use the array F just as a normal Python variable, and it's up to numpy magics to recognize the need for a loop.
Remember: you can use numpy's arrays in your expressions just like you would use a scalar and, as far as your usage is a sensible one, Python will compute an array-zed result.
Side Note use np.pi instead of 3.14159...

You need to define a function that takes in parameters. then you can call in that function based on a set of parameters. Bellow I show you an example with a sin(x) function because I'm sure you can figure out hwo to adapt it to your needs
import math
points = np.arange(20, 100, 10)
def sine(x, a=1.0, b=2.0):
return a*math.sin(b*x)
for i in points:
print(sine(i,b=i), end=" ")
-0.8509193596391765 0.9978032744219705 -0.8012247906768953 -0.6501275235748956 -0.2620839590180966 -0.7736233386803075 -0.5444763096196569 0.8272184413093554
this will call your sin function with various parameter b and will calculate it in different points x
The way you're doing it should always print out exactly the same number. Since you've already calculated everything. You don't have a function anywhere, and by putting it all in a for loop won't automagically change anything.
Unless it's some for of pandas magic in interpreter I don't know about. (In that case sorry, I'm not a big user of pandas)

Currently your code is giving Q and h a specific value, and never changes that value again.
Explanation:
In the code below
Q = 4
W = Q
print(Q)
print(W)
you are saying: name Q refers to value 4 and name W refers to the current value of Q(that is 4, regardless whether Q changes later on or not).
If you type Q = 10, W will not change:
Q = 10
print(Q)
print(W)
Therefore you need functions that will be calculated each time you call them:
def Q():
return (pi*(Ta-Ts))/(((1/ha*Do))+(1/(2*k))*math.log(Do/Di)) * L
def h():
return (Q()*3600)/F
for i in np.arange(20,100,10):
F = 20 + i
print h()
The above explanation is a bit oversimplified, so you might want to check this link.