We Keep Coding

Assign values to different index positions as the actual in a loop python

I have the next variables which are List, floats and a numpy array.
dt=list(range(1,12))
c=18
limit=2.75
Energy=np.zeros(len(dt))
I want to assign the value c=18 in the Numpy array Energy. However, there is a condition. The value in the Energy vector can not be greater than limit=2.75, so as c=18 is greater than limit=2.75, it should be cut to 2.5 and assigned in the actual index position of the loop and in the next index positions of the vector Energy until the value 18 is reached. I made this code but it does not really work efficiently.
for i in range(0,1):
if c>limit:
tmp2=c-(limit)
if tmp2>(limit):
tmp3=tmp2-(limit)
if tmp3>limit:
tmp4=tmp3-(limit)
if tmp4>(limit):
tmp5=tmp4-(limit)
if tmp5>(limit):
tmp6=tmp5-(limit)
if tmp6>limit:
tmp7=tmp6-(limit)
if tmp7>(limit):
tmp8=tmp7-(limit)
else:
Energy[i]=limit
Energy[i+1]=limit
Energy[i+2]=limit
Energy[i+3]=limit
Energy[i+4]=limit
Energy[i+5]=limit
Energy[i+6]=tmp7
Do you have an idea of how to make it better? Thank you!

Welcome to stackoverflow!
Your code presently uses a loop where it doesn't need one and doesn't use a loop where it could be used.
Stepping into your code:
for i in range(0,1):
If we change this to:
for i in range(0,1):
print (i)
We will get the result 0 - it only runs once so there is no need to loop it - i isn't referred to in your code so there is no need to loop through it.
You could use a loop to allocate your c to an array but it isn't needed and I'll leave that as an exercise for yourself.
It can be approached in a different, more efficient way.
First of all when you're assigning variables try and make them descriptive and readable - you'll spend a lot more time coming back to code than you do reading it.
I don't know what system you're describing so I've just given generic names:
import numpy as np
length_arrary=12
limit=2.75
value_to_be_assigned=18
energy_result=np.zeros(length_arrary)
Now what we are really asking is two things, how many times does value_to_be_assigned divide into the limit (an integer) and what is the remainder.
Python has two operations for this floor division (//) and modulus which give:
11//5 = 2.0
1%5 = 1.0
So we know the first (value_to_be_assigned//limit elements - 1) of the array need to be equal to the limit and the final element needs to be equal to value_to_be_assigned % limit
Finally Python has an easy way to access elements of a list - we can set the first x elements to be equal to a value with:
array[:x]=value
x just needs to be an integer.
Putting it together we get:
filled_values=int(value_to_be_assigned//limit)
energy_result[:filled_values]=limit
energy_result[filled_values] = value_to_be_assigned % limit
and we can check with
energy_result.sum() # gives us 18

Faster way of centering lists

I'm looking for a better, faster way to center a couple of lists. Right now I have the following:
import random
m = range(2000)
sm = sorted(random.sample(range(100000), 16000))
si = random.sample(range(16005), 16000)
# Centered array.
smm = []
print sm
print si
for i in m:
if i in sm:
smm.append(si[sm.index(i)])
else:
smm.append(None)
print m
print smm
Which in effect creates a list (m) containing a range of random numbers to center against, another list (sm) from which m is centered against and a list of values (si) to append.
This sample runs fairly quickly, but when I run a larger task with much more variables performance slows to a standstill.

your mainloop contains this infamous line:
if i in sm:
it seems to be nothing but since sm is a result of sorted it is a list, hence O(n) lookup, which explains why it's slow with a big dataset.
Moreover you're using the even more infamous si[sm.index(i)], which makes your algorithm O(n**2).
Since you need the indexes, using a set is not so easy, and there's better to do:
Since sm is sorted, you could use bisect to find the index in O(log(n)), like this:
for i in m:
j = bisect.bisect_left(sm,i)
smm.append(si[j] if (j < len(sm) and sm[j]==i) else None)
small explanation: bisect gives you the insertion point of i in sm. It doesn't mean that the value is actually in the list so we have to check that (by checking if the returned value is within existing list range, and checking if the value at the returned index is the searched value), if so, append, else append None.

Cropping Python lists by value instead of by index

Good evening, StackOverflow.
Lately, I've been wrestling with a Python program which I'll try to outline as briefly as possible.
In essence, my program plots (and then fits a function to) graphs. Consider this graph.
The graph plots just fine, but I'd like it to do a little more than that: since the data is periodic over an interval OrbitalPeriod (1.76358757), I'd like it to start with our first x value and then iteratively plot all of the points OrbitalPeriod away from it, and then do the same exact thing over the next region of length OrbitalPeriod.
I know that there is a way to slice lists in Python of the form
croppedList = List[a:b]
where a and b are the indices of the first and last elements you'd like to include in the new list, respectively. However, I have no idea what the indices are going to be for each of the values, or how many values fall between each OrbitalPeriod-sized interval.
What I want to do in pseudo-code looks something like this.
croppedList = fullList on the domain [a + (N * OrbitalPeriod), a + (N+1 * OrbitalPeriod)]
where a is the x-value of the first meaningful data point.
If you have a workaround for this or a cropping method that would accept values instead of indices as arguments, please let me know. Thanks!

If you are working with numpy, you can use it inside the brackets
m = x
M = x + OrbitalPeriod
croppedList = List[m <= List]
croppedList = croppedList[croppedList < M]

python normal distribution

I have a list of numbers, with sample mean and SD for these numbers. Right now I am trying to find out the numbers out of mean+-SD,mean +-2SD and mean +-3SD.
For example, in the part of mean+-SD, i made the code like this:
ND1 = [np.mean(l)+np.std(l,ddof=1)]
ND2 = [np.mean(l)-np.std(l,ddof=1)]
m=sorted(l)
print(m)
ND68 = []
if ND2 > m and m< ND1:
ND68.append(m<ND2 and m>ND1)
print (ND68)
Here is my question:
1. Could number be calculated by the list and arrange. If so, which part I am doing wrong. Or there is some package I can use to solve this.

This might help. We will use numpy to grab the values you are looking for. In my example, I create a normally distributed array and then use boolean slicing to return the elements that are outside of +/- 1, 2, or 3 standard deviations.
import numpy as np
# create a random normally distributed integer array
my_array = np.random.normal(loc=30, scale=10, size=100).astype(int)
# find the mean and standard dev
my_mean = my_array.mean()
my_std = my_array.std()
# find numbers outside of 1, 2, and 3 standard dev
# the portion inside the square brackets returns an
# array of True and False values. Slicing my_array
# with the boolean array return only the values that
# are True
out_std_1 = my_array[np.abs(my_array-my_mean) > my_std]
out_std_2 = my_array[np.abs(my_array-my_mean) > 2*my_std]
out_std_3 = my_array[np.abs(my_array-my_mean) > 3*my_std]

You are on the right track there. You know the mean and standard deviation of your list l, though I'm going to call it something a little less ambiguous, say, samplePopulation.
Because you want to do this for several intervals of standard deviation, I recommend crafting a small function. You can call it multiple times without too much extra work. Also, I'm going to use a list comprehension, which is just a for loop in one line.
import numpy as np
def filter_by_n_std_devs(samplePopulation, numStdDevs):
# you mostly got this part right, no need to put them in lists though
mean = np.mean(samplePopulation) # no brackets needed here
std = np.std(samplePopulation) # or here
band = numStdDevs * std
# this is the list comprehension
filteredPop = [x for x in samplePopulation if x < mean - band or x > mean + band]
return filteredPop
# now call your function with however many std devs you want
filteredPopulation = filter_by_n_std_devs(samplePopulation, 1)
print(filteredPopulation)
Here's a translation of the list comprehension (based on your use of append it looks like you may not know what these are, otherwise feel free to ignore).
# remember that you provide the variable samplePopulation
# the above list comprehension
filteredPop = [x for x in samplePopulation if x < mean - band or x > mean + band]
# is equivalent to this:
filteredPop = []
for num in samplePopulation:
if x < mean - band or x > mean + band:
filteredPop.append(num)
So to recap:
You don't need to make a list object out of your mean and std calculations
The function call let's you plug in your samplePopulation and any number of standard deviations you want without having to go in and manually change the value
List comprehensions are one line for loops, more or less, and you can even do the filtering you want right inside it!

Replace loop with broadcasting in numpy -> memory error

I have an 2D-array (array1), which has an arbitrary number of rows and in the first column I have strictly monotonic increasing numbers (but not linearly), which represent a position in my system, while the second one gives me a value, which represents the state of my system for and around the position in the first column.
Now I have a second array (array2); its range should usually be the same as for the first column of the first array, but does not matter to much, as you will see below.
I am now interested for every element in array2:
1. What is the argument in array1[:,0], which has the closest value to the current element in array2?
2. What is the value (array1[:,1]) of those elements.
As usually array2 will be longer than the number of rows in array1 it is perfectly fine, if I get one argument from array1 more than one time. In fact this is what I expect.
The value from 2. is written in the second and third column, as you will see below.
My striped code looks like this:
from numpy import arange, zeros, absolute, argmin, mod, newaxis, ones
ysize1 = 50
array1 = zeros((ysize1+1,2))
array1[:,0] = arange(ysize1+1)**2
# can be any strictly monotonic increasing array
array1[:,1] = mod(arange(ysize1+1),2)
# in my current case, but could also be something else
ysize2 = (ysize1)**2
array2 = zeros((ysize2+1,3))
array2[:,0] = arange(0,ysize2+1)
# is currently uniformly distributed over the whole range, but does not necessarily have to be
a = 0
for i, array2element in enumerate(array2[:,0]):
a = argmin(absolute(array1[:,0]-array2element))
array2[i,1] = array1[a,1]
It works, but takes quite a lot time to process large arrays. I then tried to implement broadcasting, which seems to work with the following code:
indexarray = argmin(absolute(ones(array2[:,0].shape[0])[:,newaxis]*array1[:,0]-array2[:,0][:,newaxis]),1)
array2[:,2]=array1[indexarray,1] # just to compare the results
Unfortunately now I seem to run into a different problem: I get a memory error on the sizes of arrays I am using in the line of code with the broadcasting.
For small sizes it works, but for larger ones where len(array2[:,0]) is something like 2**17 (and could be even larger) and len(array1[:,0]) is about 2**14. I get, that the size of the array is bigger than the available memory. Is there an elegant way around that or to speed up the loop?
I do not need to store the intermediate array(s), I am just interested in the result.
Thanks!

First lets simplify this line:
argmin(absolute(ones(array2[:,0].shape[0])[:,newaxis]*array1[:,0]-array2[:,0][:,newaxis]),1)
it should be:
a = array1[:, 0]
b = array2[:, 0]
argmin(abs(a - b[:, newaxis]), 1)
But even when simplified, you're creating two large temporary arrays. If a and b have sizes M and N, b - a and abs(...) each create a temporary array of size (M, N). Because you've said that a is monotonically increasing, you can avoid the issue all together by using a binary search (sorted search) which is much faster anyways. Take a look at the answer I wrote to this question a while back. Using the function from this answer, try this:
closest = find_closest(array1[:, 0], array2[:, 0])
array2[:, 2] = array1[closest, 1]