Let's say I have a portfolio with weights, sum = 1.
Then I want to define pockets (0, 1, 2) with some assets included in those pockets, and sum(weights_pocket_assets) < pocket_max_weight
On my UI, I have a 3 columns for each pocket, filled with 1 if asset is in pocket, 0 otherwise (this array is called 'pockets')
mask = list(map(int, pockets[0]))
print(pocket_max[0], mask)
constr0 = {'type': 'ineq', 'fun': lambda x: pocket_max[0] - np.sum(np.ma.array(x, mask=np.logical_not(mask)))}
mask = list(map(int, pockets[1]))
print(pocket_max[1], mask)
constr1 = {'type': 'ineq', 'fun': lambda x: pocket_max[1] - np.sum(np.ma.array(x, mask=np.logical_not(mask)))}
mask = list(map(int, pockets[2]))
print(pocket_max[2], mask)
constr2 = {'type': 'ineq', 'fun': lambda x: pocket_max[2] - np.sum(np.ma.array(x, mask=np.logical_not(mask)))}
constr = [{'type': 'eq', 'fun': lambda x: np.sum(x) - 1}, constr0, constr1, constr2]
print(constr)
gives as output:
0.04 [1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
0.08 [0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
0.05 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0]
[{'fun': <function Book.optimize.<locals>.<lambda> at 0x00000060CCD9AD90>, 'type': 'eq'},
{'fun': <function Book.optimize.<locals>.<lambda> at 0x00000060CCD9AD08>, 'type': 'ineq'},
{'fun': <function Book.optimize.<locals>.<lambda> at 0x00000060CCD9ABF8>, 'type': 'ineq'},
{'fun': <function Book.optimize.<locals>.<lambda> at 0x00000060CCD9AAE8>, 'type': 'ineq'}]
which seems correct.
Problem is optimize only with eq and last ineq (ie sum(w) = 1 and sum(w_pocket_2) = 0.05)
Also if I have only 2 pockets, it optimizes with last only.
For short, it only takes last ineq in optimization... don't know what's wrong.
EDIT:
if pockets are equals (ie, same components for each pocket, ie same mask) then all 3 'ineq' constraints are taken into account (ie the most constraining in fact, but no problem if it's the first 'ineq', the second or the third that is the most constraining)
as soon as pockets have not similar components (ex: we add a component for the second 'ineq'), only the last 'ineq' is taken by optimization.
As often, it was a problem with variable scope (odd stuff since variables were still defined, but overwritten. I did not investigate more to fully understand). The following code works as expected (all pockets constraints taken into account):
constr += ({'type': 'ineq',
'fun': lambda x, mask, pocket_max, i: pocket_max[i] - np.sum(np.ma.array(x, mask=np.logical_not(mask))),
'args': (mask, pocket_max, i )}, )
Related
I'm trying to figure out how an iterator works with this example:
There is the function that produces stream generator for given iterable object (list, generator, etc) whose elements contain position and value and sorted by order of apperance.
Stream generator is equal to initial stream (without position), gaps filled with zeroes.
from itertools import count
def gen_stream(total, sorted_iterable, extractor=lambda x: x):
sorted_iterator = iter(sorted_iterable)
iterable = count() if total is None else range(total)
try:
current_extracted_record = extractor(next(sorted_iterator))
except StopIteration:
current_extracted_record = None
for i in iterable:
if current_extracted_record:
if i == current_extracted_record[0]:
try:
yield current_extracted_record[1]
current_extracted_record = extractor(next(sorted_iterator))
except StopIteration:
current_extracted_record = None
else:
yield 0
else:
yield 0
Here are other, most likely, more advanced options for writing this function.
For example:
gen = gen_stream(9,[(4,111),(7,12)])
list(gen)
[0, 0, 0, 0, 111, 0, 0, 12, 0] # first element has zero index, so 111 located on fifth position, 12 located on 8th position
This function also support custom position-value extractor for more advanced cases, e.g.
def day_extractor(x):
months = [31, 28, 31, 30, 31, 31, 30, 31, 30, 31, 30, 31]
acc = sum(months[:x[1] - 1]) + x[0] - 1
return acc, x[2]
precipitation_days = [(3,1,4),(5,2,6)]
list(gen_stream(59,precipitation_days,day_extractor)) #59: January and February to limit output
[0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
precipitation_days format is following: (d,m,mm), where d - day in month, m - month, mm - precipitation in millimeters
So, in example:
(3,1,4) # January,3 precipitation: 4 mm
(5,2,6) # February,5 precipitation: 6 mm
Extractor passed as optional third parameter with default value - lambda function that handles (position, value) pairs like in first example.
Question 4
Following the example of this statement
print(list(gen_stream(59, precipitation_days, day_extractor)))
How to add examples with None (instead 59) but limit output using the loop for.
Answer
gen = gen_stream(None, [(4, 111), (7, 12)])
print([next(gen) for _ in range(20)], '\n')
Let's assume I have a list:
list1 = [16620, 22032, 0, 0, 0, 136813, 137899, 0, 199546, 204804]
I am looking for a new list that subtracts every 'non-zero' value from the following 'non-zero' value, e.g. 22032-16620, 137899-136813. 'Zero' values will stay untouched.
In addition to that, the subtracted 'non-zero' value should change to zero.
The output would look something like:
list2 = [0, 5412, 0, 0, 0, 0, 1086, 0, 0, 5258]
Please note that the numbers, the length of the list and its distribution of elements may vary, e.g. a list could also look like
list1 = [0, 0, 0, 0, 95472, 0, 0, 104538, 0, 0, 0, 0, 187649, 0, 0, 204841, 0, 0, 0, 0, 0, 0, 0, 0]
which should turn into:
list2 = [0, 0, 0, 0, 0, 0, 0, 9066, 0, 0, 0, 0, 0, 0, 0, 17192, 0, 0, 0, 0, 0, 0, 0, 0]
As you can see, the number of elements stays the same for list1 and list2. Also, there is always an even number of 'zero' values and an even number of 'non-zero' values.
Help is greatly appreciated!
What I have so far:
from itertools import cycle, chain
list1 = [11545, 15334, 71341, 73861, 0, 0, 170374, 171671]
newlist = [list1[i + 1] - list1[i] for i in range(len(list1)-1)]
list2 = list(chain.from_iterable(zip(newlist[0::2], cycle([int()]))))
print list2
The output print list2 looks like I imagine it to be, yet it won't work for a list that looks like:
list1 = [16620, 22032, 0, 0, 0, 136813, 137899, 0, 199546, 204804]
Copy over, but adjust non-zero values, keeping track of the previous one.
list2 = []
prev = None
for curr in list1:
if curr:
if prev:
curr -= prev
prev = None
else:
prev = curr
curr = 0
list2.append(curr)
A teacher is in the process of generating few reports based on the marks scored by the students of her class in a project based assessment.
Assume that the marks of her 10 students are available in a tuple. The marks are out of 25.
Write a python program to implement the following functions:
find_more_than_average(): Find and return the percentage of students who have scored more than the average mark of the class
sort_marks(): Sort the marks in the increasing order from 0 to 25. The sorted values should be populated in a list and returned
generate_frequency(): Find how many students have scored the same marks. For example, how many have scored 0, how many have scored 1, how many have scored 3….how many have scored 25. The result should be populated in a list and returned.
i got the average and sorted parts correct.but for the frequency, if the element is repeated twice i got the frequency as 1
list_of_marks=(12,18,25,24,2,5,18,20,20,21)
def find_more_than_average():
sumi=0
count=0
sumi=sum(list_of_marks)
avg=sumi/len(list_of_marks)
for i in list_of_marks:
if(i>avg):
count=count+1
morethanavg=(count/len(list_of_marks))*100
return morethanavg
#Remove pass and write your logic here
def sort_marks():
return sorted(list_of_marks)
#Remove pass and write your logic here
def generate_frequency():
#Remove pass and write your logic here
gener=[]
for i in range(0,26):
if i in list_of_marks:
gener.append(1)
else:
gener.append(0)
return gener
print(find_more_than_average())
print(generate_frequency())
print(sort_marks())
expected-[0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 2, 1, 0, 0, 1, 1]
actual-[0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1]
When you do:
if i in list_of_marks:
gener.append(1)
else:
gener.append(0)
it should be clear that you can never get a value other than 0 or 1. But you want the counts of those values not just a 1 indicating the value is in the list. One options is to create a list of zeros first, then step through the marks and add one to the index corresponding to the mark:
def generate_frequency():
gener = [0] * 26
for m in list_of_marks:
gener[m] += 1
return gener
Now when you see 20 twice you will increase generator[20] twice with the result:
[0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 2, 1, 0, 0, 1, 1]
I still don't understand how to only capture one pulse per data file
My professor and I are trying to modify the code below to send a uniquely named .txt file after every pulse we receive from our Arduino.So far, our efforts have been fruitless. Right now it sends us all of the data received after a period of time and compiles all of the pulse data into one file. Since the file name isn't changing that data stored in the file is only temporary.
The Arduino Sketch is called Girino and can be found here. The code that we wish to modify is the accompanying python code called readgirino.py that's found under Step #23.
I think we have to modify this loop;
for i in range(5):
stream.write('s')
eventString = stream.read(1280)
eventData.append(np.array(unpack( '1280B', eventString )))
print eventData
stream.write('S')
stream.flushInput()
np.savetxt( "data.txt", eventData[4] )
Any suggestions?
Thank you!
UPDATE
-Creating a unique file name was solved with:
stream.write('S')
stream.flushInput()
n = n + 1
np.savetxt( "data" +str( n) + ".txt", eventData[4] )
Although I still don't understand how to only capture one pulse per data file.
The Girino code makes no mention of eventData, but I think it means the array that appears in my terminal window after I run the readgirino.py code:
Girino tells me that it is ready
Buffer size: 1280
Baud rate: 115200
Wait duration: 1248
Prescaler: 128
Trigger event: 3
Threshold: 127
Setting prescaler to: 128
Setting threshold to: 50
Buffer size: 1280
Baud rate: 115200
Wait duration: 1248
Prescaler: 128
Trigger event: 3
Threshold: 50
[array([0, 0, 0, ..., 0, 0, 0])]
[array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0])]
[array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0])]
[array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0])]
[array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0]), array([0, 0, 0, ..., 0, 0, 0])]
Change the "data.txt" in
np.savetxt( "data.txt", eventData[4] )
to something that changes every pulse. You can increment a number (data1.txt, data2.txt) for instance.
namestr = "data" + str(n) + ".txt"
n = n + 1
If you want to save the constituent arrays in eventData to different files, you'll need an additional for loop:
for event in eventData:
n = n + 1
np.savetxt( "data" + str(n) + ".txt", event )
Numpy has a library function, np.unpackbits, which will unpack a uint8 into a bit vector of length 8. Is there a correspondingly fast way to unpack larger numeric types? E.g. uint16 or uint32. I am working on a question that involves frequent translation between numbers, for array indexing, and their bit vector representations, and the bottleneck is our pack and unpack functions.
You can do this with view and unpackbits
Input:
unpackbits(arange(2, dtype=uint16).view(uint8))
Output:
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0]
For a = arange(int(1e6), dtype=uint16) this is pretty fast at around 7 ms on my machine
%%timeit
unpackbits(a.view(uint8))
100 loops, best of 3: 7.03 ms per loop
As for endianness, you'll have to look at http://docs.scipy.org/doc/numpy/user/basics.byteswapping.html and apply the suggestions there depending on your needs.
This is the solution I use:
def unpackbits(x, num_bits):
if np.issubdtype(x.dtype, np.floating):
raise ValueError("numpy data type needs to be int-like")
xshape = list(x.shape)
x = x.reshape([-1, 1])
mask = 2**np.arange(num_bits, dtype=x.dtype).reshape([1, num_bits])
return (x & mask).astype(bool).astype(int).reshape(xshape + [num_bits])
This is a completely vectorized solution that works with any dimension ndarray and can unpack however many bits you want.
I have not found any function for this too, but maybe using Python's builtin struct.unpack can help make the custom function faster than shifting and anding longer uint (note that I am using uint64).
>>> import struct
>>> N = np.uint64(2 + 2**10 + 2**18 + 2**26)
>>> struct.unpack('>BBBBBBBB', N)
(2, 4, 4, 4, 0, 0, 0, 0)
The idea is to convert those to uint8, use unpackbits, concatenate the result. Or, depending on your application, it may be more convenient to use structured arrays.
There is also built-in bin() function, which produces string of 0s and 1s, but I am not sure how fast it is and it requires postprocessing too.
This works for arbitrary arrays of arbitrary uint (i.e. also for multidimensional arrays and also for numbers larger than the uint8 max value).
It cycles over the number of bits, rather than over the number of array elements, so it is reasonably fast.
def my_ManyParallel_uint2bits(in_intAr,Nbits):
''' convert (numpyarray of uint => array of Nbits bits) for many bits in parallel'''
inSize_T= in_intAr.shape
in_intAr_flat=in_intAr.flatten()
out_NbitAr= numpy.zeros((len(in_intAr_flat),Nbits))
for iBits in xrange(Nbits):
out_NbitAr[:,iBits]= (in_intAr_flat>>iBits)&1
out_NbitAr= out_NbitAr.reshape(inSize_T+(Nbits,))
return out_NbitAr
A=numpy.arange(256,261).astype('uint16')
# array([256, 257, 258, 259, 260], dtype=uint16)
B=my_ManyParallel_uint2bits(A,16).astype('uint16')
# array([[0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
# [1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
# [0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
# [1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
# [0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]], dtype=uint16)