I'm writing a program that scrapes option_chain data off the TMX website and suggests an optimized Covered Call options portfolio based on those data. for the optimization process, I used the z3py library as discussed on this website by many users. the optimization process works by maximizing the premiums and setting the portfolio delta to a user-specified amount.
initially, I made a mistake in my calculation for portfolio delta which made everything work so smoothly but I'm facing issues after correcting it. portfolio delta is calculated by taking the Weighted average delta of non-zero positions in the portfolio. to achieve this I used the following setup:
eng = Optimize()
Weights = [Real(row.symbol) for row in df.itertuples()]
#to count non-zero positions
TotCount = Real("TotCount")
eng.add(TotCount == Sum([If(w > 0, 1, 0) for w in Weights]))
eng.add(TotCount == If(TotCount >= 0, TotCount, 1))
#to get portfolio delta
eng.add(TotDelta == (Sum([(w * row.delta) for w, row in zip(Weights, df.itertuples())]) / TotCount))
eng.add(TotDelta == delta)
res = eng.check()
the weird behavior happens when I run this as a function in a loop with different values for delta, the first loop actually does the calculation and spits out an answer but after that my code gets stuck on the last line for hours without any progress. i tried a few things like completely reformatting it but nothing seems to make a difference. I was wondering if anyone knows what's happening here or not?
Unfortunately, there isn't much in your description for anyone to go on without knowing further details of your setup. All one can deduce from your text is that the constraints are hard to solve for those values of delta where it takes longer. Without seeing the rest of your program, it's impossible to opine on what else might be going on.
See if you can isolate the value of delta that it runs slowly on, and just run it for that one instance separately to see if there's interaction coming from elsewhere. That's by no means a solution of course, but it's one way to get started.
One thing I noticed, though, is this line you have:
eng.add(TotCount == If(TotCount >= 0, TotCount, 1))
Let's see what this line is saying: If TotCount >= 0, then it says TotCount == TotCount. i.e., it puts no restrictions on it. Otherwise, it says TotCount == 1; i.e., if TotCount < 0, then TotCount == 1. Well this latter statement is obviously false. That is if, the other constraints force TotCount < 0, then your program would be unsat. This essentially means that whole line is functionally equivalent to:
eng.add(TotCount >= 0)
It's hard for anyone to tell if that's what you intended; but I suspect perhaps it isn't; since you'd have just written the simpler form above. Perhaps you wanted to say something more along the lines of if TotCount < 0, then make it 1. But looking at the previous line, (i.e., the Sum expression), we see that this'll never be the case. So, something is fishy there.
Hope that helps to get you started; note that you cannot model "sequential assignment" like in a programming language in this way. (You'd need to do what's known as single static assignment, a.k.a. SSA, conversion.) But then again, without knowing your exact intention, it's hard to opine.
Related
Use case
This is just an easy example for understanding, how and why it is not working as expected.
There is a set of processes, which have a start and a finishing timestamp.
the start timestamp of a process must be after the finishing timestamp of it's predecessor. So far, so good.
Consideration
Regarding constraints: Shouldn't it be possible to carry out more complex operations than arithmetic equations (e.g. queries and case distinctions)?
This is illustrated in the code below.
The standard formulation of the constraint works properly.
But it fails, if you put a function call into the equation.
def func(p):
if self.start_timestamps[p] >= self.end_timestamps[p-1]:
return 1
return 0
# constraint for precedences of processes
for process_idx in self.processes:
if process_idx > 0:
# works fine !
model += self.start_timestamps[process_idx] >= self.end_timestamps[process_idx-1]
# doesn't work, but should?!
model += func(process_idx) == 1
Questions
Are there ways to resolve that via function call? (In a more complex case, for example, you would have to make different queries and iterations within the function.)
If it is not possible with PuLP, are there other OR libraries, which can process things like that?
Thanks!
I am coding a black hole (actually photons orbiting a black hole) and I need to handle an exception for radius values that are smaller than the limit distance
I've tried using if and while True
def Hamiltonian(r, pt, pr, pphi):
H = (-((1-rs/r)**-1)*(pt**2)/2 + (1-rs/r)*(pr**2)/2 + (pphi**2)/(2* (r**2)))
if np.amax(H) < 10e-08:
print("Your results are correct")
else:
print("Your results are wrong")
return(H)
def singularity(H, r):
if (r).any < 1.5*rs:
print(H)
else:
print("0")
return(H, r)
print(Hamiltonian(r00, pt00, pr00, pphi00))
I'd like to handle the case r < 1.5*rs so that I don't have the division error message and weird orbits anymore. So far my code doesn't change anything to the problem, I still get this error message :
"RuntimeWarning: divide by zero encountered in true_divide
H = (-((1-rs/r)**-1)*(pt**2)/2 + (1-rs/r)*(pr**2)/2 + (pphi**2)/(2*(r**2)))"
and my orbits are completely wrong (for example my photon should go right in the black hole, but since there's a singularity at r < 1.5*rs, they go in and leave again and so on)
I'd like to delete the photons causing the problem but I don't know how, can anyone please help me?
I think you mention that we do not know exactly what goes on at a singularity. Any answer provided here most likely would not be accurate, but let's just assume that you know the behavior/dynamics at the near 0 neighborhood. I do not know how you are calling the Hamiltonian function but I can imagine you are doing it one of two ways.
You have a pre-defined loop that loops through each value you are passing into the function and outputting the resulting H for each value in that pre-defined loop.
You are passing in vectors of the same length and doing element by element math in the H function.
In the first case, you could either do a pre-check and write a new function for the near 0 behavior and call that new function if you are in the near zero neighborhood. You could also just have the check in the hamiltonian function itself and call the new function if you are in the near 0 neighborhood. This latter method is what I would prefer as it keeps the front-end (the best word I have for it) fairly clean and encapsulates the logic/math in your Hamiltonian function. In your comments you say you want to delete the photon within some radius, and that would be extremely easy to do with this method by just terminating the for loop by breaking out at that time step and plotting what you had until that time step.
In the second case, you would have to manually build the new vector by having checks throughout your vectors if they fall within the singularity neighborhood. This would be a little more difficult and would depend on the shape of your inputs and doing element by element math is more difficult to debug in my opinion.
I'm trying to use scipy's ode solver to plot the interaction between a 2D system of equations. I'm attempting to alter the parameters passed to the solver by the following block of code:
# define maximum number of iteration steps for ode solver iteration
m = 1 #power of iteration
N = 2**m #number of steps
# setup a try-catch formulation to increase the number of steps as needed for solution to converge
while True:
try:
z = ode(stateEq).set_integrator("vode",nsteps=N,method='bdf',max_step=5e5)
z.set_initial_value(x0, t0)
for i in range(1, t.size):
if i%1e3 == 0:
print 'still integrating...'
x[i, :] = z.integrate(t[i]) # get one more value, add it to the array
if not z.successful():
raise RuntimeError("Could not integrate")
break
except:
m += 1
N = 2**m
if m%2 == 0:
print 'increasing nsteps...'
print 'nsteps = ', N
Running this never breaks the while loop. It keeps increasing the nsteps forever and the system never gets solved. If I don't put it in the while loop, the system gets solved, I think, because the solution gets plotted. Is the while loop necessary? Am I formulating the solver incorrectly?
The parameter nsteps regulates how many integration steps can be maximally performed during one sampling step (i.e., a call of z.integrate). Its default value is okay if your sampling step is sufficiently small to capture the dynamics. If you want to integrate over a huge time span in one large sampling step (e.g., to get rid of transient dynamics), the value can easily be too small.
The point of this parameter is to avoid problems arising from unexpectedly very long integrations. For example, if you want to perform a given integration for 100 values of a control parameter in a loop over night, you do not want to see on the next morning that the No. 14 was pathological and is still running.
If this is not relevant to you, just set nsteps to a very high value and stop worrying about it. There is certainly no point to successively increase nsteps, you are just performing the same calculations all over again.
Running this never breaks the while loop. It keeps increasing the nsteps forever and the system never gets solved.
This suggests that you have a different problem than nsteps being exceeded, most likely that the problem is not well posed. Carefully read the error message produced by the integrator. I also recommend that you check your differential equations. It may help to look at the solutions until the integration fails to see what is going wrong, i.e., plot x after running this:
z = ode(stateEq)
z.set_integrator("vode",nsteps=1e10,method='bdf',max_step=5e5)
z.set_initial_value(x0, t0)
for i,time in enumerate(t):
x[i,:] = z.integrate(time)
if not z.successful():
break
Your value for max_step is very high (this should not be higher than the time scale of your dynamics). Depending on your application, this may very well be reasonable, but then it suggests that you are working with large numbers. This in turn may mean that the default values of the parameters atol and first_step are not suited for your situation and you need to adjust them.
I've been working on the following code which sort of maximizes the number of unique (in lowest common denominator) p by q blocks with some constraints. It is working perfectly. For small inputs. E.g. input 50000, output 1898.
I need to run it on numbers greater than 10^18, and while I have a different solution that gets the job done, this particular version gets super slow (made my desktop reboot at one point), and this is what my question is about.
I'm trying to figure out what is causing the slowdown in the following code, and to figure out in what order of magnitude they are slow.
The candidates for slowness:
1) the (-1)**(i+1) term? Does Python do this efficiently, or is it literally multiplying out -1 by itself a ton of times?
[EDIT: still looking for how operation.__pow__ works, but having tested setting j=-j: this is faster.]
2) set instantiation/size? Is the set getting too large? Obviously this would impact membership check if the set can't get built.
3) set membership check? This indicates O(1) behavior, although I suppose the constant continues to change.
Thanks in advance for insight into these processes.
import math
import time
a=10**18
ti=time.time()
setfrac=set([1])
x=1
y=1
k=2
while True:
k+=1
t=0
for i in xrange(1,k):
mo = math.ceil(k/2.0)+((-1)**(i+1))*(math.floor(i/2.0)
if (mo/(k-mo) not in setfrac) and (x+(k-mo) <= a and y+mo <= a):
setfrac.add(mo/(k-mo))
x+=k-mo
y+=mo
t+=1
if t==0:
break
print len(setfrac)+1
print x
print y
to=time.time()-ti
print to
I'm reading a book on Python3 (Introducing Python by Bill Lubanovic), and came across something I wasn't sure is a Python preference, or just a "simplification" due to being a book and trying to describe something else.
It's on how to write to a file using chunks instead of in one shot.
poem = '''There was a young lady named Bright,
Whose speed was far faster than light;
She started one day
In a relative way,
And returned on the previous night.'''
fout = open('relativity', 'wt')
size = len(poem)
offset = 0
chunk = 100
while True:
if offset > size:
break
fout.write(poem[offset:offset+chunk])
offset += chunk
fout.close()
I was about to ask why it has while True instead of while (offset > size), but decided to try it for myself, and saw that while (offset > size) doesn't actually do anything in my Python console.
Is that just a bug in the console, or does Python really require you to move the condition inside the while loop like that? With all of the changes to make it as minimal as possible, this seems very verbose.
(I'm coming from a background in Java, C#, and JavaScript where the condition as the definition for the loop is standard.)
EDIT
Thanks to xnx's comment, I realized that I had my logic incorrect in what I would have the condition be.
This brings me back to a clearer question that I originally wanted to focus on:
Does Python prefer to do while True and have the condition use a break inside the loop, or was that just an oversight on the author's part as he tried to explain a different concept?
I was about to ask why it has while True instead of while (offset <= size), but decided to try it for myself,
This is actually how I would have written it. It should be logically equivelent.
and saw that while (offset > size) doesn't actually do anything in my Python console.
You needed to use (offset <= size), not (offset > size). The current logic stops as soon as the offset is greater than size, so reverse the condition if you want to put it in the while statement.
does Python really require you to move the condition inside the while loop like that?
No, Python allows you write write the condition in the while loop directly. Both options are fine, and it really is more a matter of personal preference in how you write your logic. I prefer the simpler form, as you were suggesting, over the original author's version.
This should work fine:
while offset <= size:
fout.write(poem[offset:offset+chunk])
offset += chunk
For details, see the documentation for while, which specifically states that any expression can be used before the :.
Edit:
Does Python prefer to do while True and have the condition use a break inside the loop, or was that just an oversight on the author's part as he tried to explain a different concept?
Python does not prefer while True:. Either version is fine, and it's completely a matter of preference for the coder. I personally prefer keeping the expression in the while statement, as I find the code more clear, concise, and maintainable using while offset <= size:.
It is legal Python code to put the conditional in the loop. Personally I think:
while offset <= size:
is clearer than:
while True:
if offset < size:
break
I prefer the first form because there's one less branch to follow but the logic is not any more complex. All other things being equal, lower levels of indentation are easier to read.
If there were multiple different conditions that would break out of the loop then it might be preferable to go for the while True syntax.
As for the observed behavior with the incorrect loop logic, consider this snippet:
size = len(poem)
offset = 0
while offset > size:
#loop code
The while loop will never be entered as offset > size starts off false.
while True:
if offset > size:
break
func(x)
is exactly equivalent to
while offset <= size:
func(x)
They both run until offset > size. It is simply a different way of phrasing it -- both are acceptable, and I'm not aware of any performance differences. They would only run differently if you had the break condition at the bottom of the while loop (i.e. after func(x))
edit:
According to the Python wiki, in Python 2.* "it slows things down a lot" to put the condition inside the while loop: "this is due to first testing the True condition for the while, then again testing" the break condition. I don't know what measure they use for "a lot", but it seems miniscule enough.
When reading from a file, you usually do
output = []
while True:
chunk = f.read(chunksize)
if len(chunk) == 0:
break
output.append(chunk)
It seems to me like the author is more used to doing reading than writing, and in this case the reading idiom of using while True has leaked through to the writing code.
As most of the folks answering the question can attest to, using simply while offset <= size is probably more Pythonic and simpler, though even simpler might be just to do
f.write(poem)
since the underlying I/O library can handle the chunked writes for you.
Does Python prefer to do while True and have the condition use a break
inside the loop, or was that just an oversight on the author's part as
he tried to explain a different concept?
No it doesn't, this is a quirk or error of the author's own.
There are situations where typical Python style (and Guido van Rossum) actively advise using while True, but this isn't one of them. That said, they don't disadvise it either. I imagine there are cases where a test would be easier to read and understand as "say when to break" than as "say when to keep going". Even though they're just logical negations of each other, one or other might make express things a little more simply:
while not god_unwilling() and not creek_risen():
while not (god_unwilling() or creek_risen()):
vs.
while True:
if god_unwilling() or creek_risen():
break
I still sort of prefer the first, but YMMV. Even better introduce functions that correspond to the English idiom: god_willing() and creek_dont_rise()
The "necessary" use is when you want to execute the break test at the end or middle of the loop (that is to say, when you want to execute part or all of the loop unconditionally the first time through). Where other languages have a greater variety of more complex loop constructs, and other examples play games with a variable to decide whether to break or not, Guido says "just use while True". Here's an example from the FAQ, for a loop that starts with an assignment:
C code:
while (line = readline(f)) {
// do something with line
}
Python code:
while True:
line = f.readline()
if not line:
break
# do something with line
The FAQ remarks (and this relates to typical Python style):
An interesting phenomenon is that most experienced Python programmers
recognize the while True idiom and don’t seem to be missing the
assignment in expression construct much; it’s only newcomers who
express a strong desire to add this to the language.
It also points out that for this particular example, you can in fact avoid the whole problem anyway with for line in f.
In my opinion, while True is better than other ways, in big programs; which has long codes. Because you cant see actually that a variable may change because of some functions or etc. while True means start if its true which means start this loop whatever happens except closed the program. So that maybe writer of the book want you used to use while True, is a little less risky than others.
It's better used to while True and define the variable which is may stop this loop.