We Keep Coding

Using global parameters of z3py

I'm trying to use SMT solver over a scheduling problem and could not find anything helping in the documentation.
It seems using following ways of setting parameters do not have any effect on the solver.
from z3 import *
set_param(logic="QF_UFIDL")
s = Optimize() # or even Solver()
or even
from z3 import *
s = Optimize()
s.set("parallel.enable", True)
So how can I set [global] parameters effectively in z3py. to be most specific I need to set parameters below:
parallel.enable=True
auto_confic=False
smtlib2_compliant=True
logic="QF_UFIDL"

Use global parameter statements like the following on separate lines before creating Solver or Optimize object:
set_param('parallel.enable', True)
set_param('parallel.threads.max', 4) # default 10000
To set non-global parameters specific to a Solver or Optimize object, you can use the help() function to show available parameters:
o = Optimize()
o.help()
s = Solver()
s.help()
The following example shows how to set an Optimize parameter:
opt = Optimize()
opt.set(priority='pareto')

Use set_param, as described here: https://z3prover.github.io/api/html/namespacez3py.html#a4ae524d5f91ad1b380d8821de01cd7c3
It isn't clear what's not working for you. Are you getting an error message back? From your description, I understand that the setting does indeed take place, but you don't see any change in behavior? For that, you'll have to provide a concrete example we can look at. Note that for most parameters, the effects will only be visible with benchmarks that trigger the option, and even then it'll be hard to tell what (if any) effect it had, unless you dig into verbose log output.
Also, parallel-solving features, which you seem to be interested in, isn't going to gain you much. See Section 9.2 of https://z3prover.github.io/papers/z3internals.html: Essentially it boils down to attempting to solve the same problem with different seeds to see if one of them goes faster. If you have many cores lying around it might be worth a try, but don't expect magic out of it.

Is there a way for a caller of multiple functions to forward a function ref to selected functions in a purely functional way?

Problem
I have a function make_pipeline that accepts an arbitrary number of functions, which it then calls to perform sequential data transformation. The resulting call chain performs transformations on a pandas.DataFrame. Some, but not all functions that it may call need to operate on a sub-array of the DataFrame. I have written multiple selector functions. However at present each member-function of the chain has to be explicitly be given the user-selected selector/filter function. This is VERY error-prone and accessibility is very important as the end-code is addressed to non-specialists (possibly with no Python/programming knowledge), so it must be "batteries-included". This entire project is written in a functional style (that's what's always worked for me).
Sample Code
filter_func = simple_filter()
# The API looks like this
make_pipeline(
load_data("somepath", header = [1,0]),
transform1(arg1,arg2),
transform2(arg1,arg2, data_filter = filter_func),# This function needs access to user-defined filter function
transform3(arg1,arg2,, data_filter = filter_func),# This function needs access to user-defined filter function
transform4(arg1,arg2),
)
Expected API
filter_func = simple_filter()
# The API looks like this
make_pipeline(
load_data("somepath", header = [1,0]),
transform1(arg1,arg2),
transform2(arg1,arg2),
transform3(arg1,arg2),
transform4(arg1,arg2),
)
Attempted
I thought that if the data_filter alias is available in the caller's namespace, it also becomes available (something similar to a closure) to all functions it calls. This seems to happen with some toy examples but wont work in the case (UnboundError).
What's a good way to make a function defined in one place available to certain interested functions in the call chain? I'm trying to avoid global.
Notes/Clarification
I've had problems with OOP and mutable states in the past, and functional programming has worked quite well. Hence I've set a goal for myself to NOT use classes (to the extent that Python enables me to anyways). So no classes.
I should have probably clarified this initially: In the pipeline the output of all functions is a DataFrame and the input of all functions (except load data obviously) is a DataFrame. The functions are decorated with a wrapper that calls functools.partial because we want the user to supply the args to each function but not execute it. The actual execution is done be a forloop in make_pipeline.
Each function accepts df:pandas.DataFrame plus all arguements that are specific to that function. The statement seen above transform1(arg1,arg2,...) actually calls the decorated transform1 witch returns functools.partial(transform, arg1,arg2,...) which is now has a signature like transform(df:pandas.DataFrame).
load_dataframe is just a convenience function to load the initial dataframe so that all other functions can begin operating on it. It just felt more intuitive to users to have it part of the chain rather that a separate call
The problem is this: I need a way for a filter function to be initialized (called) in only on place, such that every function in the call chain that needs access to the filter function, gets it without it being explicitly passed as argument to said function. If you're wondering why this is the case, it's because I feel that end users will find it unintuitive and arbitrary. Some functions need it, some don't. I'm also pretty certain that they will make all kinds of errors like passing different filters, forgetting it sometimes etc.
(Update) I've also tried inspect.signature() in make_pipeline to check if each function accepts a data_filter argument and pass it on. However, this raises an incorrect function signature error so some unclear reason (likely because of the decorators/partial calls). If signature could the return the non-partial function signature, this would solve the issue, but I couldn't find much info in the docs

Turns out it was pretty easy. The solution is inspect.signature.
def make_pipeline(*args, data_filter:Optional[Callable[...,Any]] = None)
d = args[0]
for arg in args[1:]:
if "data_filter" in inspect.signature(arg):
d = arg(d, data_filter = data_filter)
else:
d= arg(d)
Leaving this here mostly for reference because I think this is a mini design pattern. I've also seen an function._closure_ on unrelated subject. That may also work, but will likely be more complicated.

Best practice for "get" functions

I am new to Python.
Assume I have a dictionary which holds power supply admin state.
(OK = Turned on. FAIL = Turned off).
There are several way to write the "get" function:
1st way
is_power_supply_off(dictionary)
gets the admin state from dictionary.
returns true if turned off.
returns false if turned on.
is_power_supply_on(dictionary)
gets the admin state from dictionary.
returns true if turned on.
returns false if turned off.
2nd way
is_power_supply_on_or_off(dictionary, on_or_off)
gets the admin state from dictionary.
returns true/false based on the received argument
3rd way
get_power_supply_admin_state(dictionary)
gets the admin state from dictionary.
return value.
Then, I can ask in the function which calls the get function
if get_power_supply_admin_state() == turned_on/turned_off...
My questions are:
Which of the above is considered best practice?
If all three ways are OK, and it`s just a matter of style, please let me know.
Is 1st way considered as "code duplication"? I am asking this because I can combine the two functions to be just one (by adding an argument, as I did in the 2nd way. Still, IMO, 1st way is more readable than 2nd way.
I will appreciate if you can share your thoughts on EACH of the ways I specified.
Thanks in advance!

I would say that the best approach would be to have only a is_power_supply_on function. Then, to test if it is off, you can do not is_power_supply_on(dictionary).
This could even be a lambda (assuming state is the key of the admin state)::
is_power_supply_on = lambda mydict: mydict['state'].lower() == 'ok'
The problem with the first approach is that, as you say, it wastes codes.
The problem with the second approach is that, at best, you save two characters compared to not (if you use 0 or 1 for on_or_off), and if you use a more idiomatic approach (like on=True or on_or_off="off") you end up using more characters. Further, it results in slower and more complicated code since you need to do anif` test.
The problem with the third approach is in most cases you will also likely be wasting characters compared to just getting the dict value by key manually.

Even if this solution isn't in your propositions, I think the most pythonic way of creating getters is to use properties. As it, you'll be able to know whether the power supply is on or off, but the user will use this property as it was a simple class member:
#property
def state(self):
# Here, get whether the power supply is on or off
# and put it in value
return value
Also, you could create two class constants, PowerSupply.on = True and PowerSupply.off = False, which would make the code easier to understand

The general Pythonic style is to not repeat yourself unnecessarily, so definitely the first method seems pointless because it's actually confusing to follow (you need to notice whether it's on or off)
I'd gravitate most to
get_power_supply_admin_state(dictionary)
gets the admin state from dictionary
return value
And, if I'm reading this correctly, you could go even further.
power_supply_on(dictionary)
return the admin state from dictionary == turned on
This will evaluate to True if it's on and False otherwise, creating the simplest test because then you can run
if power_supply_on(dictionary):

It's more Pythonic to store the dictionary in a class:
class PowerSupply(object):
def __init__(self):
self.state = {'admin': 'FAIL'}
def turn_on(self):
self.state['admin'] = 'OK'
def is_on(self):
return self.state['admin'] == 'OK'
(add more methods as needed)
Then you can use it like this:
ps = PowerSupply()
if not ps.is_on():
# send an alert!

result = is_power_supply_off(state)
result = is_power_supply_on(state)
result = not is_power_supply_on(state) # alternatively, two functions are certainly not needed
I strongly prefer this kind of naming for sake of readability. Let's just consider alternatives, not in function definition but where function is used.
result = is_power_supply_on_or_off(state, True)
pass
result = is_power_supply_on_or_off(state, False)
pass
if get_power_supply_admin_state(state):
pass
if not get_power_supply_admin_state(state):
pass
All of these codes requires map of what True and False means in this context. And to be honest, is not that clear. In many embedded systems 0 means truthy value. What if this function analyses output from system command? 0 (falsy) value is indicator of correct state/execution. In a result, intuitive True means OK is not always valid. Therefore I strongly advice for first option - precisely named function.
Obviously, you'll have some kind of private function like _get_power_supply_state_value(). Both function will call it and manipulate it's output. But point is - it will be hidden inside a module which knows what means what considering power supply state. Is implementation detail and API users does not need to know it.

How should I use #pm.stochastic in PyMC?

Fairly simple question: How should I use #pm.stochastic? I have read some blog posts that claim #pm.stochasticexpects a negative log value:
#pm.stochastic(observed=True)
def loglike(value=data):
# some calculations that generate a numeric result
return -np.log(result)
I tried this recently but found really bad results. Since I also noticed that some people used np.log instead of -np.log, I give it a try and worked much better. What is really expecting #pm.stochastic? I'm guessing there was a small confusion on the sign required due to a very popular example using something like np.log(1/(1+t_1-t_0)) which was written as -np.log(1+t_1-t_0)
Another question: What is this decorator doing with the value argument? As I understand it, we start with some proposed value for the priors that need to enter in the likelihood and the idea of #pm.stochastic is basically produce some number to compare this likelihood to the number generated by the previous iteration in the sampling process. The likelihood should receive the value argument and some values for the priors, but I'm not sure if this is all value is doing because that's the only required argument and yet I can write:
#pm.stochastic(observed=True)
def loglike(value=[1]):
data = [3,5,1] # some data
# some calculations that generate a numeric result
return np.log(result)
And as far as I can tell, that produces the same result as before. Maybe, it works in this way because I added observed=True to the decorator. If I would have tried this in a stochastic variable with observed=False by default, value would be changed in each iteration trying to obtain a better likelihood.

#pm.stochastic is a decorator, so it is expecting a function. The simplest way to use it is to give it a function that includes value as one of its arguments, and returns a log-likelihood.
You should use the #pm.stochastic decorator to define a custom prior for a parameter in your model. You should use the #pm.observed decorator to define a custom likelihood for data. Both of these decorators will create a pm.Stochastic object, which takes its name from the function it decorates, and has all the familiar methods and attributes (here is a nice article on Python decorators).
Examples:
A parameter a that has a triangular distribution a priori:
#pm.stochastic
def a(value=.5):
if 0 <= value < 1:
return np.log(1.-value)
else:
return -np.inf
Here value=.5 is used as the initial value of the parameter, and changing it to value=1 raises an exception, because it is outside of the support of the distribution.
A likelihood b that has is normal distribution centered at a, with a fixed precision:
#pm.observed
def b(value=[.2,.3], mu=a):
return pm.normal_like(value, mu, 100.)
Here value=[.2,.3] is used to represent the observed data.
I've put this together in a notebook that shows it all in action here.

Yes confusion is easy since the #stochastic returns a likelihood which is the opposite of the error essentially. So you take the negative log of your custom error function and return THAT as your log-likelihood.

Does exist built-in function in Python to find best value that satisfies some condition?

Does exist built-in function in Python, that can find best value that satisfies some condition (specified by a funciton)? For example something instead this code:
def argmin(seq, fn):
best = seq[0]; best_score = fn(best)
for x in seq:
x_score = fn(x)
if x_score < best_score:
best, best_score = x, x_score
return best

I presume by 'best' you mean highest, in which case the answer is simple - max().
It takes a key argument, which would be your function.
max(data, key=score)
Naturally, if 'best' means lowest, as DSM points out, then you just want min() instead, it does what it says on the tin.

In general, you want the reduce function, but like #Lattyware said earlier, your specific example, max will suffice.

If all you want is a strict optimization of one parameter, max and min work perfectly fine. That said, most interesting problems (in my opinion) rely on the optimization of at least two parameters at once according to some cost parameter. For any kind of non-linear optimization I would highly recommend scipy's optimize module. Documentation for that can be found here.
SLSQP is the most flexible optimization algorithm, so that may be helpful to you.