I have this tree datatype, where I want to be able to go up and down. But actually, any structure will work, as long as I can get the parent/left child/right child of a node.
Tree = Datatype('Tree')
Tree.declare('nil')
Tree.declare('node', ('label', IntSort()), ('left', Tree), ('up', Tree), ('right', Tree))
Tree = Tree.create()
But I don't know how to fill the datastructure... I thought I could create the tree one node at a time like the following, but that seems to be the wrong way.
trees = [Const('t' + str(i), Tree) for i in range(3)]
Tree.up(trees[0]) = Tree.nil
Tree.left(trees[0]) = trees[1]
Tree.right(trees[0]) = trees[2]
Tree.up(trees[1]) = trees[0]
Tree.up(trees[2]) = trees[0]
Thanks for your help.
I have the feeling that you are slightly mistaken about how such data structures work. Z3's language, and thus also Z3py's, is the language of first-order logic (modulo theories), in which only assumptions/facts exist. One of the major differences with respect to imperative programming languages is, that you don't have destructive updates*, i.e., you cannot update the value of an entity.
In your code you try to build the data structures in an imperative way by essentially manipulating pointers, e. g., by the (attempted) assignment
Tree.up(trees[0]) = Tree.nil
What you probably had in mind was "making trees[0].up point to nil". However, since destructive updates are not part of a first-order language, you instead have to "think functional", i.e., "assume that up(trees[0]) == nil". Adding such assumptions can be done as follows (online on rise4fun):
s = Solver()
s.add(
Tree.up(trees[0]) == Tree.nil,
Tree.left(trees[0]) == trees[1],
Tree.right(trees[0]) == trees[2],
Tree.up(trees[1]) == trees[0],
Tree.up(trees[2]) == trees[0]
)
print s.check() # sat -> your constraints are satisfiable
A consequence of not having destructive updates is that you cannot modify your data structure in a way typical for imperative programming languages. With such updates it would be possible to modify the tree such that trees[1].up.left != trees[1], for example, by the assignment trees[1].up.left = trees[2] (for trees[1] != trees[2]). However, If you add corresponding assumptions
s.add(
trees[1] != trees[2],
Tree.left(Tree.up(trees[1])) == trees[2]
)
print s.check() # unsat
you'll see that your constraints are no longer satisfiable, because the old and new assumptions contradict.
By the way, your definition of the Tree datatype makes it possible to assume that Tree.up(nil) == someTree. Assuming that t1 != t2 are leaves of a tree and that left(t1) == right(t1) == nil and analogous for t2, then you'll have a contradiction if up(nil) == t1 and up(nil) == t2. I don't know how to prevent this on the level of Z3, though.
* Destructive updates could be added as syntactic sugar because they can be turned into assumptions by a so-called passification transformation. This, for example, is done in the intermediate verification language Boogie.
Related
My algorithm needs to modify the children() of the existing logic gate. Suppose i have the following code
a = Bool('a')
b = Bool('b')
c = Bool('c')
or_gate = Or(a, b)
I want to modify or_gate to be Or(a, c).
I have tried the following:
or_gate.children()[1] = c
print(or_gate)
The above code doesn't work, or_gate is still Or(a, b). So how do i change the children of a logic gate in z3?
Edit: i can't create new Or() that contains the children i want due to nested gate. For example consider the following
a = Bool('a')
b = Bool('b')
c = Bool('c')
d = Bool('d')
e = Bool('e')
or_gate_one = Or(a, b)
or_gate_two = Or(or_gate_one, c)
or_gate_three = Or(or_gate_two, d)
If we create new object instead of directly modifying the children of or_gate_one, then or_gate_two and or_gate_three should also be modified, which is horrible for large scale. I need to modify the children of or_gate_one directly.
Use the substitute method:
from z3 import *
a = Bool('a')
b = Bool('b')
c = Bool('c')
or_gate = Or(a, b)
print(or_gate)
or_gate_c = substitute(or_gate, *[(a, c)])
print(or_gate_c)
This prints:
Or(a, b)
Or(c, b)
I wouldn't worry about efficiency, unless you observe it to be a problem down the road: (1) z3 will "share" the internal AST when you use substitute as much as it can, and (2) in most z3 programming, run-time is dominated by solving the constraints, not building them.
If you find that building the constraints is very costly, then you should switch to the lower-level APIs (i.e., C/C++), instead of using Python so you can avoid the extra-level of interpretation in between. But again, only do this if you have run-time evidence that building is the bottleneck, not the solving. (Because in the common case of the latter, switching to C/C++ isn't going to save you much anyhow.)
For my current assignment, I am to establish the stability of intersection/equilibrium points between two nullclines, which I have defined as follows:
def fNullcline(F):
P = (1/k)*((1/beta)*np.log(F/(1-F))-c*F+v)
return P
def pNullcline(P):
F = (1/delta)*(pD-alpha*P+(r*P**2)/(m**2+P**2))
return F
I also have a method "stability" that applies the Hurwitz criteria on the underlying system's Jacobian:
def dPdt(P,F):
return pD-delta*F-alpha*P+(r*P**2)/(m**2+P**2)
def dFdt(P,F):
return s*(1/(1+sym.exp(-beta*(-v+c*F+k*P)))-F)
def stability(P,F):
x = sym.Symbol('x')
ax = sym.diff(dPdt(x, F),x)
ddx = sym.lambdify(x, ax)
a = ddx(P)
# shortening the code here: the same happens for b, c, d
matrix = [[a, b],[c,d]]
eigenvalues, eigenvectors = np.linalg.eig(matrix)
e1 = eigenvalues[0]
e2 = eigenvalues[1]
if(e1 >= 0 or e2 >= 0):
return 0
else:
return 1
The solution I was looking for was later provided. Basically, values became too small! So this code was added to make sure no too small values are being used for checking the stability:
set={0}
for j in range(1,210):
for i in range(1,410):
x=i*0.005
y=j*0.005
x,y=fsolve(System,[x,y])
nexist=1
for i in set:
if(abs(y-i))<0.00001:
nexist=0
if(nexist):
set.add(y)
set.discard(0)
I'm still pretty new to coding so the function in and on itself is still a bit of a mystery to me, but it eventually helped in making the little program run smoothly :) I would again like to express gratitude for all the help I have received on this question. Below, there are still some helpful comments, which is why I will leave this question up in case anyone might run into this problem in the future, and can find a solution thanks to this thread.
After a bit of back and forth, I came to realise that to avoid the log to use unwanted values, I can instead define set as an array:
set = np.arange(0, 2, 0.001)
I get a list of values within this array as output, complete with their according stabilities. This is not a perfect solution as I still get runtime errors (in fact, I now get... three error messages), but I got what I wanted out of it, so I'm counting that as a win?
Edit: I am further elaborating on this in the original post to improve the documentation, however, I would like to point out again here that this solution does not seem to be working, after all. I was too hasty! I apologise for the confusion. It's a very rocky road for me. The correct solution has since been provided, and is documented in the original question.
I'm trying to develop an equation parser using a compiler approach in Python. The main issue that I encounter is that it is more likely that I don't have all the variables and need therefore to look for sub-formulas. Let's show an example that is worth a thousand words ;)
I have four variables whom I know the values: vx, vy, vz and c:
list_know_var = ['vx', 'vy', 'vz', 'c']
and I want to compute the Mach number (M) defined as
equation = 'M = V / c'
I already know the c variable but I don't know V. However, I know that the velocity V that can be computed using the vx, vy and vz and this is stored in a dictionary with other formulas (here only one sub formula is shown)
know_equations = {'V': '(vx ** 2 + vy ** 2 + vz ** 2) ** 0.5'}
Therefore, what I need is to parse the equation and check if I have all the variables. If not, I shall look into the know_equations dictionary to see if an equation is defined for it and this recursively until my equation is well defined.
For now on, I have been able using the answer given here to parse my equation and know if I know all the variables. The issue is that I did not find a way to replace the unknown variable (here V) by its expression in know_equation:
parsed_equation = compiler.parse(equation)
list_var = re.findall("Name\(\'(\w*)\'\)", str(parsed_equation.getChildNodes()[0]))
list_unknow_var = list(set(list_var) - set(list_know_var))
for var in list_unknow_var:
if var in know_equations:
# replace var in equation by its expression given in know_equations
# and repeate until all the variables are known or raise Error
pass
Thank you in advance for your help, much appreciate!
Adrien
So i'm spitballing a bit, but here goes.
The compiler.parse function returns an instance of compiler.ast.Module which contains an abstract syntax tree. You can traverse this instance using the getChildNodes method. By recursively examining the left and right attributes of the nodes as you traverse the tree you can isolate compiler.ast.Name instances and swap them out for your substitution expressions.
So a worked example might be:
import compiler
def recursive_parse(node,substitutions):
# look for left hand side of equation and test
# if it is a variable name
if hasattr(node.left,"name"):
if node.left.name in substitutions.keys():
node.left = substitutions[node.left.name]
else:
# if not, go deeper
recursive_parse(node.left,substitutions)
# look for right hand side of equation and test
# if it is a variable name
if hasattr(node.right,"name"):
if node.right.name in substitutions.keys():
node.right = substitutions[node.right.name]
else:
# if not, go deeper
recursive_parse(node.right,substitutions)
def main(input):
substitutions = {
"r":"sqrt(x**2+y**2)"
}
# each of the substitutions needs to be compiled/parsed
for key,value in substitutions.items():
# this is a quick ugly way of getting the data of interest
# really this should be done in a programatically cleaner manner
substitutions[key] = compiler.parse(substitutions[key]).getChildNodes()[0].getChildNodes()[0].getChildNodes()[0]
# compile the input expression.
expression = compiler.parse(input)
print "Input: ",expression
# traverse the selected input, here we only pass the branch of interest.
# again, as with above, this done quick and dirty.
recursive_parse(expression.getChildNodes()[0].getChildNodes()[0].getChildNodes()[1],substitutions)
print "Substituted: ",expression
if __name__ == "__main__":
input = "t = r*p"
main(input)
I have admittedly only tested this on a handful of use cases, but I think the basis is there for a generic implementation that can handle a wide variety of inputs.
Running this, I get the output:
Input: Module(None, Stmt([Assign([AssName('t', 'OP_ASSIGN')], Mul((Name('r'), Name('p'))))]))
Substituted: Module(None, Stmt([Assign([AssName('t', 'OP_ASSIGN')], Mul((CallFunc(Name('sqrt'), [Add((Power((Name('x'), Const(2))), Power((Name('y'), Const(2)))))], None, None), Name('p'))))]))
EDIT:
So the compiler module is depreciated in Python 3.0, so a better (and cleaner) solution would be to use the ast module:
import ast
from math import sqrt
# same a previous recursion function but with looking for 'id' not 'name' attribute
def recursive_parse(node,substitutions):
if hasattr(node.left,"id"):
if node.left.id in substitutions.keys():
node.left = substitutions[node.left.id]
else:
recursive_parse(node.left,substitutions)
if hasattr(node.right,"id"):
if node.right.id in substitutions.keys():
node.right = substitutions[node.right.id]
else:
recursive_parse(node.right,substitutions)
def main(input):
substitutions = {
"r":"sqrt(x**2+y**2)"
}
for key,value in substitutions.items():
substitutions[key] = ast.parse(substitutions[key], mode='eval').body
# As this is an assignment operation, mode must be set to exec
module = ast.parse(input, mode='exec')
print "Input: ",ast.dump(module)
recursive_parse(module.body[0].value,substitutions)
print "Substituted: ",ast.dump(module)
# give some values for the equation
x = 3
y = 2
p = 1
code = compile(module,filename='<string>',mode='exec')
exec(code)
print input
print "t =",t
if __name__ == "__main__":
input = "t = r*p"
main(input)
This will compile the expression and execute it in the local space. The output should be:
Input: Module(body=[Assign(targets=[Name(id='t', ctx=Store())], value=BinOp(left=Name(id='r', ctx=Load()), op=Mult(), right=Name(id='p', ctx=Load())))])
Substituted: Module(body=[Assign(targets=[Name(id='t', ctx=Store())], value=BinOp(left=Call(func=Name(id='sqrt', ctx=Load()), args=[BinOp(left=BinOp(left=Name(id='x', ctx=Load()), op=Pow(), right=Num(n=2)), op=Add(), right=BinOp(left=Name(id='y', ctx=Load()), op=Pow(), right=Num(n=2)))], keywords=[], starargs=None, kwargs=None), op=Mult(), right=Name(id='p', ctx=Load())))])
t = r*p
t = 3.60555127546
I am implementing an acoustic feature extraction system in python, and I need to implement a makefile-style algorithm to ensure that all blocks in the feature extraction system are run in the correct order, and without repeating any feature extractions stages.
The input to this feature extraction system will be a graph detailing the links between the feature extraction blocks, and I'd like to work out which functions to run when based upon the graph.
An example of such a system might be the following:
,-> [B] -> [D] ----+
input --> [A] ^ v
`-> [C] ----+---> [E] --> output
and the function calls (assuming each block X is a function of the form output = X(inputs) might be something like:
a = A(input)
b = B(a)
c = C(a)
d = D(b,c) # can't call D until both b and c are ready
output = E(d,c) # can't call E until both c and d are ready
I already have the function graph loaded in the form of a dictionary with each dictionary entry of the form (inputs, function) like so:
blocks = {'A' : (('input'), A),
'B' : (('A'), B),
'C' : (('A'), C),
'D' : (('B','C'), D),
'output' : (('D','C'), E)}
I'm just currently drawing a blank on what the makefile algorithm does exactly, and how to go about implementing it. My google-fu appears to be not-very-helpful here too. If someone at least can give me a pointer to a good discussion of the makefile algorithm that would probably be a good start.
Topological sorting
blocks is basically an adjacency list representation of the (acyclic) dependency graph. Hence, to get the correct order to process the blocks, you can perform a topological sort.
As the other helpful answerers have already pointed out, what I'm after is a topological sort, but I think my particular case is a little simpler because the function graph must always start at input and end at output.
So, here is what I ended up doing (I've edited it slightly to remove some context-dependent stuff, so it may not be completely correct):
def extract(func_graph):
def _getsignal(block,signals):
if block in signals:
return signals[block]
else:
inblocks, fn = func_graph[block]
inputs = [_getsignal(i,signals) for i in inblocks]
signals[block] = fn(inputs)
return signals[block]
def extract_func (input):
signals = dict(input=input)
return _getsignal('output',signals)
return extract_func
So now given I can set up the function with
fn = extract(blocks)
And use it as many times as I like:
list_of_outputs = [fn(i) for i in list_of_inputs]
I should probably also put in some checks for loops, but that is a problem for another day.
For code in many languages, including Python try these Rosetta code links: Topological sort, and Topological sort/Extracted top item.
I'm rewriting a data-driven legacy application in Python. One of the primary tables is referred to as a "graph table", and does appear to be a directed graph, so I was exploring the NetworkX package to see whether it would make sense to use it for the graph table manipulations, and really implement it as a graph rather than a complicated set of arrays.
However I'm starting to wonder whether the way we use this table is poorly suited for an actual graph manipulation library. Most of the NetworkX functionality seems to be oriented towards characterizing the graph itself in some way, determining shortest distance between two nodes, and things like that. None of that is relevant to my application.
I'm hoping if I can describe the actual usage here, someone can advise me whether I'm just missing something -- I've never really worked with graphs before so this is quite possible -- or if I should be exploring some other data structure. (And if so, what would you suggest?)
We use the table primarily to transform a user-supplied string of keywords into an ordered list of components. This constitutes 95% of the use cases; the other 5% are "given a partial keyword string, supply all possible completions" and "generate all possible legal keyword strings". Oh, and validate the graph against malformation.
Here's an edited excerpt of the table. Columns are:
keyword innode outnode component
acs 1 20 clear
default 1 100 clear
noota 20 30 clear
default 20 30 hst_ota
ota 20 30 hst_ota
acs 30 10000 clear
cos 30 11000 clear
sbc 10000 10199 clear
hrc 10000 10150 clear
wfc1 10000 10100 clear
default 10100 10101 clear
default 10101 10130 acs_wfc_im123
f606w 10130 10140 acs_f606w
f550m 10130 10140 acs_f550m
f555w 10130 10140 acs_f555w
default 10140 10300 clear
wfc1 10300 10310 acs_wfc_ebe_win12f
default 10310 10320 acs_wfc_ccd1
Given the keyword string "acs,wfc1,f555w" and this table, the traversal logic is:
Start at node 1; "acs" is in the string, so go to node 20.
None of the presented keywords for node 20 are in the string, so choose the default, pick up hst_ota, and go to node 30.
"acs" is in the string, so go to node 10000.
"wfc1" is in the string, so go to node 10100.
Only one choice; go to node 10101.
Only one choice, so pick up acs_wfc_im123 and go to node 10130.
"f555w" is in the string, so pick up acs_f555w and go to node 10140.
Only one choice, so go to node 10300.
"wfc1" is in the string, so pick up acs_wfc_ebe_win12f and go to node 10310.
Only one choice, so pick up acs_wfc_ccd1 and go to node 10320 -- which doesn't exist, so we're done.
Thus the final list of components is
hst_ota
acs_wfc_im123
acs_f555w
acs_wfc_ebe_win12f
acs_wfc_ccd1
I can make a graph from just the innodes and outnodes of this table, but I couldn't for the life of me figure out how to build in the keyword information that determines which choice to make when faced with multiple possibilities.
Updated to add examples of the other use cases:
Given a string "acs", return ("hrc","wfc1") as possible legal next choices
Given a string "acs, wfc1, foo", raise an exception due to an unused keyword
Return all possible legal strings:
cos
acs, hrc
acs, wfc1, f606w
acs, wfc1, f550m
acs, wfc1, f555w
Validate that all nodes can be reached and that there are no loops.
I can tweak Alex's solution for the first two of these, but I don't see how to do it for the last two.
Definitely not suitable for general purpose graph libraries (whatever you're supposed to do if more than one of the words meaningful in a node is in the input string -- is that an error? -- or if none does and there is no default for the node, as for node 30 in the example you supply). Just write the table as a dict from node to tuple (default stuff, dict from word to specific stuff) where each stuff is a tuple (destination, word-to-add) (and use None for the special "word-to-add" clear). So e.g.:
tab = {1: (100, None), {'acs': (20, None)}),
20: ((30, 'hst_ota'), {'ota': (30, 'hst_ota'), 'noota': (30, None)}),
30: ((None, None), {'acs': (10000,None), 'cos':(11000,None)}),
etc etc
Now handling this table and an input comma-separated string is easy, thanks to set operations -- e.g.:
def f(icss):
kws = set(icss.split(','))
N = 1
while N in tab:
stuff, others = tab[N]
found = kws & set(others)
if found:
# maybe error if len(found) > 1 ?
stuff = others[found.pop()]
N, word_to_add = stuff
if word_to_add is not None:
print word_to_add
Adding an answer to respond to the further requirements newly edited in...: I still wouldn't go for a general-purpose library. For "all nodes can be reached and there are no loops", simply reasoning in terms of sets (ignoring the triggering keywords) should do: (again untested code, but the general outline should help even if there's some typo &c):
def add_descendants(someset, node):
"auxiliary function: add all descendants of node to someset"
stuff, others = tab[node]
othernode, _ = stuff
if othernode is not None:
someset.add(othernode)
for othernode, _ in others.values():
if othernode is not None:
someset.add(othernode)
def islegal():
"Return bool, message (bool is True for OK tab, False if not OK)"
# make set of all nodes ever mentioned in the table
all_nodes = set()
for node in tab:
all_nodes.add(node)
add_desendants(all_nodes, node)
# check for loops and connectivity
previously_seen = set()
currently_seen = set([1])
while currently_seen:
node = currently_seen.pop()
if node in previously_seen:
return False, "loop involving node %s" % node
previously_seen.add(node)
add_descendants(currently_seen, node)
unreachable = all_nodes - currently_seen
if unreachable:
return False, "%d unreachable nodes: %s" % (len(unreachable), unreachable)
else:
terminal = currently_seen - set(tab)
if terminal:
return True, "%d terminal nodes: %s" % (len(terminal), terminal)
return True, "Everything hunky-dory"
For the "legal strings" you'll need some other code, but I can't write it for you because I have not yet understood what makes a string legal or otherwise...!