We Keep Coding

Passing some values as variables

I'm a physics graduate student with some basic knowledge of Python and I'm facing some problems that challenge my abilities.
I'm trying to pass some variables as dummies and some not. I have a function that receives a function as the first argument, but I need that some values to be declared "a posteriori".
What I'm trying to mean is the following:
lead0 = add_leads(lead_shape_horizontal(W, n), (0, 0, n), sym0)
The function "add_leads" takes some function as well as a tuple and a third argument which is fine. But n hasn't any definition yet. I want that n has an actual sense when it enters "add_leads".
Here is the actual function add_leads
def add_leads(shape, origin_2D, symm):
lead_return = []
lead_return_reversed = []
for m in range(L):
n = N_MIN + m
origin_3D = list(origin_2D)+[n]
lead_return.append(kwant.Builder(symm))
lead_return[m][red.shape(shape(n), tuple(origin_3D))] = ONN + HBAR*OMEGA*n
lead_return[m][[kwant.builder.HoppingKind(*hopping) for
hopping in hoppings_leads]] = HOPP
lead_return[m].eradicate_dangling()
Note that n is defined under for, so, I wish to put the value of n in shape(n) (in this case leads_shape_horizontal with a fixed value for W, not for n).
I need this to be this way because eventually the function which is the argument for lead_shape might have more than 2 input values but still just need to vary n
Can I achieve this in Python? If I can, How to do so?
Help will be really appreciated.
Sorry for my english!
Thanks in advance

You probably should pass in the function lead_shape_horizontal, not the function with argument lead_shape_horizontal(W, n)
Because the latter one will return the result of the function, not function object itself. Unless the return value is also a function, you'll get an error when you later call shape(n), which is identical to lead_shape_horizontal(W, n)(n)
As for providing a fix value for W but not for n, you can either give W a default value in the function or just don't make it an argument
For example,
def lead_shape_horizontal(n, W=some_value):
# do stuff
or If you always fix W, then it doesn't have to be an argument
def lead_shape_horizontal(n):
W = some_value
# do stuff
Also note that you didn't define n when calling function, so you can't pass in n to the add_leads function.
Maybe you have to construct the origin_2D inside the function
like origin_2D = origin_2D + (n,)
Then you can call the function like this lead0 = add_leads(lead_shape_horizontal, (0, 0), sym0)
See Python Document to understand how default value works.
Some advice: Watch out the order of arguments when you're using default value.
Also watch out when you're passing in mutable object as default value. This is a common gotcha

Using string as literal expression in function argument in Python

Let's say I have a function that can take various kinds of parameter values, but I don't want to (as a constraint) pass arguments explicitly. Instead, I want to pass them as a string.:
def func(param)
return param+param
a = 'param=4'
func(<do something to a>(a))
>>8
Is this possible in python?
I want to use this idea in Django to create Query filters based on GET parameters in a dictionary and then just chain them using their keys.
lookup_dic = {'user': 'author=user',
'draft': 'Q(publish_date_lte=timezone.now())|
Q(publish_date_isnull=True)'}
Based on whether the user and draft keywords are passed in the GET parameters, this would be read out like:
queryset.objects.filter(author=user).filter(Q(publish_date_lte=timezone.now())|
Q(publish_date_isnull=True))
I understand that I can do this by replacing the author=user by Q(author__name=user), but I wanted to know if this string comprehension feature is implemented in python in general?

Use eval
def func(param=0):
return param+param
a = 'param=4'
eval('func(' + a +')')

Are you looking for this?
def func(param):
return param + param
a = 'param=4'
parameter, value = a.split("=")
print(func(**{parameter: int(value)}))
# >> 8

list of functions with parameters

I need to obtain a list of functions, where my function is defined as follows:
import theano.tensor as tt
def tilted_loss(y,f,q):
e = (y-f)
return q*tt.sum(e)-tt.sum(e[e<0])
I attempted to do
qs = np.arange(0.05,1,0.05)
q_loss_f = [tilted_loss(q=q) for q in qs]
however, get the error TypeError: tilted_loss() missing 2 required positional arguments: 'y' and 'f'. I attempted the simpler a = tilted_loss(q=0.05) with the same result.
How do you go about creating this list of functions when parameters are required? Similar questions on SO consider the case where parameters are not involved.

You can use functools.partial:
q_loss_f = [functools.partial(tilted_loss, q=q) for q in qs]

There are 2 ways you can solve this problem. Both ways require you know the default values for y and f.
With the current function, there's simply no way for the Python interpreter to know the value of y and f when you call tilted_loss(q=0.05). y and f are simply undefined & unknown.
Solution (1): Add default values
We can fix this by adding default values for the function, for example, if default values are: y = 0, f = 1:
def tilted_loss(q, y=0, f=1):
# original code goes here
Note that arguments with default values have to come AFTER non-default arguments (i.e q).
Solution (2): Specify default values during function call
Alternatively, just specify the default values every time you call that function. (Solution 1 is better)

How to correctly call function with optional parameters in python

I'm a beginner with python and I'm facing a problem with a function that requires optional parameters.
This function gets as parameters a variable number of file paths, that can be from 2 to n parameters.
After that, a certain number of optional parameters can be passed to this function.
I tried to do something like that:
def compareNfilesParameters(*args):
start_time = time.time()
listFiles = []
listParameters = []
for argument in args:
if str(argument).endswith(".vcf"):
listFiles.append(str(argument))
else:
listParameters.append(argument)
So if the parameters has the file extension it is considered as one of the file path parameters, the others are seen as the optional parameters.
What I want to do is letting the user call the function like:
function('a.vcf', 'b.vcf', 'c.vcf')
or
function('a.vcf', 'b.vcf', 'c.vcf', 0, 1)
or
function('a.vcf', 'b.vcf', 'c.vcf', 0, 1, 4,...,3)
I tried different approaches but none of them satisfies me.
The first approach is declaring the function as:
def compareNfilesParameters(*args)
but this way, if I get for example 3 parameters, 2 will certainly be the files path, and the last one I don't know on which variable it refers. So I need to specify every value and pass '-1' for the parameters that I want to use default value.
The 2nd approach is the following:
def compareNfilesParameters(*args, par1 = 10, par2 = 15 ..)
But this way I need to call the function like:
compareNfilesParameters(path1, path2, path3, par1 = 10)
and not like
compareNfilesParameters(path1, path2, path3, 10)
or the 10 will be considered in the args input, right? I wouldn't like to use this approach because it becomes very verbose to call the function.
How would you do this?

Make the user pass in the filenames as a sequence; don't try to cram everything into separate arguments:
def compareNfilesParameters(files, *params):
and call this as:
compareNfilesParameters(('a.vcf', 'b.vcf', 'c.vcf'), 0, 1, 4)
This makes the files explicit and removes the need to separate files from other parameters.
If your remaining parameters are distinct options (and not a homogenous series of integers), I'd use keyword arguments:
def compareNfilesParameters(files, op1=default_value, op2=default_value, op3=default_value):
You don't have to use keyword arguments with keywords when calling; you can still treat them as positional:
compareNfilesParameters(('a.vcf', 'b.vcf', 'c.vcf'), 0, 1, 4)
would give op1 the value 0, op2 the value 1, and op3 the value 4. Only if you want to specify values out of order or for a specific option do you have to use keyword arguments in the call:
compareNfilesParameters(('a.vcf', 'b.vcf', 'c.vcf'), op3=4)

Ok, I solved like using the keyword parameters as suggested.
def compareNfilesParameters(listFiles, **kwargs):
start_time = time.time()
if len(listFiles) < MINUMUM_FILES_NUMBER :
print "You need to specify at least "+ str(MINUMUM_FILES_NUMBER) +" files."
return
try:
operationType = int(kwargs.get("op", DEFAULT_OPERATION_TYPE))
except ValueError:
print "Operation type filter has to be an integer."
return
if operationType not in [0,1]:
print "Operation type must be 0 (intersection), 1 (union)"
return
and so on for all the parameters.
Like this I need to put all the files paths in a list and pass it as a single required parameter, and searching kwargs dictionary for optionals parameters setting the default values if not expressed.

Need help understanding function passing in Python

I am trying to teach myself Python by working through some problems I came up with, and I need some help understanding how to pass functions.
Let's say I am trying to predict tomorrow's temperature based on today's and yesterday's temperature, and I have written the following function:
def predict_temp(temp_today, temp_yest, k1, k2):
return k1*temp_today + k2*temp_yest
And I have also written an error function to compare a list of predicted temperatures with actual temperatures and return the mean absolute error:
def mean_abs_error(predictions, expected):
return sum([abs(x - y) for (x,y) in zip(predictions,expected)]) / float(len(predictions))
Now if I have a list of daily temperatures for some interval in the past, I can see how my prediction function would have done with specific k1 and k2 parameters like this:
>>> past_temps = [41, 35, 37, 42, 48, 30, 39, 42, 33]
>>> pred_temps = [predict_temp(past_temps[i-1],past_temps[i-2],0.5,0.5) for i in xrange(2,len(past_temps))]
>>> print pred_temps
[38.0, 36.0, 39.5, 45.0, 39.0, 34.5, 40.5]
>>> print mean_abs_error(pred_temps, past_temps[2:])
6.5
But how do I design a function to minimize my parameters k1 and k2 of my predict_temp function given an error function and my past_temps data?
Specifically I would like to write a function minimize(args*) that takes a prediction function, an error function, some training data, and that uses some search/optimization method (gradient descent for example) to estimate and return the values of k1 and k2 that minimize my error given the data?
I am not asking how to implement the optimization method. Assume I can do that. Rather, I would just like to know how to pass my predict and error functions (and my data) to my minimize function, and how to tell my minimize function that it should optimize the parameters k1 and k2, so that my minimize function can automatically search a bunch of different settings of k1 and k2, applying my prediction function with those parameters each time to the data and computing error (like I did manually for k1=0.5 and k2=0.5 above) and then return the best results.
I would like to be able to pass these functions so I can easily swap in different prediction and error functions (differing by more than just parameter settings that is). Each prediction function might have a different number of free parameters.
My minimize function should look something like this, but I don't know how to proceed:
def minimize(prediction_function, which_args_to_optimize, error_function, data):
# 1: guess initial parameters
# 2: apply prediction function with current parameters to data to compute predictions
# 3: use error function to compute error between predictions and data
# 4: if stopping criterion is met, return parameters
# 5: update parameters
# 6: GOTO 2
Edit: It's that easy?? This is no fun. I am going back to Java.
On a more serious note, I think I was also getting hung up on how to use different prediction functions with different numbers of parameters to tune. If I just take all the free parameters in as one tuple I can keep the form of the function the same so it easy to pass and use.

Here is an example of how to pass a function into another function. apply_func_to will take a function f and a number num as parameters and return f(num).
def my_func(x):
return x*x
def apply_func_to(f, num):
return f(num)
>>>apply_func_to(my_func, 2)
4
If you wanna be clever you can use lambda (anonymous functions too). These allow you to pass functions "on the fly" without having to define them separately
>>>apply_func_to(lambda x:x*x, 3)
9
Hope this helps.

Function passing in Python is easy, you just use the name of the function as a variable which contains the function itself.
def predict(...):
...
minimize(predict, ..., mean_abs_error, ...)
As for the rest of the question: I'd suggest looking at the way SciPy implements this as a model. Basically, they have a function leastsq which minimizes the sum of the squares of the residuals (I presume you know what least-squares minimization is ;-). What you pass to leastsq is a function to compute the residuals, initial guesses for the parameters, and an arbitrary parameter which gets passed on to your residual-computing function (the closure), which includes the data:
# params will be an array of your k's, i.e. [k1, k2]
def residuals(params, measurements, times):
return predict(params, times) - measurements
leastsq(residuals, initial_parameters, args = (measurements, times))
Note that SciPy doesn't actually concern itself with how you come up with the residuals. The measurements array is just passed unaltered to your residuals function.
I can look up an example I did recently if you want more information - or you can find examples online, of course, but in my experience they're not quite as clear. The particular bit of code I wrote would relate well to your scenario.

As David and and Il-Bhima note, functions can be passed into other functions just like any other type of object. When you pass a function in, you simply call it like you ordinarily would. People sometimes refer to this ability by saying that functions are first class in Python. At a slightly greater level of detail, you should think of functions in Python as being one type of callable object. Another important type of callable object in Python is class objects; in this case, calling a class object creates an instance of that object. This concept is discussed in detail here.
Generically, you will probably want to leverage the positional and/or keyword argument feature of Python, as described here. This will allow you to write a generic
minimizer that can minimize prediction functions taking different sets of parameters. I've written an example---it's more complicated than I'd like (uses generators!) but it works for prediction functions with arbitrary parameters. I've glossed over a few details, but this should get you started:
def predict(data, k1=None, k2=None):
"""Make the prediction."""
pass
def expected(data):
"""Expected results from data."""
pass
def mean_abs_err(pred, exp):
"""Compute mean absolute error."""
pass
def gen_args(pred_args, args_to_opt):
"""Update prediction function parameters.
pred_args : a dict to update
args_to_opt : a dict of arguments/iterables to apply to pred_args
This is a generator that updates a number of variables
over a given numerical range. Equivalent to itertools.product.
"""
base_args = pred_args.copy() #don't modify input
argnames = args_to_opt.keys()
argvals = args_to_opt.values()
result = [[]]
# Generate the results
for argv in argvals:
result = [x+[y] for x in result for y in argv]
for prod in result:
base_args.update(zip(argnames, prod))
yield base_args
def minimize(pred_fn, pred_args, args_to_opt, err_fn, data):
"""Minimize pred_fn(data) over a set of parameters.
pred_fn : function used to make predictions
pred_args : dict of keyword arguments to pass to pred_fn
args_to_opt : a dict of arguments/iterables to apply to pred_args
err_fn : function used to compute error
data : data to use in the optimization
Returns a tuple (error, parameters) of the best set of input parameters.
"""
results = []
for new_args in gen_args(pred_args, args_to_opt):
pred = pred_fn(data, **new_args) # Unpack dictionary
err = err_fn(pred, expected(data))
results.append((err, new_args))
return sorted(results)[0]
const_args = {k1: 1}
opt_args = {k2: range(10)}
data = [] # Whatever data you like.
minimize(predict, const_args, opt_args, mean_abs_err, data)