My programs are growing larger and more sophisticated. As a result I am using more and more functions. My question is, should I "fetch" a value from a function once and then "tote" it around, sending it into other functions as a parameter, or just call, "fetch," the value again from within the other function(s)?
I am sure resources, and speed, are a factor, but what is the general rule, if any?
For example, should I call my sigmoid function, and then use that value as a parameter in a call to the next function that uses it, or just call the sigmoid function again from within that next function?
I know that this question borders on opinion, but I did not attend a CS school, and so find myself wondering what the "norm" for some things are.
Thanks.
You are correct that this question relates more to software engineering theory than just a language (Python). There are programming paradigms which promote one variant over the other but the most general rule of thumb you should aim for is:
High cohesion and low coupling
i.e., within a module of software (which roughly corresponds to a Python module, if you are using them), the functions should have dependence on each other and you should call them to fetch the value. However, across modules, you should not have functional calls and should tie them up at a higher level module (or the main function) by fetching values from one module and passing it to the other.
See also: Memoization.
Related
Question is as above. After reading the documentation, I can change the integrator itself (RK45,RK23, DOP853, etc), however I cannot find information on the order of these integrators, or on ways to limit the integrator to 1st order.
How can this be done? Do I have to use a particular ODE solver method that is by default 1st order, or can I edit any method to be 1st order?
For many integrators, the order is a fixed property. There are some methods – let’s call them meta integrators – that switch between different integrators, but they are still limited to the order of these integrators. Thus, you cannot simply control the order of the integrator and leave everything else the same.
If you really want a first-order method, it’s easy to implement the Euler method – unless you want step-size adaption.
Mind that the order of an integrator denotes how its error behaves for small step sizes. In this respect, a higher order is nothing that should cause a problem per se. I would therefore find it remarkable if using a first-order method solves a problem. Sometimes, individual methods can have problems or the problem is stiff, but here the solution is to use another solver (for stiff problems), not a first-order solver. If you consistently observe a result for all solvers, it is by far more likely that this your true result or you made a mistake defining your derivative or similar.
I have a Python function where I want to find the global minimum.
I am looking for a resource on how to choose the appropriate algorithm for it.
When looking at scipy.optimize.minimize documentation, I can find some not very specific statements like
has proven good performance even for non-smooth optimizations
Suitable for large-scale problems
recommended for medium and large-scale problems
Apart from that I am unsure whether for my function basinhopping might be better or even bayesian-optimization. For the latter I am unsure whether that is only meant for machine learning cost functions or can be used generally.
I am looking for a cheat sheet like
but just for minimization problems.
Does something like that already exist?
If not, how would I be able to choose the most appropriate algorithm based on my constraints (that are: time consuming to compute, 4 input variables, known boundaries)?
I have read and studied the TFF guide and APIs pages precisely. But I am confused about the usage of the functions and how to control them.
For example, in tutorials, there is a function that is responsible for aggregating metrics.
#tff.federated_computation
def aggregate_mnist_metrics_across_clients(metrics):
return {
'num_examples': tff.federated_sum(metrics.num_examples),
'loss': tff.federated_mean(metrics.loss, metrics.num_examples),
'accuracy':tff.federated_mean(metrics.accuracy,metrics.num_examples)
}
It is called in the MODEL class. But I need to have access to the elements of the metric after it is called in the class. I want to modify the metrics after it is called in the model and call them in other functions.
However, for example, I can not call them (e.g. with tff.Type such as .type_signature, since it needs namedTuple for __getattr__). And I did not understand the total intuitive behind the concept of how they can be used in other function's bodies of the code?
In TFF, I expect every function has a placement in either the server or clients side, but both of them can be accessible in any function which makes it confusing. Who is responsible for calculating? #CLIENT or #SERVER?
could anyone help me?
Perhaps one misconception is that tff.learning.Model is the interface used by the tff.learning module and is not required if not any module methods such as tff.learning.build_federated_averaging_process(). Currently the federated averaging implementation does not have a hook to modify the metrics post aggregation but before returning from the computation.
I highly recommend looking Custom Federated Algorithms, Part 2: Implementing Federated Averaging, which steps through how to implement Federated Averaging without using tff.learning, which would allow extending the computations in any direction desired.
Additional clarification:
aggregate_mnist_metrics_across_clients is being returned in the tff.learning.Model.federated_output_computation method of class MnistModel, a few cells down in the tutorial. The metrics parameter is the return value of tff.learning.Model.report_local_outputs(). Both of these methods are interfaces that need to be implemented for the tff.learning module methods (e.g. tff.learning.build_federated_averaging_process()) to connect up the computation in the correct way.
In TFF, I expect every function has a placement in either the server or clients side, but both of them can be accessible in any function which makes it confusing.
In TFF the data has placement, not the computations. Computations that accept unplaced values can be used on placed values using TFF intrinsics (e.g. tff.federated_mean(), or tff.federated_map()). Also, a TFF computation may accept value placed at CLIENTS or SERVER and also return values placed at either.
If you haven't already, I'd highly recommend looking at the two part tutorial Custom Federated Algorithms. It spends time introducing the programming and execution model of TFF:
Part 1: Introduction to the Federated Core
Part 2: Implementing Federated Averaging
I want to map a multi-step function onto a set of arrays. The multi-step (parent) function is composed of several helper functions. Currently, I am using map(parentFunction, argArrays**). The parent function and the helper functions all take in individual floats. Would it be more computationally efficient to instead make the parent function take in the arrays, and use map(helperFunction_i, someArgArrays**)?
I'm struggling here because I don't fully understand what's going on under the hood here. And while a fractionally small speed increase may not matter, an order of magnitude speed increase may matter because the parent function here is going to be nested inside a monte carlo loop, that monte carlo loop as a whole may be called thousands of times for a high-level function evaluation of a model I'm building, and I expect to need thousands of those high-level function evaluations for my research purposes.
While I'd most appreciate a direct and unambiguous answer to my question if an unambiguous answer exists, I'd also appreciate any recommended readings that would help me to understand how to write efficient code in python.
I have a multivariate optimization problem that I want to run. Each evaluation is quite slow. So obviously the ability to thread it out to multiple machines would be quite nice. I have no trouble writing the code to dispatch jobs to other machines. However, scipy.optimize.minimize calls each evaluation call sequentially; it won't give me another set of parameters to evaluate until the previous one returns.
Now, I know that the "easy" solution to this would be "run your evaluation task in a parallel manner - break it up". Indeed, while it is possible to do this to some extent, it only goes so far; the bandwidth rises the more you split it up until splitting it up further actually starts to slow you down. Having another axis in which to parallelize - aka, the minimization function itself - would greatly increase scaleability.
Is there no way to do this with scipy.optimize.minimize? Or any other utility that performs in a roughly functionally equivalent manner (trying to find as low of a minimum as possible)? Surely it's possible for a minimization utility to make use of parallelism, particularly on a multivariate optimization problem where there's many axes whose gradients relative to each other at given datapoints need to be examined.