I have read and studied the TFF guide and APIs pages precisely. But I am confused about the usage of the functions and how to control them.
For example, in tutorials, there is a function that is responsible for aggregating metrics.
#tff.federated_computation
def aggregate_mnist_metrics_across_clients(metrics):
return {
'num_examples': tff.federated_sum(metrics.num_examples),
'loss': tff.federated_mean(metrics.loss, metrics.num_examples),
'accuracy':tff.federated_mean(metrics.accuracy,metrics.num_examples)
}
It is called in the MODEL class. But I need to have access to the elements of the metric after it is called in the class. I want to modify the metrics after it is called in the model and call them in other functions.
However, for example, I can not call them (e.g. with tff.Type such as .type_signature, since it needs namedTuple for __getattr__). And I did not understand the total intuitive behind the concept of how they can be used in other function's bodies of the code?
In TFF, I expect every function has a placement in either the server or clients side, but both of them can be accessible in any function which makes it confusing. Who is responsible for calculating? #CLIENT or #SERVER?
could anyone help me?
Perhaps one misconception is that tff.learning.Model is the interface used by the tff.learning module and is not required if not any module methods such as tff.learning.build_federated_averaging_process(). Currently the federated averaging implementation does not have a hook to modify the metrics post aggregation but before returning from the computation.
I highly recommend looking Custom Federated Algorithms, Part 2: Implementing Federated Averaging, which steps through how to implement Federated Averaging without using tff.learning, which would allow extending the computations in any direction desired.
Additional clarification:
aggregate_mnist_metrics_across_clients is being returned in the tff.learning.Model.federated_output_computation method of class MnistModel, a few cells down in the tutorial. The metrics parameter is the return value of tff.learning.Model.report_local_outputs(). Both of these methods are interfaces that need to be implemented for the tff.learning module methods (e.g. tff.learning.build_federated_averaging_process()) to connect up the computation in the correct way.
In TFF, I expect every function has a placement in either the server or clients side, but both of them can be accessible in any function which makes it confusing.
In TFF the data has placement, not the computations. Computations that accept unplaced values can be used on placed values using TFF intrinsics (e.g. tff.federated_mean(), or tff.federated_map()). Also, a TFF computation may accept value placed at CLIENTS or SERVER and also return values placed at either.
If you haven't already, I'd highly recommend looking at the two part tutorial Custom Federated Algorithms. It spends time introducing the programming and execution model of TFF:
Part 1: Introduction to the Federated Core
Part 2: Implementing Federated Averaging
Related
I’m interested in defining a Markov Decision Process as a python function. It would need to interface with PyTorch API for reinforcement learning, however that constraint shapes the function’s form, inputs and outputs.
For context, my problem involves optimally placing items in a warehouse, not knowing the value of future items which might arrive. Anticipating these arrivals would limit greedy behavior of algorithm, effectively reserving some high value locations for high value items which might arrive as learned by the RL model.
How can I best define such a function? (Not asking about business logic but about requirements of its form, inputs outputs etc) What is PyTorch expecting of an MDP?
Use CleanRL
Make custom environment using Gymnasium https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation.html
I'm doing my final degree project. I need to create an extended version of the word2vec algorithm, changing the default objective function of the original paper. This has already been done (check this paper). In that paper, they only say the new objective function, but they do not say how they have run the model.
Now, I need to extend that model too, with another function, but I'm not sure if I have to implement word2vec myself with the new function, or there is a way to replace it in the Gensim word2vec implementation.
I have checked the Word2Vec Gensim documentation but I have not seen any parameter to do this. Do you have any idea how to do it? It is even possible?
I was unsure if this StackExchange site was the correct one, maybe https://ai.stackexchange.com/ is more appropriate.
There's no official support in Gensim for simply dropping in your own objective function.
However, the full source code is available – https://github.com/RaRe-Technologies/gensim – so by editing it, or using it as a model for your own implementation, you could theoretically do anything.
Beware, though:
the code has gone through a lot of optimization & customization for new options that may not be relevant to your needs, so may not be the most clean & simple starting point
for performance, the core routines are written in Cython – see the .pyx files – which can be especially hard to debug, and rely on library bulk array functions that may obscure how to implement your alternate function instead
My desktop application process documents (10 types exist) to provide intelligence using 100s of parameters. Thru supervised training, I came up with a predictive model that uses 100 of the parameters to indicate whether a document is a reliable source of information or not. The training and testing of the Machine Learning model was done in Python. Currently I need to implement the prediction part, which uses the parameters weights from the training part, into my [MFC/VC++] desktop application.
The suggested architecture is to provide a DLL plus header that exposes a function:
bool isDocumentReliable(int docID)
Based on the type of the document, the prediction uses a set of parameters to calculate the probability of the document being reliable. Based on risk assessment (requirements of the business) we translate the probability into a true/false answer.
I am looking for some architectural/implementation information to guide my implementation.
That's my first Machine Learning project and my questions are:
What are the questions I need to be asking?
Should the parameters be hard-coded into my functions? or
Should the parameters be read from text files at runtime?
I strongly suspect that the function should be bool isDocumentReliable(std::wstring pathname), but that's a minor detail. The main question, by far, should be: How are we going to prototype this? Don't expect this to work straight away.
If you've got a boss that thinks Machine Learning is like writing software, and it either works or not, tell him he's flat out wrong.
My programs are growing larger and more sophisticated. As a result I am using more and more functions. My question is, should I "fetch" a value from a function once and then "tote" it around, sending it into other functions as a parameter, or just call, "fetch," the value again from within the other function(s)?
I am sure resources, and speed, are a factor, but what is the general rule, if any?
For example, should I call my sigmoid function, and then use that value as a parameter in a call to the next function that uses it, or just call the sigmoid function again from within that next function?
I know that this question borders on opinion, but I did not attend a CS school, and so find myself wondering what the "norm" for some things are.
Thanks.
You are correct that this question relates more to software engineering theory than just a language (Python). There are programming paradigms which promote one variant over the other but the most general rule of thumb you should aim for is:
High cohesion and low coupling
i.e., within a module of software (which roughly corresponds to a Python module, if you are using them), the functions should have dependence on each other and you should call them to fetch the value. However, across modules, you should not have functional calls and should tie them up at a higher level module (or the main function) by fetching values from one module and passing it to the other.
See also: Memoization.
I am trying to devise an iterative markov decision process (MDP) agent in Python with the following characteristics:
observable state
I handle potential 'unknown' state by reserving some state space
for answering query-type moves made by the DP (the state at t+1 will
identify the previous query [or zero if previous move was not a query]
as well as the embedded result vector) this space is padded with 0s to
a fixed length to keep the state frame aligned regardless of query
answered (whose data lengths may vary)
actions that may not always be available at all states
reward function may change over time
policy convergence should incremental and only computed per move
So the basic idea is the MDP should make its best guess optimized move at T using its current probability model (and since its probabilistic the move it makes is expectedly stochastic implying possible randomness), couple the new input state at T+1 with the reward from previous move at T and reevaluate the model. The convergence must not be permanent since the reward may modulate or the available actions could change.
What I'd like to know is if there are any current python libraries (preferably cross-platform as I necessarily change environments between Windoze and Linux) that can do this sort of thing already (or may support it with suitable customization eg: derived class support that allows redefining say reward method with one's own).
I'm finding information about on-line per-move MDP learning is rather scarce. Most use of MDP that I can find seems to focus on solving the entire policy as a preprocessing step.
Here is a python toolbox for MDPs.
Caveat: It's for vanilla textbook MDPs and not for partially observable MDPs (POMDPs), or any kind of non-stationarity in rewards.
Second Caveat: I found the documentation to be really lacking. You have to look in the python code if you want to know what it implements or you can quickly look at their documentation for a similar toolbox they have for MATLAB.