federated | A framework | Machine Learning library
kandi X-RAY | federated Summary
kandi X-RAY | federated Summary
TensorFlow Federated (TFF) is an open-source framework for machine learning and other computations on decentralized data. TFF has been developed to facilitate open research and experimentation with Federated Learning (FL), an approach to machine learning where a shared global model is trained across many participating clients that keep their training data locally. For example, FL has been used to train prediction models for mobile keyboards without uploading sensitive typing data to servers. TFF enables developers to use the included federated learning algorithms with their models and data, as well as to experiment with novel algorithms. The building blocks provided by TFF can also be used to implement non-learning computations, such as aggregated analytics over decentralized data.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Return theural representation of a composition .
- Create a convolutional network .
- Creates a security sum that can be used by the metric_finalizer .
- Create a functional model from a keras model .
- Create a histogram aggregation factory .
- Given a Lambda function computes the corresponding Lambda function .
- Compose a computation using a computation .
- Build a client update function .
- Builds a Computation .
- Construct a single round - trip computation .
federated Key Features
federated Examples and Code Snippets
Lines of Code : 22
License : No License
Copysh run_fedavg_distributed_reg.sh 6 1 1 1 graphsage homo 150 1 1 0.0015 256 256 0.3 256 256 freesolv "./../../../data/freesolv/" 0
##run on background
nohup sh run_fedavg_distributed_reg.sh 6 1 1 1 graphsage homo 150 1 1 0.0015 256 256 0.3 256 256 fr pip install flsim
git clone https://github.com/facebookresearch/FLSim.git
cd FLSim
pip install -e .
$ export ARTIFACTORY_URL_2=http://localhost:9081
$ make acceptance
$ make acceptance_federated
Community Discussions
Trending Discussions on federated
QUESTION
I am implementing federated learning through tensorflow-federated. The tutorial and all other material available compared the accuracy of the federated (global) model after each communication round. Is there a way I can compute the accuracy of each local model to compare against federated (global) model.
Summary: Total number of clients: 15 For each communication round: Local vs Federated Model performance
References:
...ANSWER
Answered 2021-Nov-09 at 01:29I dont know how you can achieve this with tff.learning.build_federated_averaging_process but I recommend you to take a look at this simple fedavg implementation. Here you can use test_data -the same evaluation dataset you use in server model- for each client. I would suggest you to do client_test_datasets = [test_data for x in sampled_train_ids]. Then pass this as iterative_process.next(server_state, sampled_train_data, client_test_datasets ). Here you need to change the signatures for run_one_round and client_update_fn in the simple_fedavg_tff.py. In each case the signatures from test datasets shall be same as the ones for training dataset. Dont forget actually passing appropriate test datasets as input to each. Now move on to simple_fedavg_tf.py and change your client_update. Here you basically need to write evaluation very similar to one done for server model. Thereafter print the evaluation results if you wish or change the outputs for each level (tf.function, tff.tf_computation, and tff.federated_computation) and pass the eval results as output. If you go this way dont forget to update the output of iterative_process.next
edit: I assumed you wanted the accuracy of clients when the test dataset is the same as server test dataset.
QUESTION
Following the previous code here I am in process to evaluate the federated learning model and I got couple of issues. This is the code for evaluation
...ANSWER
Answered 2022-Mar-17 at 07:35The test dataset has a different format during evaluation. Try:
QUESTION
I am following TFF tutorials to build my FL model My data is contained in different CSV files which are considered as different clients. Following this tutorial, and build the Keras model function as following
...ANSWER
Answered 2022-Mar-15 at 15:48A couple problems: Your data has ten separate features, which means you actually need 10 separate inputs for your model. However, you can also stack the features into a tensor and then use a single input with the shape (10,). Here is a working example, but please note that it uses dummy data and therefore may not make much sense in reality.
Create dummy data:
QUESTION
I am working on pre-processed data that were already siloed into separated csv files to represent separated local data for federated learning.
To correct implement the federated learning with these multiple CSVs on TensorFlow Federated, I am just trying to reproduce the same approach with a toy example in the iris dataset. However, when trying to use the method tff.simulation.datasets.TestClientData, I am getting the error:
ANSWER
Answered 2022-Mar-10 at 17:04I did some search of public code in github using class tff.simulation.datasets.TestClientData, then I found the following implementation (source here):
QUESTION
I am new in federated learning I am currently experimenting with a model by following the official TFF documentation. But I am stuck with an issue and hope I find some explanation here.
I am using my own dataset, the data are distributed in multiple files, each file is a single client (as I am planning to structure the model). and the dependant and independent variables have been defined.
Now, my question is how can I split the data into training and testing sets in each client(file) in federated learning? like what we -normally- do in the centralized ML models The following code is what I have implemented so far: note my code is inspired by the official documentation and this post which is almost similar to my application, but it aims to split the clients as training and testing clients itself while my aim is to split the data inside these clients.
...ANSWER
Answered 2022-Mar-10 at 13:35See this tutorial. You should be able to create two datasets (train and test) based on the clients and their data:
QUESTION
I am trying to mimic the federated learning implementation provided here: Working with tff's clientData in order to understand the code clearly. I reached to this point where I need clarification in.
...ANSWER
Answered 2022-Mar-02 at 12:57In this line:
QUESTION
I am working on Federated Learning experiments using Intel OpenFL. I want to distribute my dataset (MNIST) using different non-iidness scenarios. I am following their official documentation: https://openfl.readthedocs.io/en/latest/source/utilities/splitters_data.html
This is my original working code:
...ANSWER
Answered 2022-Mar-01 at 08:01The only solution I found until now is to reduce the rank of each envoy:
train_idx = train_splitter.split(self.y, self.worldsize)[self.rank-1]
QUESTION
I am building a federated learning model.
I have written the code below, but I keep getting the error, which is also not true
please let me know how to use the function train_test_client_split properly?
ANSWER
Answered 2022-Feb-28 at 13:57You just need the correct data structure. Try something like the following.
Create dummy dataQUESTION
I am building a federated learning model using my own dataset. I aim to build a multi classification model. The data are presented in separate 8 CSV files.
I followed the instructions in this post As shown in the code below.
...ANSWER
Answered 2022-Feb-26 at 15:23In this setup it maybe useful to consider tff.simulation.datasets.FilePerUserClientData and tf.data.experimental.CsvDataset.
This might look like (this makes some test CSV data for the sake of the example, the dataset your working with likely has other shapes):
QUESTION
I am working to build a federated learning model using TFF and I have some questions:
I am preparing the dataset, I have separate files of data, with same features and different samples. I would consider each of these files as a single client. How can I maintain this in TFF?
The data is not balanced, meaning, the size of data varies in each file. Is this affecting the modeling process?
The size of the data is a bit small, one file (client) is having 300 records and another is 1500 records, is it suitable to build a federated learning model?
Thanks in advance
...ANSWER
Answered 2022-Feb-19 at 00:49- You can create a ClientData for your dataset, see Working with tff's ClientData.
- The dataset doesn't have to balanced to build a federated learning model. In https://arxiv.org/abs/1602.05629, the server takes weighted federated averaging of client's model updates, where the weights are the number of samples each client has.
- A few hundred records per client is no less than the EMNIST dataset, so that would be fine. About the total number of clients: this tutorial shows FL with 10 clients, you can run the colab with smaller
NUM_CLIENTSto see how it works on the example dataset.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install federated
See the get started documentation for instructions on how to use TensorFlow Federated.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
