federated | A framework | Machine Learning library

 by   tensorflow Python Version: v0.59.0 License: Apache-2.0

kandi X-RAY | federated Summary

kandi X-RAY | federated Summary

federated is a Python library typically used in Institutions, Learning, Education, Artificial Intelligence, Machine Learning, Deep Learning, Tensorflow applications. federated has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has high support. You can install using 'pip install federated' or download it from GitHub, PyPI.

TensorFlow Federated (TFF) is an open-source framework for machine learning and other computations on decentralized data. TFF has been developed to facilitate open research and experimentation with Federated Learning (FL), an approach to machine learning where a shared global model is trained across many participating clients that keep their training data locally. For example, FL has been used to train prediction models for mobile keyboards without uploading sensitive typing data to servers. TFF enables developers to use the included federated learning algorithms with their models and data, as well as to experiment with novel algorithms. The building blocks provided by TFF can also be used to implement non-learning computations, such as aggregated analytics over decentralized data.
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              federated has a highly active ecosystem.
              It has 2119 star(s) with 557 fork(s). There are 71 watchers for this library.
              There were 6 major release(s) in the last 12 months.
              There are 74 open issues and 306 have been closed. On average issues are closed in 39 days. There are 182 open pull requests and 0 closed requests.
              OutlinedDot
              It has a negative sentiment in the developer community.
              The latest version of federated is v0.59.0

            kandi-Quality Quality

              federated has 0 bugs and 0 code smells.

            kandi-Security Security

              federated has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              federated code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              federated is licensed under the Apache-2.0 License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              federated releases are available to install and integrate.
              Deployable package is available in PyPI.
              Build file is available. You can build the component from source.
              Installation instructions are available. Examples and code snippets are not available.

            Top functions reviewed by kandi - BETA

            kandi has reviewed federated and discovered the below as its top functions. This is intended to give you an instant insight into federated implemented functionality, and help decide if they suit your requirements.
            • Return theural representation of a composition .
            • Create a convolutional network .
            • Creates a security sum that can be used by the metric_finalizer .
            • Create a functional model from a keras model .
            • Create a histogram aggregation factory .
            • Given a Lambda function computes the corresponding Lambda function .
            • Compose a computation using a computation .
            • Build a client update function .
            • Builds a Computation .
            • Construct a single round - trip computation .
            Get all kandi verified functions for this library.

            federated Key Features

            No Key Features are available at this moment for federated.

            federated Examples and Code Snippets

            copy iconCopy
            sh run_fedavg_distributed_reg.sh 6 1 1 1 graphsage homo 150 1 1 0.0015 256 256 0.3 256 256 freesolv "./../../../data/freesolv/" 0
            
            ##run on background
            nohup sh run_fedavg_distributed_reg.sh 6 1 1 1 graphsage homo 150 1 1 0.0015 256 256 0.3 256 256 fr  
            Federated Learning Simulator (FLSim),Installation
            Pythondot img2Lines of Code : 4dot img2License : Permissive (BSD-3-Clause)
            copy iconCopy
            pip install flsim
            
            git clone https://github.com/facebookresearch/FLSim.git
            cd FLSim
            pip install -e .
              
            Terraform Provider Artifactory,Testing,Testing Federated repos
            Godot img3Lines of Code : 3dot img3License : Permissive (Apache-2.0)
            copy iconCopy
            $ export ARTIFACTORY_URL_2=http://localhost:9081
            
            $ make acceptance
            
            $ make acceptance_federated
              

            Community Discussions

            QUESTION

            Local Model performance in Tensorflow Federated
            Asked 2022-Mar-24 at 03:16

            I am implementing federated learning through tensorflow-federated. The tutorial and all other material available compared the accuracy of the federated (global) model after each communication round. Is there a way I can compute the accuracy of each local model to compare against federated (global) model.

            Summary: Total number of clients: 15 For each communication round: Local vs Federated Model performance

            References:

            1. (https://colab.research.google.com/github/tensorflow/federated/blob/main/docs/tutorials/federated_learning_for_image_classification.ipynb#scrollTo=blQGiTQFS9_r)
            ...

            ANSWER

            Answered 2021-Nov-09 at 01:29

            I dont know how you can achieve this with tff.learning.build_federated_averaging_process but I recommend you to take a look at this simple fedavg implementation. Here you can use test_data -the same evaluation dataset you use in server model- for each client. I would suggest you to do client_test_datasets = [test_data for x in sampled_train_ids]. Then pass this as iterative_process.next(server_state, sampled_train_data, client_test_datasets ). Here you need to change the signatures for run_one_round and client_update_fn in the simple_fedavg_tff.py. In each case the signatures from test datasets shall be same as the ones for training dataset. Dont forget actually passing appropriate test datasets as input to each. Now move on to simple_fedavg_tf.py and change your client_update. Here you basically need to write evaluation very similar to one done for server model. Thereafter print the evaluation results if you wish or change the outputs for each level (tf.function, tff.tf_computation, and tff.federated_computation) and pass the eval results as output. If you go this way dont forget to update the output of iterative_process.next

            edit: I assumed you wanted the accuracy of clients when the test dataset is the same as server test dataset.

            Source https://stackoverflow.com/questions/69891610

            QUESTION

            ValueError: Missing data for input "input_2". You passed a data dictionary with keys ['y', 'x']. Expected the following keys: ['input_2']
            Asked 2022-Mar-17 at 18:08

            Following the previous code here I am in process to evaluate the federated learning model and I got couple of issues. This is the code for evaluation

            ...

            ANSWER

            Answered 2022-Mar-17 at 07:35

            The test dataset has a different format during evaluation. Try:

            Source https://stackoverflow.com/questions/71506975

            QUESTION

            ValueError: Layer "sequential" expects 1 input(s), but it received 10 input tensors
            Asked 2022-Mar-15 at 15:48

            I am following TFF tutorials to build my FL model My data is contained in different CSV files which are considered as different clients. Following this tutorial, and build the Keras model function as following

            ...

            ANSWER

            Answered 2022-Mar-15 at 15:48

            A couple problems: Your data has ten separate features, which means you actually need 10 separate inputs for your model. However, you can also stack the features into a tensor and then use a single input with the shape (10,). Here is a working example, but please note that it uses dummy data and therefore may not make much sense in reality.

            Create dummy data:

            Source https://stackoverflow.com/questions/71428904

            QUESTION

            Loading multiple CSV files (silos) to compose Tensorflow Federated dataset
            Asked 2022-Mar-10 at 17:04

            I am working on pre-processed data that were already siloed into separated csv files to represent separated local data for federated learning.

            To correct implement the federated learning with these multiple CSVs on TensorFlow Federated, I am just trying to reproduce the same approach with a toy example in the iris dataset. However, when trying to use the method tff.simulation.datasets.TestClientData, I am getting the error:

            ...

            ANSWER

            Answered 2022-Mar-10 at 17:04

            I did some search of public code in github using class tff.simulation.datasets.TestClientData, then I found the following implementation (source here):

            Source https://stackoverflow.com/questions/71396780

            QUESTION

            splitting the data into training and testing in federated learning
            Asked 2022-Mar-10 at 13:35

            I am new in federated learning I am currently experimenting with a model by following the official TFF documentation. But I am stuck with an issue and hope I find some explanation here.

            I am using my own dataset, the data are distributed in multiple files, each file is a single client (as I am planning to structure the model). and the dependant and independent variables have been defined.

            Now, my question is how can I split the data into training and testing sets in each client(file) in federated learning? like what we -normally- do in the centralized ML models The following code is what I have implemented so far: note my code is inspired by the official documentation and this post which is almost similar to my application, but it aims to split the clients as training and testing clients itself while my aim is to split the data inside these clients.

            ...

            ANSWER

            Answered 2022-Mar-10 at 13:35

            See this tutorial. You should be able to create two datasets (train and test) based on the clients and their data:

            Source https://stackoverflow.com/questions/71330639

            QUESTION

            Tff: define the usage of Tensorflow.take() function
            Asked 2022-Mar-02 at 16:12

            I am trying to mimic the federated learning implementation provided here: Working with tff's clientData in order to understand the code clearly. I reached to this point where I need clarification in.

            ...

            ANSWER

            Answered 2022-Mar-02 at 12:57

            QUESTION

            MNIST Shard Descriptor: IndexError: list index out of range
            Asked 2022-Mar-01 at 08:01

            I am working on Federated Learning experiments using Intel OpenFL. I want to distribute my dataset (MNIST) using different non-iidness scenarios. I am following their official documentation: https://openfl.readthedocs.io/en/latest/source/utilities/splitters_data.html

            This is my original working code:

            ...

            ANSWER

            Answered 2022-Mar-01 at 08:01

            The only solution I found until now is to reduce the rank of each envoy: train_idx = train_splitter.split(self.y, self.worldsize)[self.rank-1]

            Source https://stackoverflow.com/questions/71209010

            QUESTION

            TFF: train_test_client_split to partition each client data
            Asked 2022-Feb-28 at 13:57

            I am building a federated learning model. I have written the code below, but I keep getting the error, which is also not true please let me know how to use the function train_test_client_split properly?

            ...

            ANSWER

            Answered 2022-Feb-28 at 13:57

            You just need the correct data structure. Try something like the following.

            Create dummy data

            Source https://stackoverflow.com/questions/71289273

            QUESTION

            tff.simulation.datasets.ClientData to build federated learning model from CSV files
            Asked 2022-Feb-26 at 15:23

            I am building a federated learning model using my own dataset. I aim to build a multi classification model. The data are presented in separate 8 CSV files.

            I followed the instructions in this post As shown in the code below.

            ...

            ANSWER

            Answered 2022-Feb-26 at 15:23

            In this setup it maybe useful to consider tff.simulation.datasets.FilePerUserClientData and tf.data.experimental.CsvDataset.

            This might look like (this makes some test CSV data for the sake of the example, the dataset your working with likely has other shapes):

            Source https://stackoverflow.com/questions/71273332

            QUESTION

            How to build federated learning model of unbalanced and small dataset
            Asked 2022-Feb-19 at 00:49

            I am working to build a federated learning model using TFF and I have some questions:

            1. I am preparing the dataset, I have separate files of data, with same features and different samples. I would consider each of these files as a single client. How can I maintain this in TFF?

            2. The data is not balanced, meaning, the size of data varies in each file. Is this affecting the modeling process?

            3. The size of the data is a bit small, one file (client) is having 300 records and another is 1500 records, is it suitable to build a federated learning model?

            Thanks in advance

            ...

            ANSWER

            Answered 2022-Feb-19 at 00:49
            1. You can create a ClientData for your dataset, see Working with tff's ClientData.
            2. The dataset doesn't have to balanced to build a federated learning model. In https://arxiv.org/abs/1602.05629, the server takes weighted federated averaging of client's model updates, where the weights are the number of samples each client has.
            3. A few hundred records per client is no less than the EMNIST dataset, so that would be fine. About the total number of clients: this tutorial shows FL with 10 clients, you can run the colab with smaller NUM_CLIENTS to see how it works on the example dataset.

            Source https://stackoverflow.com/questions/71167600

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install federated

            See the install documentation for instructions on how to install TensorFlow Federated as a package or build TensorFlow Federated from source.
            See the get started documentation for instructions on how to use TensorFlow Federated.

            Support

            There are a number of ways to contribute depending on what you're interested in:. Please be sure to review the contribution guidelines for guidelines on how to contribute.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries

            Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link