Skip to content

Home

Jump to bottom
Nicolas Raymond edited this page Oct 29, 2020 · 5 revisions

Welcome to the FLSimulator wiki!

Basics of federated learning

Context

In many real world applications, it may be difficult (if not impossible) to gather all data needed to train a supervised learning algorithm within a single dataset. Considering that we have many decentralized datasets sharing the same set of features but different observations, federated learning proposes to learn a global model that is trained and shared among them.

Concepts

Here we define few concepts in order to clarify FLSimulator modules' names.

  • Node
    The object containing one decentralized datasets needed to train a global model is referred as a node.

  • Central Server
    The object managing training of a global model across different nodes is referred as a central server.

  • Network
    The entity regrouping a set of nodes and a single central server is named a network.

Training process

In order to train a global model across a network, the federated learning approach leans on an iterative algorithm where each iteration is known as a training round. In the figure below, we present the main procedure executed throughout the t-th training round.

  1. The central server select a subset of k nodes among the n nodes available. When k=n, we will refer to this situation as full node participation in opposition to a partial participation situation when k<n, which is more common in a high scale network where we want to reduce communication cost between central server and nodes.

  2. The central server broadcasts its current model to each node of the subset.

  3. Each node of the subset locally updates a copy of the model, executing a predetermined number of optimization iterations.

  4. The central server aggregates the last updated parameters sent back by the selected nodes in order to get the new parameters and hence, the new global model.

What is the purpose FLSimulator?

FLSimulator provides tools in order to easily simulate federated supervised learning.

Modules

Here we summarize the user friendly modules provided by FLSimulator.

  • BasisFunctions
    Module that provides functions to transform original input (vector) observations.

  • LearningModels
    Module that gives access to 2 linear supervised learning models:

    • Linear Regression (trained via SGD)
    • Logistic Regression

  • DataManagement
    Module that gives access to objects that can generate datasets for our nodes.

  • Server
    Module that contains the CentralServer class, the core of a federated learning network.

  • Network
    Module that hosts the Node and Network classes.

! Please have a look at the rest of the Wiki for more information on modules and classes.

References

  • Xu, Jie, and Fei Wang. “Federated Learning for Healthcare Informatics.” ArXiv:1911.06270 [Cs], Nov. 2019, http://arxiv.org/abs/1911.06270.
  • Li, Xiang, et al. “On the Convergence of FedAvg on Non-IID Data.” ArXiv:1907.02189 [Cs, Math, Stat], Feb. 2020, http://arxiv.org/abs/1907.02189.
  • Yang, Qiang, et al. “Federated Machine Learning: Concept and Applications.” ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, Jan. 2019, p. 12:1–12:19, doi:10.1145/3298981.

Clone this wiki locally