|
6 | 6 | | **Author(s) ** | Jason Mancuso ( [email protected]) |
|
7 | 7 | | **Sponsor ** | Michael Reneer ( [email protected]) |
|
8 | 8 | | **Updated** | YYYY-MM-DD (TODO) |
|
9 |
| -| **Obsoletes** | TF-RFC it replaces, else remove this header | |
10 | 9 |
|
11 | 10 | ## Objective
|
12 | 11 |
|
13 |
| -What are we doing and why? What problem will this solve? What are the goals and |
14 |
| -non-goals? This is your executive summary; keep it short, elaborate below. |
15 |
| -Add an explicit placement for Aggregators to the TensorFlow Federated Core. |
| 12 | +This document proposes adding an explicit `tff.AGGREGATORS` placement to the |
| 13 | +Federated Core (FC) in TensorFlow Federated (TFF). This would allow users to express |
| 14 | +custom aggregation protocols directly in TFF without adding unnecessary |
| 15 | +assumptions about where aggregation functions must be placed and computed. |
16 | 16 |
|
17 | 17 | ## Motivation
|
18 | 18 |
|
19 |
| -Why this is a valuable problem to solve? What background information is needed |
20 |
| -to show how this design addresses the problem? |
21 |
| - |
22 |
| -Which users are affected by the problem? Why is it a problem? What data supports |
23 |
| -this? What related work exists? |
| 19 | +*Why this is a valuable problem to solve? What background information is needed |
| 20 | +to show how this design addresses the problem?* |
| 21 | + |
| 22 | +When approaching federated learning with an eye for security or privacy, it is |
| 23 | +useful to divide federated computation into two categories: computations performing |
| 24 | +aggregations, and computations performing on-device computation. Security and |
| 25 | +privacy issues tend to show up during the aggregation phase. This is particularly |
| 26 | +clear when looking at common methods of adding security guarantees to traditional, |
| 27 | +parameter-server style federated learning, for example with secure aggregation or |
| 28 | +differentially private federated averaging (DP-FedAvg). |
| 29 | + |
| 30 | +In security-heightened settings, it is often worthwhile to separate computation |
| 31 | +done in this aggregation phase from computation performed by the server in the |
| 32 | +traditional parameter server setup. This amounts to delegating aggregations to a |
| 33 | +third-party service. For example, when the clients are mistrustful of the server, |
| 34 | +aggregations might be delegated to a trusted execution environment or to a cluster |
| 35 | +of machines engaging in a secure multi-party computation protocol. Another example |
| 36 | +is secure aggregation in the |
| 37 | +[Encode-Shuffle-Analyze (ESA)](https://arxiv.org/abs/1710.00901) |
| 38 | +model, which in a federated context generally assumes an additional |
| 39 | +party to perform the secure shuffling needed to realize a differential privacy |
| 40 | +guarantee. Since this is an established area of the literature with strong |
| 41 | +motivations and results, we see this as an important line of work for TFF to |
| 42 | +support in order to keep with its |
| 43 | +[project goals](https://github.com/tensorflow/federated#tensorflow-federated). |
| 44 | + |
| 45 | +In general, any secure aggregation protocol can be represented as a coordinated |
| 46 | +computation between three groups of parties: a server, a (potentially singleton) |
| 47 | +set of aggregators, and a set of clients. Note that these need not be mutually |
| 48 | +exclusive, so for example the traditional parameter server setting can be recovered |
| 49 | +as a special case by treating the server as a singleton aggregators set. |
| 50 | + |
| 51 | +The TFF Federated Core (FC) language currently realizes logically-distinct parties |
| 52 | +as "placements". While there exist `tff.SERVER` and `tff.CLIENTS` placements in FC, |
| 53 | +there is no `tff.AGGREGATORS` placement. Without such a placement, implementing new |
| 54 | +aggregation protocols in TFF can require low-level programming of the TFF executor |
| 55 | +stacks, as evidenced by |
| 56 | +[this community attempt to integrate secure aggregation](https://github.com/tf-encrypted/rfcs/blob/master/20190924-tensorflow-federated/integration-strategies.md). |
| 57 | +By adding a `tff.AGGREGATORS` placement, users can more easily implement new |
| 58 | +aggregation protocols by expressing them as federated computations in FC. |
24 | 59 |
|
25 | 60 | ## User Benefit
|
26 | 61 |
|
27 |
| -How will users (or other contributors) benefit from this work? What would be the |
28 |
| -headline in the release notes or blog post? |
| 62 | +*How will users (or other contributors) benefit from this work? What would be the |
| 63 | +headline in the release notes or blog post?* |
| 64 | + |
| 65 | +Users can now express custom aggregation protocols in the Federated Core by working |
| 66 | +with federated data placed on `tff.AGGREGATORS`. Users will be unencumbered by the |
| 67 | +constraints of the current federated types in FC. |
29 | 68 |
|
30 | 69 | ## Design Proposal
|
31 | 70 |
|
32 |
| -This is the meat of the document, where you explain your proposal. If you have |
| 71 | +*This is the meat of the document, where you explain your proposal. If you have |
33 | 72 | multiple alternatives, be sure to use sub-sections for better separation of the
|
34 | 73 | idea, and list pros/cons to each approach. If there are alternatives that you
|
35 | 74 | have eliminated, you should also list those here, and explain why you believe
|
36 | 75 | your chosen approach is superior.
|
37 | 76 |
|
38 |
| -Make sure you’ve thought through and addressed the following sections. If a section is not relevant to your specific proposal, please explain why, e.g. your RFC addresses a convention or process, not an API. |
| 77 | +Make sure you’ve thought through and addressed the following sections. If a section is not relevant to your specific proposal, please explain why, e.g. your RFC addresses a convention or process, not an API.* |
39 | 78 |
|
40 | 79 |
|
41 | 80 | ### Alternatives Considered
|
42 |
| -* Make sure to discuss the relative merits of alternatives to your proposal. |
| 81 | +A lower effort alternative might be to expect users to write custom executors, or custom executor stacks, to include additional "aggregator" parties when executing intrinisics. AGGREGATORS would stay outside of the FC type system, but could be sill be included in federated computations. This allows library designers to extend TFF for their own use cases. This is a major disadvantage, since users are only expected to be familiar with TFF Federated Learning (FL) or FC APIs, and this is a feature that would be useful to the majority of TFF users. |
43 | 82 |
|
44 | 83 | ### Performance Implications
|
45 | 84 | * Do you expect any (speed / memory)? How will you confirm?
|
|
0 commit comments