-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
update of recent task force work and plans
- Loading branch information
Showing
1 changed file
with
22 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
## Privacy Metrics PATCG Work Summary Report | ||
|
||
Feb 28, 2024 | ||
|
||
The Privacy Metrics subgroup/taskforce has been meeting weekly to investigate and develop metrics to measure and compare the privacy-loss of solutions in Ads Measurement. | ||
|
||
The core goal is to develop a metric (or small set of metrics) to enable us to: | ||
1. Provide a clear and intuitive explanation of the privacy systems have created, making it more comprehensible for all stakeholders. | ||
2. Offer a standardized metric or framework for evaluating the efficacy of different privacy models in real-world scenarios and under specific threat models | ||
3. Facilitate easier comparisons between various protection mechanisms, assisting in choosing the most appropriate model and parameters for datasets. | ||
4. Promote transparency in the application of privacy measures, ensuring that all users can understand and trust the privacy protections in place. | ||
|
||
The products will be a White Paper with our comparison findings reviewing the performance of various metrics, as well as a reference synthetic dataset for future work to continue to build on, and for outside researchers to continue benchmarking new approaches. We have a version of this dataset that is now available. We aim to distribute the White Paper to PATCG members by the April meetings. In setting the foundation for this White Paper we have constructed a number of documents to help align on: the goals and desired properties of any metric, the threat model under consideration, a literature review of metrics used in research, and a comparison of the front-running metrics against our desiderata. | ||
|
||
For interested parties, these can be found below. | ||
* Proposed Metric [Goals and Desiderata](https://docs.google.com/document/d/1eCVc1t-geMDgc1OoPNdHiYq5A2i7wCe1lTV5RguNJg4/edit?usp=sharing) | ||
* Attribution [Synthetic Data Proposal](https://docs.google.com/document/d/1Vxq4LrMe3A2WIlu-7IYP1Hycr_nz3_qTpPAICX9fLcw/edit?usp=sharing), files and [generation code](https://colab.research.google.com/drive/1u8HmA36RLP2gZD41lMyRJM5w7q1xkCQY?usp=sharing), and [comparison to Criteo dataset](https://docs.google.com/presentation/d/1CA3WZbMp_h6dXmNqD4MS0Pr0z1ldHxVq/edit?usp=sharing&ouid=117973408570053353528&rtpof=true&sd=true) | ||
* Literature review of [Privacy Loss Metrics](https://docs.google.com/document/d/1IFI6m4FU0ex-X7s_qdPZBnc9QSnat1F5_zlSZ-L7slw/edit?usp=sharing) | ||
* Table of [Threat Model Comparison](https://docs.google.com/document/d/1C0zziceHv60DYBhVpCWnHGAfJJwwNh9K3aJLs_VBoe0/edit?usp=sharing) | ||
* Table of [Privacy Loss Metric Comparison](https://docs.google.com/document/d/12yrwwqd-Su-SJEK86IauOQZ65MlzV1u-PRa-8Mb6U60/edit#heading=h.o4mwmd9po6fl) | ||
* Proposed [White Paper Outline](https://docs.google.com/document/d/1rF8zfnnFSZQRh-4WjC83UgZdvYtU3sl_xT2r3UKgmpE/edit#heading=h.sfozyjtf6om) | ||
|