Skip to content

Commit e063011

Browse files
authored
repo sync (#236)
* repo-sync-2025-01-14T10:53:24+0800 * repo-sync-2025-01-14T14:10:30+0800 * repo-sync-2025-01-14T14:46:56+0800 * repo-sync-2025-01-14T15:08:00+0800 * repo-sync-2025-01-14T15:12:10+0800 * repo-sync-2025-01-14T16:30:46+0800 * repo-sync-2025-01-14T19:22:17+0800
1 parent 8ead92f commit e063011

File tree

125 files changed

+14320
-1267
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

125 files changed

+14320
-1267
lines changed

.circleci/release.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ jobs:
6161
name: "Copy binary"
6262
command: |
6363
mkdir -p /tmp/binary/x86
64-
cp bazel-bin/psi/main /tmp/binary/x86
64+
cp bazel-bin/psi/apps/psi_launcher/main /tmp/binary/x86
6565
- persist_to_workspace:
6666
root: "/tmp/binary"
6767
paths:
@@ -95,7 +95,7 @@ jobs:
9595
name: "Copy binary"
9696
command: |
9797
mkdir -p /tmp/binary/arm
98-
cp bazel-bin/psi/main /tmp/binary/arm
98+
cp bazel-bin/psi/apps/psi_launcher/main /tmp/binary/arm
9999
- persist_to_workspace:
100100
root: "/tmp/binary"
101101
paths:

LEGAL.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
1-
Legal Disclaimer
1+
# Legal Disclaimer
22

3-
Within this source code, the comments in Chinese shall be the original, governing version. Any comment in other languages are for reference only. In the event of any conflict between the Chinese language version comments and other language version comments, the Chinese language version shall prevail.
3+
Within this source code, the comments in Chinese shall be the original, governing version.
4+
Any comment in other languages are for reference only.
5+
In the event of any conflict between the Chinese language version comments and other language version comments, the Chinese language version shall prevail.
46

57
法律免责声明
68

7-
关于代码注释部分,中文注释为官方版本,其它语言注释仅做参考。中文注释可能与其它语言注释存在不一致,当中文注释与其它语言注释存在不一致时,请以中文注释为准。
9+
关于代码注释部分,中文注释为官方版本,其它语言注释仅做参考。中文注释可能与其它语言注释存在不一致,当中文注释与其它语言注释存在不一致时,请以中文注释为准。

MODULE.bazel

-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,6 @@ bazel_dep(name = "abseil-cpp", version = "20240722.0")
3535
bazel_dep(name = "gflags", version = "2.2.2")
3636
bazel_dep(name = "rapidjson", version = "1.1.0.bcr.20241007")
3737
bazel_dep(name = "boost.math", version = "1.83.0")
38-
bazel_dep(name = "boost.uuid", version = "1.83.0.bcr.1")
3938
bazel_dep(name = "boost.algorithm", version = "1.83.0.bcr.1")
4039
bazel_dep(name = "boost.multiprecision", version = "1.83.0")
4140
bazel_dep(name = "zlib", version = "1.3.1.bcr.3")

README.md

+96-92
Original file line numberDiff line numberDiff line change
@@ -22,97 +22,96 @@ In the following example, we are going to run PSI at a single host.
2222

2323
2. Prepare data and config.
2424

25-
receiver.config:
26-
27-
```json
28-
{
29-
"psi_config": {
30-
"protocol_config": {
31-
"protocol": "PROTOCOL_RR22",
32-
"role": "ROLE_RECEIVER",
33-
"broadcast_result": true
34-
},
35-
"input_config": {
36-
"type": "IO_TYPE_FILE_CSV",
37-
"path": "/root/receiver/receiver_input.csv"
38-
},
39-
"output_config": {
40-
"type": "IO_TYPE_FILE_CSV",
41-
"path": "/root/receiver/receiver_output.csv"
42-
},
43-
"keys": [
44-
"id0",
45-
"id1"
46-
],
47-
"debug_options": {
48-
"trace_path": "/root/receiver/receiver.trace"
49-
}
50-
},
51-
"self_link_party": "receiver",
52-
"link_config": {
53-
"parties": [
54-
{
55-
"id": "receiver",
56-
"host": "127.0.0.1:5300"
25+
receiver.config:
26+
27+
```json
28+
{
29+
"psi_config": {
30+
"protocol_config": {
31+
"protocol": "PROTOCOL_RR22",
32+
"role": "ROLE_RECEIVER",
33+
"broadcast_result": true
5734
},
58-
{
59-
"id": "sender",
60-
"host": "127.0.0.1:5400"
35+
"input_config": {
36+
"type": "IO_TYPE_FILE_CSV",
37+
"path": "/root/receiver/receiver_input.csv"
38+
},
39+
"output_config": {
40+
"type": "IO_TYPE_FILE_CSV",
41+
"path": "/root/receiver/receiver_output.csv"
42+
},
43+
"keys": [
44+
"id0",
45+
"id1"
46+
],
47+
"debug_options": {
48+
"trace_path": "/root/receiver/receiver.trace"
6149
}
62-
]
50+
},
51+
"self_link_party": "receiver",
52+
"link_config": {
53+
"parties": [
54+
{
55+
"id": "receiver",
56+
"host": "127.0.0.1:5300"
57+
},
58+
{
59+
"id": "sender",
60+
"host": "127.0.0.1:5400"
61+
}
62+
]
63+
}
6364
}
64-
}
65-
```
65+
```
6666

67-
sender.config:
67+
sender.config:
6868

69-
```json
70-
{
71-
"psi_config": {
72-
"protocol_config": {
73-
"protocol": "PROTOCOL_RR22",
74-
"role": "ROLE_SENDER",
75-
"broadcast_result": true
76-
},
77-
"input_config": {
78-
"type": "IO_TYPE_FILE_CSV",
79-
"path": "/root/sender/sender_input.csv"
80-
},
81-
"output_config": {
82-
"type": "IO_TYPE_FILE_CSV",
83-
"path": "/root/sender/sender_output.csv"
84-
},
85-
"keys": [
86-
"id0",
87-
"id1"
88-
],
89-
"debug_options": {
90-
"trace_path": "/root/sender/sender.trace"
91-
}
92-
},
93-
"self_link_party": "sender",
94-
"link_config": {
95-
"parties": [
96-
{
97-
"id": "receiver",
98-
"host": "127.0.0.1:5300"
69+
```json
70+
{
71+
"psi_config": {
72+
"protocol_config": {
73+
"protocol": "PROTOCOL_RR22",
74+
"role": "ROLE_SENDER",
75+
"broadcast_result": true
76+
},
77+
"input_config": {
78+
"type": "IO_TYPE_FILE_CSV",
79+
"path": "/root/sender/sender_input.csv"
9980
},
100-
{
101-
"id": "sender",
102-
"host": "127.0.0.1:5400"
81+
"output_config": {
82+
"type": "IO_TYPE_FILE_CSV",
83+
"path": "/root/sender/sender_output.csv"
84+
},
85+
"keys": [
86+
"id0",
87+
"id1"
88+
],
89+
"debug_options": {
90+
"trace_path": "/root/sender/sender.trace"
10391
}
104-
]
92+
},
93+
"self_link_party": "sender",
94+
"link_config": {
95+
"parties": [
96+
{
97+
"id": "receiver",
98+
"host": "127.0.0.1:5300"
99+
},
100+
{
101+
"id": "sender",
102+
"host": "127.0.0.1:5400"
103+
}
104+
]
105+
}
105106
}
106-
}
107-
```
108-
109-
| File Name | Location | Description |
110-
| :---------------- | :---------------------------------- | :------------------------------------------------------------------------- |
111-
| receiver.config | /tmp/receiver/receiver.config | Config for receiver. |
112-
| sender.config | /tmp/sender/sender.config | Config for sender. |
113-
| receiver_input.csv | /tmp/receiver/receiver_input.csv | Input for receiver. Make sure the file contains two id keys - id0 and id1. |
114-
| sender_input.csv | /tmp/sender/sender_input.csv | Input for sender. Make sure the file contains two id keys - id0 and id1. |
107+
```
115108

109+
| File Name | Location | Description |
110+
| :---------------- | :---------------------------------- | :------------------------------------------------------------------------- |
111+
| receiver.config | /tmp/receiver/receiver.config | Config for receiver. |
112+
| sender.config | /tmp/sender/sender.config | Config for sender. |
113+
| receiver_input.csv | /tmp/receiver/receiver_input.csv | Input for receiver. Make sure the file contains two id keys - id0 and id1. |
114+
| sender_input.csv | /tmp/sender/sender_input.csv | Input for sender. Make sure the file contains two id keys - id0 and id1. |
116115

117116
3. Run PSI
118117

@@ -122,7 +121,7 @@ In the first terminal, run the following command
122121
docker run -it --rm --network host --mount type=bind,source=/tmp/receiver,target=/root/receiver --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:latest --config receiver/receiver.config
123122
```
124123

125-
In the other terminal, run the following command simultaneously.
124+
In the other terminal, run the following command simultaneously.
126125

127126
```bash
128127
docker run -it --rm --network host --mount type=bind,source=/tmp/sender,target=/root/sender --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:latest --config sender/sender.config
@@ -131,7 +130,8 @@ docker run -it --rm --network host --mount type=bind,source=/tmp/sender,target
131130
You could also pass a minified JSON config directly. A minified JSON is a compact one without white space and line breaks.
132131

133132
e.g.
134-
```
133+
134+
```bash
135135
docker run -it --rm --network host --mount type=bind,source=/tmp/sender,target=/root/sender --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:latest --json '{"psi_config":{"protocol_config":{"protocol":"PROTOCOL_RR22","role":"ROLE_RECEIVER","broadcast_result":true},"input_config":{"type":"IO_TYPE_FILE_CSV","path":"/root/receiver/receiver_input.csv"},"output_config":{"type":"IO_TYPE_FILE_CSV","path":"/root/receiver/receiver_output.csv"},"keys":["id0","id1"],"debug_options":{"trace_path":"/root/receiver/receiver.trace"}},"self_link_party":"receiver","link_config":{"parties":[{"id":"receiver","host":"127.0.0.1:5300"},{"id":"sender","host":"127.0.0.1:5400"}]}}'
136136
```
137137

@@ -184,33 +184,37 @@ bazel build //... -c opt
184184
bazel test //...
185185
```
186186

187-
188187
### Trace
189188

190189
We use [Perfetto](https://perfetto.dev/) from Google for tracing.
191190

192191
Please use debug_options.trace_path field in PsiConfig to modify trace file path. The default path is /tmp/psi.trace.
193192

194-
After running psi binaries, please check trace by using [Trace Viewer](https://ui.perfetto.dev/). If this is not applicable, please check [this link](https://github.com/google/perfetto/issues/170) to deploy your own website.
193+
After running psi binaries, please check trace by using [Trace Viewer](https://ui.perfetto.dev/). If this is not applicable,
194+
please check [this link](https://github.com/google/perfetto/issues/170) to deploy your own website.
195195

196196
The alternate way to visualize trace is to use **chrome://tracing**:
197-
1. Download perfetto assets from https://github.com/google/perfetto/releases/tag/v37.0
197+
198+
1. Download perfetto assets from <https://github.com/google/perfetto/releases/tag/v37.0>
198199
2. You should find traceconv binary in assets folder.
199200
3. Transfer trace file to JSON format:
200201

201-
```bash
202-
chmod +x traceconv
202+
```bash
203+
chmod +x traceconv
204+
205+
./traceconv json [trace file path] [json file path]
206+
```
203207

204-
./traceconv json [trace file path] [json file path]
205-
```
206208
4. Open chrome://tracing in your chrome and load JSON file.
207209

208210

211+
212+
213+
209214
## PSI V2 Benchamrk
210215

211216
Please refer to [PSI V2 Benchmark](docs/user_guide/psi_v2_benchmark.md)
212217

213218
## APSI Benchmark
214219

215220
Please refer to [APSI Benchmark](docs/user_guide/apsi_benchmark.md)
216-

RELEASE.md

+14-4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
# RELEASE
2+
13
> NOTE:
24
>
35
> - `[Feature]` prefix for new features.
@@ -6,46 +8,57 @@
68
> - `[Improvement]` prefix for implementation improvement.
79
810
## v0.5.0.dev241107
11+
912
- `[API]` delete legacy ub psi function.
1013
- `[API]` output csv null_rep can be specified.
1114
- `[Feature]` join can be done with one receiver.
1215
- `[Feature]` join can be done with ub psi protocol.
1316

1417
## v0.4.0.dev240801
18+
1519
- [Bugfix] Fix MacOS and arm build.
1620

1721
## v0.4.0.dev240731
22+
1823
- [Improvement] Port APSI again.
1924
- [Feature] Add Bucketized APSI.
2025
- [API] Remove SealPIR.
2126

2227
## v0.4.0.dev240521
28+
2329
- [API] remove BC22 protocol
2430

2531
## v0.4.0.dev240517
32+
2633
- [Improvement] upgrade yacl to 0.4.5b0.
2734

2835
## v0.4.0.dev240514
36+
2937
- [API] add entrypoint for docker file.
3038
- [API] allow passing config JSON directly to main.
3139
- [Bugfix] fix ic mode.
3240
- [Bugfix] fix RR22, SealPIR and APSI.
3341

3442
## v0.4.0.dev240401
43+
3544
- [Improvement] upgrade download uri of xz.
3645

3746
## v0.4.0.dev240329
47+
3848
- [Improvement] upgrade yacl to 0.4.4b3.
3949

4050
## v0.3.0beta
51+
4152
- [Improvement] add uuid in system temp folder.
4253
- [Improvement] use arrow csv reader in pir.
43-
- [Bugfix] fix typo in psi config check.
54+
- [Bugfix] fix typo in psi config check.
4455

4556
## v0.3.0.dev240304
57+
4658
- [API] expose ic_mode in RunLegacyPsi api
4759

4860
## v0.3.0.dev240222
61+
4962
- [API] expose PIR API.
5063

5164
## v0.3.0.dev240219
@@ -63,13 +76,11 @@
6376
- [Bugfix] Fix flakiness in psi_test.
6477
- [Bugfix] Fix race condition in rr22.
6578

66-
6779
## v0.2.0.dev231228
6880

6981
- [Bugfix] Fix RR22 race condition.
7082
- [Improvement] modify sort buffer size.
7183

72-
7384
## v0.2.0.dev231221
7485

7586
- [API] Rename check_duplicates to skip_duplicates_check.
@@ -79,7 +90,6 @@
7990
- [Bugfix] Fix duplicate key check.
8091
- [Bugfix] Fix SyncWait.
8192

82-
8393
## v0.1.0beta
8494

8595
- [API] Add PSI v2 API.

0 commit comments

Comments
 (0)