You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: fern/docs/pages/airdrop/data-extraction.mdx
+54Lines changed: 54 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -279,3 +279,57 @@ await spawn({
279
279
```
280
280
281
281
This lets you simulate a soft timeout and validate that your worker shuts down.
282
+
283
+
## Time-scoped syncs
284
+
285
+
Time-scoped syncs allow using a custom timestamp to control the scope of data extraction. This capability enables more granular control over which data gets synchronized between external systems and DevRev.
286
+
287
+
Enable time-scoped syncs by first adding the capability to your manifest:
288
+
289
+
```yaml
290
+
imports:
291
+
- # slug and other import information ...
292
+
capabilities:
293
+
- TIME_SCOPED_SYNCS
294
+
```
295
+
296
+
Your data extraction implementation must handle two optional parameters from the event's `EventContext`:
297
+
298
+
- **`extract_from`**: Timestamp in RFC3339 format indicating the starting point of extraction. This applies to both initial and incremental syncs.
299
+
- **`reset_extract_from`**: A boolean flag for incremental syncs that determines whether data should be re-extracted.
300
+
301
+
The extraction logic depends on the sync type and parameter combination:
302
+
303
+
**Initial syncs**: Use `extract_from` as the starting timestamp for data extraction.
304
+
305
+
**Incremental syncs**:
306
+
- If `reset_extract_from` is `true`: Start from `extract_from` if it is provided; otherwise, extract all data.
307
+
- If `reset_extract_from` is `false` or not provided: Use the `adapter.state.lastSuccessfulSyncStarted` timestamp.
Copy file name to clipboardExpand all lines: fern/docs/pages/airdrop/getting-started.mdx
+7-1Lines changed: 7 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -100,4 +100,10 @@ The loader receives information about changes in DevRev since the last successfu
100
100
A 2-way sync consists of the following phases:
101
101
102
102
1. Data loading
103
-
2. Attachments loading
103
+
2. Attachments loading
104
+
105
+
### Time-scoped sync
106
+
107
+
Normally, an initial sync extracts all data from an organization, and an incremental sync only extracts changes since the last successful sync. A _time-scoped sync_, on the other hand, allows for the extraction of data beginning from a custom timestamp.
108
+
109
+
This approach is useful for initially testing the integration with a smaller data set—such as data from the last month—before conducting a full import of all available data.
0 commit comments