@@ -4,11 +4,11 @@ to retrieve all the items that should be synced with DevRev.
4
4
If the current run is an initial sync, this means all the items should be extracted.
5
5
Otherwise the extractor should retrieve all the items that were changed since the start of the last extraction.
6
6
7
- Each snap-in invocation runs in a separate runtime instance with a maximum execution time of 13 minutes.
7
+ Each snap-in invocation runs in a separate runtime instance with a maximum execution time of 13 minutes.
8
8
After 10 minutes, the Airdrop platform sends a message to the snap-in to gracefully exit.
9
9
10
- If a large amount of data needs to be extracted, it might not all be extracted within this time frame.
11
- To handle such situations, the snap-in uses a state object.
10
+ If a large amount of data needs to be extracted, it might not all be extracted within this time frame.
11
+ To handle such situations, the snap-in uses a state object.
12
12
This state object is shared across all invocations and keeps track of where the previous snap-in invocations ended in the extraction process.
13
13
14
14
## Triggering event
@@ -68,20 +68,20 @@ await adapter.emit(ExtractorEventType.ExtractionDataError, {
68
68
### Extracting and storing the data
69
69
70
70
The SDK library includes a repository system for handling extracted items.
71
- Each item type, such as users, tasks, or issues, has its own repository.
72
- These are defined in the ` repos ` array as ` itemType ` .
71
+ Each item type, such as users, tasks, or issues, has its own repository.
72
+ These are defined in the ` repos ` array as ` itemType ` .
73
73
The ` itemType ` name should match the ` record_type ` specified in the provided metadata.
74
74
75
75
``` typescript
76
76
const repos = [
77
77
{
78
- itemType: ' todos' ,
78
+ itemType: " todos" ,
79
79
},
80
80
{
81
- itemType: ' users' ,
81
+ itemType: " users" ,
82
82
},
83
83
{
84
- itemType: ' attachments' ,
84
+ itemType: " attachments" ,
85
85
},
86
86
];
87
87
```
@@ -104,14 +104,14 @@ After initialization of repositories using `initializeRepos`,
104
104
items should be then retrieved from the external system and stored in the correct repository by calling the ` push ` function.
105
105
106
106
``` typescript
107
- await adapter .getRepo (' users' )?.push (items );
107
+ await adapter .getRepo (" users" )?.push (items );
108
108
```
109
109
110
110
Behind the scenes, the SDK library stores items pushed to the repository and uploads them in batches to the Airdrop platform.
111
111
112
112
### Data normalization
113
113
114
- Extracted data must be normalized to fit the domain metadata defined in the ` external-domain-metadata.json ` file.
114
+ Extracted data must be normalized to fit the domain metadata defined in the ` external-domain-metadata.json ` file.
115
115
More details on this process are provided in the [ Metadata extraction] ( /public/snapin-development/adaas/metadata-extraction ) section.
116
116
117
117
Normalization rules:
@@ -130,15 +130,15 @@ Extracted items are automatically normalized when pushed to the `repo` if a norm
130
130
``` typescript
131
131
const repos = [
132
132
{
133
- itemType: ' todos' ,
133
+ itemType: " todos" ,
134
134
normalize: normalizeTodo ,
135
135
},
136
136
{
137
- itemType: ' users' ,
137
+ itemType: " users" ,
138
138
normalize: normalizeUser ,
139
139
},
140
140
{
141
- itemType: ' attachments' ,
141
+ itemType: " attachments" ,
142
142
normalize: normalizeAttachment ,
143
143
},
144
144
];
@@ -161,15 +161,15 @@ All other fields are contained within the `data` attribute.
161
161
"owner" : " A3A" ,
162
162
"rca" : null ,
163
163
"severity" : " fatal" ,
164
- "summary" : " Lorem ipsum" ,
164
+ "summary" : " Lorem ipsum"
165
165
}
166
166
}
167
167
```
168
168
169
169
If the item you are normalizing is a work item (a ticket, task, issue, or similar),
170
- it should also contain the ` item_url_field ` within the ` data ` attribute.
170
+ it should also contain the ` item_url_field ` within the ` data ` attribute.
171
171
This field should be assigned a URL that points to the item in the external system.
172
- This link is visible in the airdropped item in the DevRev app,
172
+ This link is visible in the airdropped item in the DevRev app,
173
173
helping users to easily locate the item in the external system.
174
174
175
175
``` json {12}
@@ -205,13 +205,13 @@ echo '{}' | chef-cli fuzz-extracted -r issue -m external_domain_metadata.json >
205
205
206
206
## State handling
207
207
208
- To enable information passing between invocations and runs, a limited amount of data can be saved as the snap-in ` state ` .
208
+ To enable information passing between invocations and runs, a limited amount of data can be saved as the snap-in ` state ` .
209
209
Snap-in ` state ` persists between phases in one sync run as well as between multiple sync runs.
210
210
211
211
You can access the ` state ` through SDK's ` adapter ` object.
212
212
213
213
``` typescript
214
- adapter .state [' users' ].completed = true ;
214
+ adapter .state [" users" ].completed = true ;
215
215
```
216
216
217
217
A snap-in must consult its state to obtain information on when the last successful forward sync started.
@@ -244,12 +244,12 @@ await spawn<DummyExtractorState>({
244
244
});
245
245
```
246
246
247
- ## Timeout handling
247
+ ## Handling lambda timeout
248
248
249
- To prevent long-running or blocked worker threads from stalling the sync process, the Airdrop SDK implements a two-tier timeout mechanism.
249
+ When a worker thread is busy with other work and doesn't respond to exit messages from the main thread, it becomes blocked and can stall the sync process. To prevent this , the Airdrop SDK implements a two-tier timeout mechanism.
250
250
251
- - The ** soft timeout** ( default: 10 minutes, configurable) sends an exit message to the worker thread, allowing it to gracefully shut down via the ` onTimeout ` function.
252
- - If the worker does not respond within the ** hard timeout** ( default: 13 minutes) , it is forcefully terminated.
251
+ - The ** soft timeout** , default of 10 minutes and configurable, sends an exit message to the worker thread, allowing it to gracefully shut down via the ` onTimeout ` function. How to configure this is shown in the example below .
252
+ - If the worker does not respond within the ** hard timeout** , default of 13 minutes, it is forcefully terminated.
253
253
254
254
This mechanism ensures that the snap-in does not hang indefinitely, and the system can recover cleanly in case of stuck or slow code execution.
255
255
0 commit comments