Skip to content

Removed manual worker path mapping from the template #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gasperzgonec merged 4 commits into from
Dec 17, 2025
Merged

Removed manual worker path mapping from the template #60

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0fe6edf
Added changes for the getWorkerPath changes to the SDK.
gasperzgonec Dec 16, 2025
e513b91
Updated SDK and removed unused import.
gasperzgonec Dec 16, 2025
fd08d64
Removed manual mapping from the loading part.
gasperzgonec Dec 16, 2025
64c3d24
Updated sdk to 1.13.1.
gasperzgonec Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions code/package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion code/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"yargs": "^17.6.2"
},
"dependencies": {
"@devrev/ts-adaas": "1.12.2",
"@devrev/ts-adaas": "1.13.0",
Comment thread
gasperzgonec marked this conversation as resolved.
Outdated
"@devrev/typescript-sdk": "1.1.63",
"axios": "^1.9.0",
"dotenv": "^16.0.3",
Expand Down
26 changes: 2 additions & 24 deletions code/src/functions/extraction/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { AirdropEvent, EventType, spawn } from '@devrev/ts-adaas';
import { AirdropEvent, spawn } from '@devrev/ts-adaas';
Comment thread
gasperzgonec marked this conversation as resolved.

import initialDomainMapping from '../external-system/initial_domain_mapping.json';

Expand All @@ -19,35 +19,13 @@ export const initialExtractorState: ExtractorState = {
attachments: { completed: false },
};

function getWorkerPerExtractionPhase(event: AirdropEvent) {
let path;
switch (event.payload.event_type) {
case EventType.ExtractionExternalSyncUnitsStart:
path = __dirname + '/workers/external-sync-units-extraction';
break;
case EventType.ExtractionMetadataStart:
path = __dirname + '/workers/metadata-extraction';
break;
case EventType.ExtractionDataStart:
case EventType.ExtractionDataContinue:
path = __dirname + '/workers/data-extraction';
break;
case EventType.ExtractionAttachmentsStart:
case EventType.ExtractionAttachmentsContinue:
path = __dirname + '/workers/attachments-extraction';
break;
}
return path;
}

const run = async (events: AirdropEvent[]) => {
for (const event of events) {
const file = getWorkerPerExtractionPhase(event);
await spawn<ExtractorState>({
event,
initialState: initialExtractorState,
workerPath: file,
initialDomainMapping,
baseWorkerPath: __dirname,

// TODO: If needed you can pass additional options to the spawn function.
// For example timeout of the lambda, batch size, etc.
Expand Down
20 changes: 2 additions & 18 deletions code/src/functions/loading/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { AirdropEvent, EventType, spawn } from '@devrev/ts-adaas';
import { AirdropEvent, spawn } from '@devrev/ts-adaas';

import initialDomainMapping from '../external-system/initial_domain_mapping.json';

Expand All @@ -11,29 +11,13 @@ export interface LoaderState {}
// This state will be used as a starting point for the loading process.
export const initialLoaderState: LoaderState = {};

function getWorkerPerLoadingPhase(event: AirdropEvent) {
let path;
switch (event.payload.event_type) {
case EventType.StartLoadingData:
case EventType.ContinueLoadingData:
path = __dirname + '/workers/load-data';
break;
case EventType.StartLoadingAttachments:
case EventType.ContinueLoadingAttachments:
path = __dirname + '/workers/load-attachments';
break;
}
return path;
}

const run = async (events: AirdropEvent[]) => {
for (const event of events) {
const file = getWorkerPerLoadingPhase(event);
await spawn<LoaderState>({
event,
initialState: initialLoaderState,
workerPath: file,
initialDomainMapping,
baseWorkerPath: __dirname,

// TODO: If needed you can pass additional options to the spawn function.
// For example timeout of the lambda, batch size, etc.
Expand Down