Basic e-commerce scraper using scrapingcourse.com.
| API | Description |
|---|---|
list |
Scrapes products from the e-commerce store with pagination support. Automatically triggers details API for each product using extendPayload |
details |
Extracts detailed information for a specific product including price, SKU, category, descriptions, images (uploaded to S3), sizes, colors, and variants |
npm install
# or
yarnIf the intuned CLI is not installed, install it globally:
npm install -g @intuned/cliAfter installing dependencies, intuned command should be available in your environment.
Before running any API, provision and deploy the project first.
intuned dev provision
intuned dev deployintuned dev run api list .parameters/api/list/default.json
intuned dev run api details .parameters/api/details/default.json/
├── api/
│ ├── list.ts # Scrape product list with pagination
│ └── details.ts # Extract detailed product information
├── utils/
│ └── typesAndSchemas.ts # Type definitions and Zod schemas
├── intuned-resources/
│ └── jobs/
│ ├── list.job.jsonc # Job for product list
│ └── details.job.jsonc # Job for product details
├── .parameters/api/ # Test parameters
├── Intuned.jsonc # Project config
├── package.json # Node.js dependencies
└── README.md
- Automatic pagination: The
listAPI automatically handles pagination to scrape multiple pages - Dynamic API chaining: Uses
extendPayloadto automatically trigger thedetailsAPI for each product found - S3 file upload: Product images are automatically uploaded to S3 using
saveFileToS3 - Job configuration: Configured as a job template with retry logic and concurrent request handling