Skip to content

Commit 971ad91

Browse files
authored
Merge pull request #54 from ScrapeGraphAI/generate-schema
Generate schema service added
2 parents 43b8250 + c95e0c6 commit 971ad91

File tree

16 files changed

+2768
-1
lines changed

16 files changed

+2768
-1
lines changed

scrapegraph-js/README.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Official JavaScript/TypeScript SDK for the ScrapeGraph AI API - Smart web scrapi
1515
- 🔍 Detailed error handling
1616
- ⚡ Automatic retries and logging
1717
- 🔐 Secure API authentication
18+
- 🔧 AI-powered schema generation
1819

1920
## 📦 Installation
2021

@@ -487,6 +488,108 @@ const feedbackText = 'This is a test feedback message.';
487488
})();
488489
```
489490

491+
### AI-Powered Schema Generation
492+
493+
Generate JSON schemas from natural language prompts using AI. This feature helps you create structured data schemas for web scraping and data extraction.
494+
495+
#### Basic Schema Generation
496+
497+
```javascript
498+
import { generateSchema } from 'scrapegraph-js';
499+
500+
const apiKey = 'your-api-key';
501+
const prompt = 'Find laptops with specifications like brand, processor, RAM, storage, and price';
502+
503+
(async () => {
504+
try {
505+
const response = await generateSchema(prompt, null, { apiKey });
506+
console.log('Generated schema:', response.generated_schema);
507+
console.log('Request ID:', response.request_id);
508+
} catch (error) {
509+
console.error('Error generating schema:', error);
510+
}
511+
})();
512+
```
513+
514+
#### Modifying Existing Schemas
515+
516+
```javascript
517+
import { generateSchema } from 'scrapegraph-js';
518+
519+
const apiKey = 'your-api-key';
520+
const existingSchema = {
521+
type: 'object',
522+
properties: {
523+
name: { type: 'string' },
524+
price: { type: 'number' }
525+
},
526+
required: ['name', 'price']
527+
};
528+
529+
const modificationPrompt = 'Add brand and rating fields to the existing schema';
530+
531+
(async () => {
532+
try {
533+
const response = await generateSchema(modificationPrompt, existingSchema, { apiKey });
534+
console.log('Modified schema:', response.generated_schema);
535+
} catch (error) {
536+
console.error('Error modifying schema:', error);
537+
}
538+
})();
539+
```
540+
541+
#### Checking Schema Generation Status
542+
543+
```javascript
544+
import { getSchemaStatus } from 'scrapegraph-js';
545+
546+
const apiKey = 'your-api-key';
547+
const requestId = '123e4567-e89b-12d3-a456-426614174000';
548+
549+
(async () => {
550+
try {
551+
const response = await getSchemaStatus(requestId, { apiKey });
552+
console.log('Status:', response.status);
553+
if (response.status === 'completed') {
554+
console.log('Generated schema:', response.generated_schema);
555+
}
556+
} catch (error) {
557+
console.error('Error checking status:', error);
558+
}
559+
})();
560+
```
561+
562+
#### Polling for Completion with Progress Tracking
563+
564+
```javascript
565+
import { pollSchemaGeneration } from 'scrapegraph-js';
566+
567+
const apiKey = 'your-api-key';
568+
const requestId = '123e4567-e89b-12d3-a456-426614174000';
569+
570+
(async () => {
571+
try {
572+
const finalResult = await pollSchemaGeneration(requestId, {
573+
apiKey,
574+
maxAttempts: 15,
575+
delay: 3000,
576+
onProgress: ({ attempt, maxAttempts, status, response }) => {
577+
if (status === 'checking') {
578+
console.log(`Checking status... (${attempt}/${maxAttempts})`);
579+
} else {
580+
console.log(`Status: ${status} (${attempt}/${maxAttempts})`);
581+
}
582+
}
583+
});
584+
585+
console.log('Schema generation completed!');
586+
console.log('Final schema:', finalResult.generated_schema);
587+
} catch (error) {
588+
console.error('Error during polling:', error);
589+
}
590+
})();
591+
```
592+
490593
## 🔧 Available Functions
491594

492595
### Scrape

0 commit comments

Comments
 (0)