[Bug] Web Scraper Hangs on Large Documents - Memory Leak in HTML Parser

[anshul23102]

Description

Scraper hangs indefinitely when parsing large HTML documents (>50MB). Memory usage grows unbounded, eventually causing out-of-memory crash. No timeout or resource limits implemented.

Steps to Reproduce

1. Scrape website with large HTML (50MB+)
2. Parser processes entire document into memory
3. Memory grows to 2-4 GB
4. Process crashes

Environment Information

• Framework: Node.js
• Parser: Cheerio/jsdom
• Memory: Unbounded
• Application version: Current main branch

Expected Behavior

Parser uses streaming for large documents. Max file size enforced (10MB). Request timeout (30s). Memory-efficient chunked parsing.

Actual Behavior

File: src/scrapers/parser.js
Loads entire HTML into memory: fs.readFileSync(large_file)

Code Reference

File: src/scrapers/parser.js
Missing: Stream processing, memory limits, timeout

Additional Context

Use streams:

fs.createReadStream('large.html')
  .pipe(parseStream)
  .on('error', () => timeout());

GSSoC Points Estimate: Level 2 (Performance/Memory)

Suggested Labels

• gssoc:approved
• type:bug
• severity:high
• area:performance

[Xenon010101]

Hi, I am a GSSoC 2026 contributor and would like to work on this issue. This project matches my expertise. Please assign it to me.

/assign

[garima-agarwall]

Hi @sharmavaibhav31
I would like to work on this issue. Please assign this to me if possible.

[sharmavaibhav31]

Thanks for reporting this. Before assigning, could you provide repository-specific reproduction details? The referenced file (src/scrapers/parser.js) and Node.js/Cheerio-based parser do not appear to match the current scraper architecture. Please share the exact file path, logs, reproduction steps, and any memory profiling evidence so we can validate the issue.