feat: Add agentic AI for automatic annotations

[amirrezaalasti]

Problem Statement

Currently, adding Web Context Interface (WCI) annotations (data-wci-* attributes) to existing codebases requires manual placement. This is a tedious, time-consuming, and error-prone process, especially for large DOM structures or dynamically generated content. We need a way to automatically deduce the semantic role of elements to scale the adoption of WCI.

Proposed Solution / API Design

We should introduce an agentic AI pipeline capable of parsing raw or unannotated HTML, analyzing the visual and structural context, and automatically applying the correct semantic WCI attributes (such as data-wci-tag and data-wci-action).

Proposed Checklist

• Implement a pipeline to process unannotated HTML.
• Introduce an agent model/prompt to deduce the role of DOM nodes.
• Automatically apply data-wci-tag, data-wci-action, and other related attributes.
• Create evaluations to measure the accuracy of the automated annotations.

Example Usage

<div class="checkout-btn" onclick="process()">Complete Order</div>
<span class="nav-item">Settings</span>

<div class="checkout-btn" onclick="process()" data-wci-tag="button" data-wci-action="checkout">Complete Order</div>
<span class="nav-item" data-wci-tag="link" data-wci-action="navigate-settings">Settings</span>

// Example TypeScript API usage for the automated pipeline
import { WCIAutoAnnotator } from '@wci/agent';

const rawHtml = `
  <main>
    <div class="user-card">Profile</div>
  </main>
`;

// Initialize the annotator with a chosen LLM provider/model
const annotator = new WCIAutoAnnotator({
  model: 'gpt-4o', // or gemini-1.5-pro, claude-3.5-sonnet
  temperature: 0.1,
});

// Process the HTML
const annotatedHtml = await annotator.process(rawHtml);
console.log(annotatedHtml);

Alternatives Considered

• Rule-Based Heuristics (AST Parsing): We considered writing static regex or AST parsers to look for specific tags (e.g., automatically tagging <a> as links). However, modern web apps frequently use generic tags (<div>, <span>) with CSS classes for interactivity, which rule-based systems fail to interpret semantically without agentic reasoning.
• Browser Extension for Manual Tagging: A UI tool to let developers click and tag elements. While useful, it doesn't solve the core problem of manual labor required for large repositories.

Additional Context

• Context Window Limits: We will need to figure out chunking strategies for extremely large HTML documents so we don't blow past the LLM's context window.
• Evals are Critical: The fourth checklist item (evaluations) will be crucial. We need a golden dataset of pre-annotated HTML to measure the agent's precision and recall to ensure it doesn't hallucinate non-existent actions or miss critical interactive elements.