Web-Based Models

HiveTraceRed supports testing AI chat interfaces that don’t have an official API through browser automation. This is particularly useful for testing web-based AI assistants, custom deployments, or models without public APIs.

Overview

When an AI model doesn’t provide an API, you can use the WebModel framework to interact with it through browser automation using Playwright. This allows you to:

  • Test web-based AI chat interfaces

  • Automate interactions with custom AI deployments

  • Test models that only have web interfaces

  • Record and replay user interactions

The workflow consists of two main components:

  1. Web Action Recorder: Records user interactions to help you identify UI elements

  2. WebModel: Automates browser interactions based on the recorded information

Web Action Recorder

The Web Action Recorder helps you capture user interactions with a web interface, making it easier to create automated test scripts.

What It Records

The recorder captures:

  • Click events: Position, target element, CSS/XPath selectors

  • Input events: Text entered into fields, element identifiers

  • Selection events: Text selected by user, context information

  • Element metadata: IDs, classes, ARIA attributes, data attributes

Command Line Usage

The recorder is available as a CLI command after installing HiveTraceRed:

# Record session starting at a specific URL
hivetracered-recorder --url https://chat.example.com

# Save to specific file
hivetracered-recorder --url https://chat.example.com --output session.json

# Run in headless mode (no visible browser window)
hivetracered-recorder --url https://chat.example.com --headless

# Quiet mode (suppress console output)
hivetracered-recorder --url https://chat.example.com --quiet

Press Ctrl-C to stop recording and save the log file.

Recording Workflow

  1. Start the recorder with your target URL

  2. Interact with the interface:

    • Log in if needed

    • Close cookie banners and popups

    • Type a test message in the input field

    • Press Enter or click the send button

    • Wait for the response

    • Select the response text with your mouse

    • Repeat 2-3 times to capture multiple interactions

  3. Stop the recorder (Ctrl-C)

  4. Review the log file to identify element selectors

Example Log Structure

The recorder saves a JSON file with detailed event information:

[
  {
    "timestamp": "2024-01-15T10:30:45.123456",
    "event_type": "click",
    "x": 850,
    "y": 400,
    "target": {
      "tagName": "BUTTON",
      "id": "send-button",
      "className": "btn-primary",
      "textContent": "Send"
    },
    "selectors": {
      "css": "button#send-button",
      "xpath": "//*[@id='send-button']"
    },
    "url": "https://chat.example.com"
  },
  {
    "timestamp": "2024-01-15T10:30:46.789012",
    "event_type": "input",
    "value": "What is the capital of France?",
    "target": {
      "tagName": "INPUT",
      "id": "message-input",
      "name": "message",
      "type": "text"
    },
    "selectors": {
      "css": "input#message-input",
      "xpath": "//*[@id='message-input']"
    }
  }
]

Creating Web Models

Once you’ve recorded a session and identified the UI elements, you can create a custom WebModel class.

Base WebModel Class

All web models inherit from the WebModel base class, which provides:

  • Browser lifecycle management (Playwright)

  • Concurrent request handling with configurable concurrency

  • Automatic context isolation between requests

  • Stable response detection for streaming UIs

  • Standard Model interface (invoke, batch, stream_abatch)

Required Implementation

You must implement one abstract method:

async def _send_message_and_get_response(self, page: Page, message: str) -> str:
    """Send message and return response text"""

Optional Overrides

You can optionally override:

async def _handle_initial_dialogs(self, page: Page) -> None:
    """Handle consent popups, login, etc."""

async def _create_browser(self) -> Browser:
    """Custom browser launch settings"""

async def _setup_context_and_page(self) -> Tuple[BrowserContext, Page]:
    """Custom context/page initialization"""

Example: Creating a Custom Web Model

Here’s a complete example based on Mistral Le Chat:

from playwright.async_api import Page, TimeoutError as PlaywrightTimeoutError
from hivetracered.models.web_model import WebModel

class MyAIChatModel(WebModel):
    """Web model for your custom AI chat interface."""

    def __init__(
        self,
        model: str = "my-ai-chat",
        max_concurrency: int = 1,
        headless: bool = False,
        **kwargs
    ):
        super().__init__(
            model=model,
            max_concurrency=max_concurrency,
            headless=headless,
            **kwargs
        )

        # Set your chat URL
        self.target_url = "https://your-ai-chat.com/chat"

    async def _handle_initial_dialogs(self, page: Page) -> None:
        """Handle any initial popups or consent dialogs."""
        try:
            # Look for consent button (use selectors from your log file)
            consent_button = await page.wait_for_selector(
                'button:has-text("Accept")',
                timeout=5000
            )
            if consent_button:
                await consent_button.click()
                await page.wait_for_timeout(1000)
        except PlaywrightTimeoutError:
            # No consent dialog, continue
            pass

    async def _send_message_and_get_response(
        self, page: Page, message: str
    ) -> str:
        """Send message and get response."""

        # 1. Find input element (use selector from your log)
        input_element = await page.wait_for_selector(
            'textarea#message-input',  # From your recorded session
            timeout=self.wait_timeout * 1000
        )

        # 2. Type the message
        await input_element.click()
        await input_element.fill('')  # Clear first
        await input_element.type(message, delay=10)

        # 3. Send the message (press Enter or click button)
        await page.keyboard.press('Enter')
        # Or click send button:
        # await page.click('button#send-button')

        # 4. Wait for response to appear (use selector from your log)
        await page.wait_for_selector(
            'div.response-message',
            timeout=self.wait_timeout * 1000
        )

        # 5. Wait for stable response (handles streaming responses)
        response_text = await self._wait_for_stable_response(
            page,
            'div.response-message',  # Selector for response elements
            timeout=self.response_wait_time,
            stable_time=self.stability_check_time,
            fallback_to_last=True
        )

        return response_text.strip()

Using Your Web Model

Once created, use it like any other model:

from my_models import MyAIChatModel

# Initialize
model = MyAIChatModel(
    model="my-ai-chat",
    headless=False,  # Set to True for production
    max_concurrency=2,
    wait_timeout=30,
    response_wait_time=60
)

# Single request
response = model.invoke("What is machine learning?")
print(response["content"])

# Batch requests (runs in parallel up to max_concurrency)
prompts = [
    "What is Python?",
    "Explain quantum computing",
    "What is blockchain?"
]
responses = model.batch(prompts)
for resp in responses:
    print(resp["content"])

# Clean up
model.close()

Helper Methods

The WebModel base class provides several helper methods:

Wait for Stable Response

For streaming UIs where text appears gradually:

response_text = await self._wait_for_stable_response(
    page,
    selector='div.message-content',
    timeout=60,  # Max wait time in seconds
    stable_time=2.0,  # Must be stable for 2 seconds
    fallback_to_last=True  # Return last element on timeout
)

Find Element with Fallbacks

Try multiple selectors until one matches:

element = await self._find_element_with_fallbacks(
    page,
    selectors=[
        'textarea#chat-input',
        'div[contenteditable="true"]',
        'input[type="text"][name="message"]'
    ],
    timeout=3000  # Timeout per selector in milliseconds
)

Best Practices

  1. Record First: Always use the Web Action Recorder to identify elements before coding

  2. Use Multiple Selectors: Provide fallback selectors in case the UI changes

  3. Handle Timeouts: Set appropriate timeouts for your target interface

  4. Start Headless=False: Debug with visible browser first, then use headless mode

  5. Respect Concurrency: Start with max_concurrency=1, increase carefully

  6. Clean Up: Always call model.close() when done

  7. Error Handling: Web UIs can change; implement robust error handling

  8. Stability Checks: Use _wait_for_stable_response for streaming responses

Configuration Parameters

All web models support these parameters:

  • model (str): Model identifier for logging

  • max_concurrency (int): Max parallel browser contexts (default: 1)

  • headless (bool): Run browser without GUI (default: False)

  • wait_timeout (int): Element wait timeout in seconds (default: 30)

  • response_wait_time (int): Max time to wait for response (default: 60)

  • stability_check_time (float): Time content must be stable (default: 2.0)

Advanced Features

Custom Browser Launch

Override _create_browser for custom settings:

async def _create_browser(self) -> Browser:
    return await self.playwright.chromium.launch(
        headless=self.headless,
        args=['--disable-blink-features=AutomationControlled'],
        proxy={'server': 'http://proxy.example.com:8080'}
    )

Session Persistence

Override _setup_context_and_page to reuse sessions:

async def _setup_context_and_page(self):
    # Load saved state (cookies, localStorage)
    context = await self.browser.new_context(
        storage_state='auth_state.json'
    )
    page = await context.new_page()
    await page.goto(self.target_url)
    return context, page

Troubleshooting

Browser Not Closing

Always call model.close() or use context managers:

# This pattern ensures cleanup
model = MyAIChatModel()
try:
    response = model.invoke("test")
finally:
    model.close()

Selector Not Found

  • Check if the page fully loaded (wait_until='networkidle')

  • Verify selector with the Web Action Recorder

  • Try alternative selectors with _find_element_with_fallbacks

Timeout Errors

  • Increase wait_timeout or response_wait_time

  • Check if elements are hidden or in iframes

  • Verify the page URL is correct

Response Cut Off

  • Increase stability_check_time for slower streaming

  • Check if response_wait_time is too short

  • Verify the response selector captures the full message

See Also