Home

A lightweight, no-strings-attached framework for your LLM that allows applying Chain-of-Thought-alike prompt schema (See related section) towards a massive textual collections.

Contents

• API
• Batching

API

Download this provider:

wget https://raw.githubusercontent.com/nicolay-r/nlp-thirdgate/refs/heads/master/llm/replicate_104.py

This is a simple example of API usage (version 1.0.1):

from bulk_chain.core.utils import dynamic_init
from bulk_chain.api import iter_content

llm = dynamic_init(class_filepath="replicate_104.py",
                   class_name="Replicate")(api_token="<API-KEY>")

content_it = iter_content(
    # 1. Your iterator of dictionaries
    data_it,
    # 2. Your third-party model implementation.
    llm=llm,
    # 3. Your schema.              
    schema="ext/schema/default.json")
    
for content in content_it:
    # Handle your LLM responses here ...

Batching

You can enable batching support via the support_batching parameter.

class YourLLM(BaseLM):

    def __init__(self, **kwargs):
        super(YourLLM, self).__init__(support_batching=True, **kwargs)
        # Any further initialization.