Proposal: Add Partial Update Feature to Bleve Index (Go)

[yakomisar]

Context

Currently Bleve supports the following operations:

• Index(id, document)
• Delete(id)

However, there is no convenient way to update only a subset of fields in an indexed document without re-building the full document externally.

In high-throughput systems where documents are partially updated frequently (for example updating price or stock), the caller is forced to:

1. Fetch or reconstruct the full document
2. Merge the changes
3. Reindex the entire document

This adds unnecessary complexity to the indexing pipeline and increases serialization overhead.

Problem

Bleve requires the full document for every update via Index(id, doc).
There is no helper API for partial document updates (patch updates).

This is problematic in systems where:

• Documents are large
• Only a few fields change
• Partial indexing occurs frequently

The service processes high RPS (e.g. ~1000 RPS)

Proposal

Introduce a Partial Update (Patch) API at the repository/service level that allows updating only the modified fields.

Example API:

type Patch map[string]any

type IndexRepository interface {
    IndexFull(ctx context.Context, id string, doc any) error
    Patch(ctx context.Context, id string, patch Patch) error
    Delete(ctx context.Context, id string) error
}

Patch() would accept only the fields that need to be updated.

Example usage:

repo.Patch(ctx, "sku123", Patch{
    "price": 129.99,
    "in_stock": true,
})

I understand that Bleve segments are immutable, true in-place updates are not possible.

type Patch map[string]any

func applyPatch(doc map[string]any, patch Patch) {
    for k, v := range patch {
        if v == nil {
            delete(doc, k)
            continue
        }

        if subPatch, ok := v.(map[string]any); ok {
            if subDoc, ok2 := doc[k].(map[string]any); ok2 {
                applyPatch(subDoc, subPatch)
                doc[k] = subDoc
                continue
            }
        }

        doc[k] = v
    }
}

Additionally, it would be useful to support a timestamp or version parameter in the patch operation in order to validate update ordering and prevent stale updates.

Expected patch workflow

1. Retrieve the existing document (from a document store or stored fields)
2. Merge the patch fields into the document
3. Reindex the merged document

This provides a simple and safe abstraction while keeping Bleve’s internal behavior unchanged.

Use Cases

Partial updates are particularly useful in systems where documents are large but only a small subset of fields changes frequently.
Typical examples include:

E-commerce catalogs

Fields such as:

• price
• discount
• stock
• availability
• rating

may change frequently, while the rest of the document remains unchanged.

Requiring a full document rebuild for every update introduces unnecessary overhead.

High-frequency updates

In systems with frequent updates (e.g. inventory updates or price changes), partial updates allow services to submit only the modified fields rather than reconstructing the entire document.

This simplifies the indexing pipeline and reduces serialization and network overhead.

Microservice architectures

In distributed systems, different services may own different parts of a document.

Example:

• pricing service updates price
• inventory service updates stock
• catalog service updates title or description

A patch-based API allows each service to update only the fields it owns without reconstructing the entire document.