Exploring Signet Cloud for background inference

[NicholaiVogel]

Signet works best when it can think in the background.

A longstanding pain-point in Signet is that the memory pipeline requires continuous inference for surfacing insights through background distillation, drawing meaningful connections in the knowledge graph, and continuously updating it. That means users are either forced to:

A. use their existing subscription-based AI plans, by letting Signet call their preferred CLI headlessly. (we just added ACPX as an official inference backend, still in beta, but this will be getting cleaner and more reliable for users that prefer to operate this way)
B. run inference locally using llama.cpp, vllm or ollama
C. turn off background processing/inference entirely.

So while users get to own their memory, most are not financially able to support the level of inference or the compute necessary to make the most of it. And that sucks so much.

So what I'm thinking is that we could build a simple cloud/API service that helps take care of the baseline inference needs for Signet users. We are exploring whether we can provide a small free monthly allowance for background inference, so new users can experience Signet properly without configuring a model or API key on day one. That could include multi-modal embeddings and a set number of tokens per month for the background pipeline, managed by Signet Cloud.

A level up from this could be a paid subscription service, ideally for around $4-8/month, that would offer more tokens, hosted sync, and best-in-class embedding models.

Nothing about Signet itself would change. It stays open source, your data stays yours, and you would continue to own your memory. This service would be opt-in only.

[NicholaiVogel]

@aaf2tbz It's clear that you pointed your llm at this discussion instead of actually reading it or thinking about what a good response might be. Which is frustrating given that I am trying to actually have a meaningful discussion here.

• Concerning ACPX, I really don't care about your llms review of it, but given the concern, we've version locked the dependency to 0.7.0, and will build around that for the foreseeable future. However, ACPX/ACP is completely irrelevant in a discussion about Signet's proposed cloud infrastructure.
• Concerning Encrypted storage, fucking duh.
• Signet Cloud is not for storing Signet secrets. That is what Onepassword and Bitwarden are for.