MultiSegmentArena: preferred segment is not part of the arena

[vigliag]

Hi. We noticed, after updating to the latest go-capnp version (v3.1.0-alpha.1), that we have some test code triggering a "new struct: preferred segment is not part of the arena" error.

Some (very simplified) code that reproduces the issue is:

struct KV {
    key @0 :Int32;
    value @1 :Data;
}

func TestMultiSegment(t *testing.T) {
	_, seg, err := capnp.NewMessage(capnp.MultiSegment(nil))
	if err != nil {
		t.Fatal(err)
	}
	for i := 0; i < 100; i++ {
		kv, err := NewKV(seg)
		if err != nil {
			t.Fatalf("NewKV(%d): %v", i, err)
		}
		kv.SetKey(int32(i))
		err = kv.SetValue(bytes.Repeat([]byte{byte(i)}, 1024)) // large enough to fill the current segment
		if err != nil {
			t.Fatalf("SetValue(%d): %v", i, err)
		}
                // in a real application, KV would be set to some field of a root message
	}
}

which results in

NewKV(5): new struct: preferred segment is not part of the arena

I think the issue might be here:

go-capnp/arena.go

Line 303 in 0b34935

if &msa.segs[i] == seg {

We keep passing in the same segment pointer, however the MultiSegmentArena compares it to the address of the segments in the msa.segs slice, whose underlying array can however be reallocated in the meantime (which is why the error only happens on the 6th iteration).

If my understanding is correct, and the issue is with MultiSegmentArena and not with the way we are using the library, a solution would be turning that slice of segment to a slice of pointers to segments. I'm happy to open a PR in that case.

Thanks in advance!

[hoa-grvt]

We're getting hit the same issue. And I think you're right. msa has []Segment (slice of values). Appending can reallocate and move the backing array; any previously taken *Segment pointers now point to the old backing array. Previous version uses indices/IDs rather than pointers: Pass around SegmentID (index into the slice) and validate bound is safer without hitting this reallocation bug.

[GiedriusS]

Maybe you have some fork/branch with your proposed fix?

[saikocat]

I have created a pull request at #607 to address this.

• Replace []Segment with []*Segment in MultiSegmentArena to avoid address-instability on append.
• Make Segment(id) return the stable pointer from msa.segs (no &msa.segs[id]).
• In Allocate, validate the preferred segment in O(1) by checking seg.id is in-range and msa.segs[seg.id] == seg; remove the linear pointer scan.
• Ensure Release() zeroises and returns buffers, then clears pointers safely.