-
Notifications
You must be signed in to change notification settings - Fork 649
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve memory alignment #673
base: main
Are you sure you want to change the base?
Conversation
fef18ee
to
f9ce17b
Compare
db.go
Outdated
//nolint | ||
dataref []byte // mmap'ed readonly, write throws SEGV | ||
txs []*Tx | ||
|
||
// Put `stats` at the first field to ensure it's 64-bit aligned. Note that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please read this comment on the alignment issues we had with some arch (ARM?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah good catch, thanks!
bbolt/db.go:38:9: struct with 336 pointer bytes could be 192
I guess still an improvement over 528 bytes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
is anyone able to validate the arm builds?
bbolt/db.go:38:9: struct of size 528 could be 496 bbolt/db.go:1012:12: struct with 40 pointer bytes could be 24 bbolt/db.go:1280:14: struct of size 104 could be 80 bbolt/freelist.go:24:15: struct with 136 pointer bytes could be 112 bbolt/node.go:12:11: struct with 88 pointer bytes could be 72 bbolt/tx.go:27:9: struct with 192 pointer bytes could be 88 bbolt/internal/common/inode.go:8:12: struct with 48 pointer bytes could be 32 bbolt/internal/common/page.go:324:15: struct with 16 pointer bytes could be 8 bbolt/cmd/bbolt/main.go:445:22: struct with 16 pointer bytes could be 8 bbolt/cmd/bbolt/main.go:1489:19: struct with 160 pointer bytes could be 104 bbolt/cmd/bbolt/main.go:1546:16: struct with 24 pointer bytes could be 16 bbolt/internal/btesting/btesting.go:28:9: struct with 48 pointer bytes could be 40 bbolt/tests/dmflakey/dmflakey.go:133:13: struct with 64 pointer bytes could be 56 Signed-off-by: Manuel Rüger <[email protected]>
Thanks for the improvement.
|
Please let me know if you have bandwidth to resolve my above comment #673 (comment) |
Thanks! I should have more bandwidth in the upcoming weeks. |
This runs betteralign to pack structs smarter. Originally proposed by etcd-io#673 Co-authored-by: Manuel Rüger <[email protected]> Signed-off-by: Thomas Jungblut <[email protected]>
Originally proposed by etcd-io#673 Co-authored-by: Manuel Rüger <[email protected]> Signed-off-by: Thomas Jungblut <[email protected]>
This runs betteralign to pack structs smarter. Originally proposed by etcd-io#673 Co-authored-by: Manuel Rüger <[email protected]> Signed-off-by: Thomas Jungblut <[email protected]>
@mrueg if you don't mind, I'm currently test running this with @ahrtr's suggestions applied in my own fork with openshift/etcd#278 - will report any e2e improvement I'm seeing. I think the benchmark tooling will need a little more time :) |
Just to leave some quick feedback here. I had some trouble with Go 1.22 and our build pipelines, so my test basically just consisted of running the installation routines (aka an IDLE cluster). There was no measurable improvement in memory usage or apiserver/etcd latency, I would only expect that under higher load though. That was also a bad apples/oranges test, because etcd 3.5 uses a different version of bbolt than what we're using here. I'll do another test run when I find some more time. |
This runs betteralign to pack structs smarter. Originally proposed by etcd-io#673 Co-authored-by: Manuel Rüger <[email protected]> Signed-off-by: Thomas Jungblut <[email protected]>
I run https://github.com/dkorunic/betteralign against the codebase and it showed the following improvements for memory alignment