How to use static library artifacts in another build system

[gtkramer]

I am attempting to create a static redistributable libtcmalloc.a with the Bazel build so that the library may be brought into a different build system via the -I and -L options to gcc, but I'm not having much luck.

I can coerce Bazel to produce static libraries for everything by running from the root of the repo directory (hacky, and probably slightly wrong):

find . -name BUILD -type f -exec sed -i -e '/alwayslink = 1/d' -e '/linkstatic = 1/d' {} +
find . -name BUILD -type f -exec sed -i -r 's/cc_library\(/&\n    linkstatic = True,/' {} +

But when I run on the libtcmalloc.a that is produced:

ar x libtcmalloc.a

I only see tcmalloc.pic.o. If I compare this to the libtcmalloc_minimal.a that is produced from gperftools, I see a number of additional object files; which is more along the lines of what I was epxecting.

I've also written a rough CMake build with the help of a bazel-to-cmake conversation tool, and I saw the same problem with the produced libtcmalloc.a as well.

What would be the recommended way to handle this while also accounting for tcmalloc's new dependency on Abseil? Or is there maybe a Bazel option I'm missing? I'm interested in statically linking everything, and ideally, libtcmalloc.a would include all objects that are needed for it to be consumed by another program and not get undefined reference to 'tcmalloc::Static::transfer_cache_' for example during the linking stage.

[ckennelly]

Side point: If you're using Bazel for the TCMalloc build, https://github.com/bazelbuild/buildtools/tree/master/buildozer can help with manipulations of the BUILD files. That said, I think your sed commands are accomplishing their goal correctly.

I think the issue is that TCMalloc is comprised of multiple libraries which ordinarily all get pulled in by statically linking the main TCMalloc target. Most of this is an implementation detail, for ease of testing (common.a, several .a's under internal, etc.), but these also have dependencies on Abseil. Linking in these libraries would likely resolve the undefined reference problem.

The broader issue, though, is with a fully static tcmalloc.a inclusive of the Abseil dependencies, since any binary that links both that flavor of TCMalloc would get ODR clashes if it also depended on Abseil.

[gtkramer]

Ah, I see. Thank you for the explanation. There are pros and cons of depending on third party libraries, I guess. I was hoping to use this version in some projects I'm working on, but I can continue to use the latest version from gperftools since my projects use autotools for building.

How is this tcmalloc different from the one that's included in gperftools? Is the one in gperftools planned to be EOL'd? Does this tcmalloc have API changes or maybe more recent cleanup/performance work that the one in gperftools doesn't have?

[ckennelly]

gperftools was released by Google as Google Performance Tools many years ago. It diverged from Google's internal version--at the time, it was hard to keep things in-sync as described by Titus Winters in his 2017 CppCon talk and the "Software Engineering at Google Book"--and ended up being adopted by the community. There's further background here.

This version follows the one used in Google production and has a number of new features and optimizations.

[gtkramer]

Why didn't I Google that? :D Long day after work maybe. Thank you so much for the help and fast response here. We might want to revisit trying to get this version of tcmalloc into our projects again, perhaps by spending time to contribute an Abesil-like CMake build as discussed in #4. As far as I'm concerned, this can be closed from my end. I think I have everything I need for now.

[zhaopengoog]

Hi, @ckennelly

Can you comment on the performance impact on single-threaded benchmarks (such as SPEC2006, SPEC2017) of these two tcmalloc versions (i.e. tcmalloc in gperftools and the newer git-tcmalloc)? There are a couple of SPEC benchmarks that benefit noticeably from tcmalloc. Does the newer tcmalloc have significant advantage over gperftools tcmallocs on these single-threaded use cases?

Thanks