compile-db-gen

Generate the JSON compilation database (compile_commands.json) from tracing the make command (or other compile tools).

Why we need the compile-db-gen

The compile-db-gen can work in complicate compile environment to generate the compile_commands.json.

Or, the compile-db-gen can save us from hardly getting the compile_commands.json scenario.

Here are some examples that getting the compile_commands.json easyly:

1. A project orgnized with CMake, just run cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON to get the compile_commands.json.
2. A project orgnized with automake, run make with Build EAR will get the compile_commands.json correctly.
3. A project orgnized with Makefile and using the CC/CXX variables, run scan-build can get the comile_commands.json.

… Beside these, it maybe hard to get the compile_commands.json for some complicated scenario.

Here is an example that can not get the compile_commands.json with previous tools:

The compile steps will invoke some elf-64 tools to build some dirs, and will invoke some elf-32 tools to build other dirs, like:

make
-> elf-32 tool
   -> gcc(32bits) ...
-> elf-64 tool
   -> gcc(64bits) ...

Then the compile-db-gen.py can do well on these dynamic compile scenario to get the compile_commands.json.

Usage

Just run the compile-db-gen.py with make and its arguments, eg:

./compile-db-gen.py make -j1

Then it will generate the compile_commands.json in current directory.

And follow commands will give more details:

./compile-db-gen.py trace make <OPTS>
./compile-db-gen.py parse compile_commands.raw compile_commands.json

For more options, please run

./compile-db-gen.py -h

How dose it work

The strace tool can trace system calls (execve, chdir) and write their arguments to log file.

Then the compile-db-gen.py follow the log file to construct the process calling tree, and output the compile commands.

Requirments

The strace-4.8 or high is required; python3 is required.

The compile-db-gen.py will track the child-process terminal message which contain parent pid from the strace log.

The follow log give hists that child(pid=937) is ended, and the pid for its parent is pid=936.

[pid 936] — SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=937, si_status=0, si_utime=0, si_stime=0} —

but the strace-4.5 seems doesn’t give the child pid.

[pid 936] — si_code=CLD_EXITED SIGCHLD 0 —

I didn’t try the compile-db-gen.py with the strace-4.6, strace-4.7, if you had run the compile-db-gen.py with starce-4.6/4.7, please let me know it’s success or failed.

Similar tools

scan-build It replaces the CC, CXX environment string, then get the compile database, but lots of projects didn’t use the CC, CXX environment settings, then scan-build won’t work correctly.

Build EAR (Bear) It inject the libbear.so via modify the environment variables LD_PRELOAD or DYLD_INSERT_LIBRARIES, then filter the compile command. But the Bear doesn’t work for a complicated sencen, for example, a makefile called both x86 and x64 exetable binaries, but the libear.so is ELF-64, then the LD_PRELOAD will fail on tracing the x86 binaries.

clang-tags trace The clang-tags trace just simply parse the strace log line by line, will give wrong result for it didn’t walk through the building tree.