Skip to content

debug.md

Latest commit

 

History

History
140 lines (92 loc) · 4.29 KB

debug.md

File metadata and controls

140 lines (92 loc) · 4.29 KB
Raw
title Debug

Debug seekdb

The supported Release build uses RelWithDebInfo, so it retains debug information while compiling with production optimization. There is no build.sh debug mode.

Build for debugging

./build.sh release --init --make

The binary is build_release/src/observer/seekdb. Record seekdb -V with any core dump or debug report so the binary revision and build flags can be matched. Optimized code may inline functions or report variables as optimized out; use logs or a narrower unit test when stepping through optimized code is impractical.

Attach GDB or LLDB

Find the process and attach GDB on Linux:

pidof seekdb
gdb build_release/src/observer/seekdb -p <pid>

On macOS, use LLDB:

lldb -p <pid>

For a core dump, use the exact binary that produced it:

gdb /path/to/seekdb /path/to/core

Set breakpoints, inspect variables, and obtain a backtrace with the usual debugger commands.

Use RPM debuginfo

The package installs the runtime binary as /usr/bin/seekdb. Install the matching debuginfo package when the configured repository provides one, or extract it without installing:

rpm2cpio seekdb-debuginfo-<version>.<arch>.rpm | cpio -idmv
find usr/lib/debug -type f -name '*seekdb*.debug'

Load the discovered file in GDB:

symbol-file /absolute/path/to/seekdb.debug

The runtime binary and debuginfo package must have the same revision and architecture. Do not rely on a fixed package path; use the file found in the extracted package.

Debug with logs

Logging is usually more effective than stopping a concurrent server in a debugger. Add structured fields with K() and rebuild the affected target:

LOG_DEBUG("insert sql generated", K(insert_sql), K(lbt()));

Use the configured base directory to locate logs. For a systemd installation, inspect both service output and the server log:

journalctl -u seekdb --since today
tail -F /var/lib/oceanbase/log/seekdb.log

If base-dir was changed in /etc/seekdb/seekdb.cnf, use the corresponding log directory. Search a request by its trace ID:

SELECT last_trace_id();

The trace ID appears in the log line; use rg or grep to find all related entries. The logging guide documents log fields, levels, rotation, and rate limiting.

Adjust logging while debugging

The following settings are dynamically effective cluster parameters:

ALTER SYSTEM SET syslog_level = 'DEBUG';
ALTER SYSTEM SET syslog_io_bandwidth_limit = '50MB';
ALTER SYSTEM SET diag_syslog_per_error_limit = 1000;
ALTER SYSTEM SET enable_async_syslog = false;

Restore the original values after debugging. Increasing log volume or disabling asynchronous logging can affect performance and disk usage.

Print and resolve a call stack

Include lbt() in a structured log when a source-level backtrace is useful:

LOG_DEBUG("state before retry", K(state), K(lbt()));

Resolve addresses with the same binary that produced the log. For example:

addr2line -pCfe build_release/src/observer/seekdb <address> ...

Use a binary with matching debug information; otherwise the output may contain only ?? frames.

SQL execution trace

Enable the session trace, run the statement, and inspect the recorded operations:

SET ob_enable_show_trace = 1;
-- run the statement to investigate
SHOW TRACE;

Disable the setting when it is no longer needed. The trace is intended for focused diagnosis and can add overhead.

Debug Sync

Debug Sync pauses a selected server thread at an existing DEBUG_SYNC point without stopping the entire process. It is useful when attaching a debugger would interfere with heartbeats or concurrent activity.

Enable the facility, configure a point, and signal it from another session:

ALTER SYSTEM SET debug_sync_timeout = '100000s';
SET ob_global_debug_sync = 'BEFORE_UNIT_MANAGER_LOAD wait_for signal_name execute 10000';
SET ob_global_debug_sync = 'now signal signal_name';

Clear the point and disable Debug Sync when finished:

SET ob_global_debug_sync = 'BEFORE_UNIT_MANAGER_LOAD clear';
ALTER SYSTEM SET debug_sync_timeout = 0;

The point name must exist in the code path being tested. Adding a new point requires adding DEBUG_SYNC(...) in source code and rebuilding the affected target.