Linux细节 - 常用Linux下c语言调试方法

本文主要总结一下Linux下调试c语言进程的常用方法。

1. dmesg

dmesg主要用于在进程崩溃掉的时候，显示内核的相关信息。

dmesg | tail -f   #打印当前信息
dmesg -c #打印后，清除缓冲区内容
dmesg -s 1024 #定义缓冲区大小

[root@localhost togo]# dmesg | tail -f
[   11.859146] sr 1:0:0:0: Attached scsi generic sg1 type 5
[   12.291406] ip_tables: (C) 2000-2006 Netfilter Core Team
[   12.380055] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[   12.417066] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   12.493635] Ebtables v2.0 registered
[   12.525655] Bridge firewalling registered
[   12.907898] e1000: eno16777736 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[   12.910863] IPv6: ADDRCONF(NETDEV_UP): eno16777736: link is not ready
[   12.911492] IPv6: ADDRCONF(NETDEV_CHANGE): eno16777736: link becomes ready
[   16.225872] Ebtables v2.0 unregistered

2. ldd

ldd命令主要查看进程依赖的共享链接库。

#Togo依赖了线程库和libevent的库

[root@localhost togo]# ldd ./togo
	linux-vdso.so.1 =>  (0x00007ffff374b000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa06c8d6000)
	libevent-2.0.so.5 => /lib64/libevent-2.0.so.5 (0x00007fa06c68e000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa06c2cc000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa06cafc000)

3. core文件

当程序偶发性崩溃的时候，我们可以借助系统的core dump来查看崩溃时候的错误信息。排查段错误等异常信息非常给力。

开启core dump，首选需要设置ulimit

[root@localhost togo]# ulimit -c unlimited

一般情况下，core dump后的文件会在当前目录，也有可能是统一的core dump目录。统一目录可以查看：

[root@localhost home]# cat  /proc/sys/kernel/core_pattern
/tmp/core-%e-%s-%u-%g-%p-%t

使用gdb命令调试：

[root@localhost home]# gdb ./main ./core-main-11-0-0-3424-1451986509 
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-51.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/main...(no debugging symbols found)...done.
[New LWP 3424]
Core was generated by `./main'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000400500 in main ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-55.el7_0.1.x86_64
(gdb) where
#0  0x0000000000400500 in main ()
(gdb) quit

输入where可以迅速查看异常的位置；输入quit迅速退出。

4. pstack

pstack一般查看线程。对于多线程c程序，容易引起死锁，使用pstack来检查是否有死锁情况。

#Togo有9个线程，一个是主线程，一个监听线程，8个libevent的epoll wait

[root@localhost home]# ps -ef | grep togo
root      2999     1  8 16:14 ?        00:08:16 ./togo
root      3518  2851  0 17:51 pts/0    00:00:00 grep --color=auto togo
[root@localhost home]# pstack 2999
Thread 10 (Thread 0x7f6f75079700 (LWP 3000)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x0000000000401d22 in togo_mt_process (args=<optimized out>) at togo_server.c:183
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x7f6f74878700 (LWP 3001)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x7f6f6ffff700 (LWP 3002)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x7f6f6f7fe700 (LWP 3003)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7f6f6effd700 (LWP 3004)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7f6f6e7fc700 (LWP 3005)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7f6f6dffb700 (LWP 3006)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7f6f6d7fa700 (LWP 3007)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f6f6cff9700 (LWP 3008)):
#0  0x00007f6f79bba5f3 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f6f79ea9803 in epoll_dispatch () from /lib64/libevent-2.0.so.5
#2  0x00007f6f79e953ea in event_base_loop () from /lib64/libevent-2.0.so.5
#3  0x000000000040208f in togo_wt_cb (args=<optimized out>) at togo_server.c:338
#4  0x00007f6f7a0d4df3 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f6f79bba01d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f6f7a4fb740 (LWP 2999)):
#0  0x00007f6f7a0d5f37 in pthread_join () from /lib64/libpthread.so.0
#1  0x00000000004026f5 in togo_server_init () at togo_server.c:103
#2  0x0000000000401b5a in togo_init (argc=1, argv=0x7fffae18a178) at togo_init.c:24
#3  0x00007f6f79ae5af5 in __libc_start_main () from /lib64/libc.so.6
#4  0x00000000004017e1 in _start ()