Linux性能分析
2019-11-06 20:52
1751 查看
生产环境服务器变慢，诊断思路和性能评估
整机：top
代码
public class JavaDemo2 {
public static void main(String[] args) {
while (true){
System.out.println(new java.util.Random().nextInt(77778888));
}
}
}
top命令查看
[root@192 ~]# top top - 11:28:50 up 7 min, 2 users, load average: 0.73, 0.39, 0.18 Tasks: 230 total, 1 running, 229 sleeping, 0 stopped, 0 zombie Cpu(s): 1.0%us, 2.3%sy, 0.0%ni, 96.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1906280k total, 649948k used, 1256332k free, 21164k buffers Swap: 2097148k total, 0k used, 2097148k free, 203820k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3672 root 20 0 3158m 94m 11m S 15.6 5.1 0:13.67 java 3397 root 20 0 100m 5388 3396 S 9.3 0.3 0:09.00 sshd 39 root 20 0 0 0 0 S 1.7 0.0 0:00.25 events/4 37 root 20 0 0 0 0 S 1.0 0.0 0:00.39 events/2 35 root 20 0 0 0 0 S 0.3 0.0 0:01.27 events/0 36 root 20 0 0 0 0 S 0.3 0.0 0:00.34 events/1 41 root 20 0 0 0 0 S 0.3 0.0 0:00.24 events/6 [root@192 ~]# uptime 11:29:09 up 7 min, 2 users, load average: 0.52, 0.36, 0.18
CPU：vmstat
[root@192 ~]# vmstat -n 2 3 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 1313028 21228 203868 0 0 54 3 112 1620 0 1 98 0 0 0 0 0 1312732 21228 203896 0 0 0 0 2579 25940 1 3 96 0 0 0 0 0 1312856 21228 203896 0 0 0 0 2677 24290 1 3 97 0 0
id:处于空闲的CPU百分比
wa:系统等待IO的CPU时间百分比
st:来自于一个虚拟机偷取的CPU时间百分比
查看额外的CPU信息
所有CPU核信息
[root@192 ~]# mpstat -P ALL 2 Linux 2.6.32-642.el6.x86_64 (192.168.1.101) 08/05/2019 _x86_64_ (8 CPU) 11:42:40 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle 11:42:42 AM all 1.07 0.00 2.89 0.00 0.00 0.00 0.00 0.00 96.04 11:42:42 AM 0 0.00 0.00 1.52 0.00 0.00 0.00 0.00 0.00 98.48 11:42:42 AM 1 6.09 0.00 9.14 0.00 0.00 0.00 0.00 0.00 84.77 11:42:42 AM 2 0.50 0.00 4.02 0.00 0.00 0.00 0.00 0.00 95.48 11:42:42 AM 3 1.00 0.00 2.00 0.00 0.00 0.00 0.00 0.00 97.00
每个进程使用CPU的用量分解信息
[root@192 ~]# pidstat -u 1 -p 3790 Linux 2.6.32-642.el6.x86_64 (192.168.1.101) 08/05/2019 _x86_64_ (8 CPU) 11:45:00 AM PID %usr %system %guest %CPU CPU Command 11:45:01 AM 3790 6.00 10.00 0.00 16.00 7 java 11:45:02 AM 3790 4.00 11.00 0.00 15.00 7 java 11:45:03 AM 3790 9.00 15.00 0.00 24.00 7 java 11:45:04 AM 3790 7.00 14.00 0.00 21.00 7 java
内存：free
应用程序可用内存数
推荐使用
free -m
[root@192 ~]# free -g total used free shared buffers cached Mem: 1 0 1 0 0 0 -/+ buffers/cache: 0 1 Swap: 1 0 1 [root@192 ~]# free -m total used free shared buffers cached Mem: 1861 553 1308 1 20 199 -/+ buffers/cache: 333 1528 Swap: 2047 0 2047
查看额外
[root@192 ~]# pidstat -p 3823 -r 2 Linux 2.6.32-642.el6.x86_64 (192.168.1.101) 08/05/2019 _x86_64_ (8 CPU) 11:51:16 AM PID minflt/s majflt/s VSZ RSS %MEM Command 11:51:18 AM 3823 1.00 0.00 3234728 47068 2.47 java 11:51:20 AM 3823 2.50 0.00 3234728 47068 2.47 java 11:51:22 AM 3823 0.50 0.00 3234728 47068 2.47 java 11:51:24 AM 3823 2.50 0.00 3234728 47068 2.47 java 11:51:26 AM 3823 0.50 0.00 3234728 47068 2.47 java
硬盘：df
查看磁盘剩余空间
[root@192 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 15G 5.6G 8.4G 41% / tmpfs 931M 72K 931M 1% /dev/shm /dev/sda1 190M 39M 142M 22% /boot
磁盘IO：iostat
[root@192 ~]# iostat -xdk 2 3 Linux 2.6.32-642.el6.x86_64 (192.168.1.101) 08/05/2019 _x86_64_ (8 CPU) Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util scd0 0.00 0.00 0.38 0.00 1.50 0.00 8.00 0.00 0.67 0.67 0.00 0.67 0.03 sda 34.33 4.94 47.36 3.18 1631.23 32.44 65.84 0.07 1.38 1.22 3.72 0.88 4.43 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util scd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.50 0.00 4.00 0.00 16.00 0.00 1.00 1.00 0.00 1.00 0.05 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util scd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
查看额外
[root@192 ~]# pidstat -d 2 -p 3485 Linux 2.6.32-642.el6.x86_64 (192.168.1.101) 08/05/2019 _x86_64_ (8 CPU) 02:16:10 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command 02:16:12 PM 3485 0.00 0.00 0.00 java 02:16:14 PM 3485 0.00 0.00 0.00 java 02:16:16 PM 3485 0.00 0.00 0.00 java 02:16:18 PM 3485 0.00 0.00 0.00 java 02:16:20 PM 3485 0.00 0.00 0.00 java 02:16:22 PM 3485 0.00 0.00 0.00 java 02:16:24 PM 3485 0.00 0.00 0.00 java
网络IO：ifstat
默认本地没有，需要下载ifstat
安装ifstat
[root@192 ifstat-1.1]# yum install flex byacc libpcap ncurses ncurses-devel libpcap-devel gcc-c++ wget http://gael.roualland.free.fr/ifstat/ifstat-1.1.tar.gz tar xzvf ifstat-1.1.tar.gz cd ifstat-1.1 ./configure make make install
[root@192 ifstat-1.1]# ifstat 1 eth0 KB/s in KB/s out 0.06 0.13 0.06 0.12 0.06 0.12 0.06 0.12 0.06 0.12 0.06 0.12 0.12 0.17 0.06 0.12
生产环境出现CPU占用过高，分析思路和定位
public class JavaDemo2 {
public static void main(String[] args) {
while (true){
System.out.println(new java.util.Random().nextInt(77778888));
}
}
}
先用top命令找出CPU占比最高的
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4294 root 20 0 3158m 37m 11m S 16.6 2.0 0:03.28 java 3428 root 20 0 100m 5444 3396 S 14.3 0.3 0:22.85 sshd 35 root 20 0 0 0 0 S 1.0 0.0 0:03.18 events/0 41 root 20 0 0 0 0 S 0.3 0.0 0:00.52 events/6 2012 root 20 0 199m 5192 4288 S 0.3 0.3 0:00.32 ManagementAgent 576 1 root 20 0 19344 1544 1228 S 0.0 0.1 0:01.83 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
ps -ef或者jps进一步定位
[root@192 ifstat-1.1]# jps 4320 Jps 4294 JavaDemo2
定位到具体线程或者代码
[root@192 ifstat-1.1]# ps -mp 4294 -o THREAD,tid,time USER %CPU PRI SCNT WCHAN USER SYSTEM TID TIME root 18.6 - - - - - - 00:00:15 root 0.0 19 - futex_ - - 4294 00:00:00 root 15.6 19 - n_tty_ - - 4295 00:00:13 root 0.2 19 - futex_ - - 4296 00:00:00 root 0.2 19 - futex_ - - 4297 00:00:00 root 0.3 19 - futex_ - - 4298 00:00:00 root 0.2 19 - futex_ - - 4299 00:00:00 root 0.3 19 - futex_ - - 4300 00:00:00 root 0.3 19 - futex_ - - 4301 00:00:00 root 0.3 19 - futex_ - - 4302 00:00:00 root 0.2 19 - futex_ - - 4303 00:00:00 root 0.0 19 - futex_ - - 4304 00:00:00 root 0.0 19 - futex_ - - 4305 00:00:00 root 0.0 19 - futex_ - - 4306 ae4 00:00:00 root 0.0 19 - futex_ - - 4307 00:00:00 root 0.0 19 - futex_ - - 4308 00:00:00 root 0.1 19 - futex_ - - 4309 00:00:00 root 0.0 19 - futex_ - - 4310 00:00:00 root 0.0 19 - futex_ - - 4311 00:00:00 root 0.0 19 - futex_ - - 4312 00:00:00 root 0.0 19 - futex_ - - 4313 00:00:00
-m:显示所有的线程
-p: pid进程使用CPU的时间
-o:该参数后是用户自定义格式
线程ID转换为16进制格式（英文小写格式）
[root@192 ifstat-1.1]# printf "%x\n" 4295 10c7
jstack 进程ID|grep tid(16进制线程ID小写英文) -A60
[root@192 ifstat-1.1]# jstack 4294 | grep 10c7 -A60 "main" #1 prio=5 os_prio=0 tid=0x00007f7f68009000 nid=0x10c7 runnable [0x00007f7f6f117000] java.lang.Thread.State: RUNNABLE at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) - locked <0x00000000e2e0e6e0> (a java.io.BufferedOutputStream) at java.io.PrintStream.write(PrintStream.java:482) - locked <0x00000000e2e0bbd8> (a java.io.PrintStream) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) - locked <0x00000000e2e0e800> (a java.io.OutputStreamWriter) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.newLine(PrintStream.java:546) - eliminated <0x00000000e2e0bbd8> (a java.io.PrintStream) at java.io.PrintStream.println(PrintStream.java:737) - locked <0x00000000e2e0bbd8> (a java.io.PrintStream) at JavaDemo2.main(JavaDemo2.java:4) "VM Thread" os_prio=0 tid=0x00007f7f68080000 nid=0x10d0 runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f7f6801e800 nid=0x10c8 runnable "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f7f68020000 nid=0x10c9 runnable "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007f7f68022000 nid=0x10ca runnable "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007f7f68024000 nid=0x10cb runnable "GC task thread#4 (ParallelGC)" os_prio=0 tid=0x00007f7f680 1484 25800 nid=0x10cc runnable "GC task thread#5 (ParallelGC)" os_prio=0 tid=0x00007f7f68027800 nid=0x10cd runnable "GC task thread#6 (ParallelGC)" os_prio=0 tid=0x00007f7f68029800 nid=0x10ce runnable "GC task thread#7 (ParallelGC)" os_prio=0 tid=0x00007f7f6802b000 nid=0x10cf runnable "VM Periodic Task Thread" os_prio=0 tid=0x00007f7f680e8000 nid=0x10d9 waiting on condition JNI global references: 5
参考
本文由博客一文多发平台 OpenWrite 发布！
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- vmstat命令之linux性能分析
- Linux下Java线程详细监控和其dump的分析使用----分析Java性能瓶颈
- 怎样使用gprof和oprofile来分析linux程序的性能
- Linux C程序性能分析
- Unix&Linux下常见的性能分析工具介绍
- linux性能分析汇总
- perf学习-linux自带性能分析工具
- 和菜鸟一起学linux之linux性能分析工具oprofile移植
- linux命令（4）：top 命令（性能分析工具）
- linux系统性能调优第一步——性能分析(vmstat)
- linux中top性能分析工具中的TIME+
- Linux五种IO模型性能分析
- Linux系统性能分析
- Linux系统性能分析——网络
- Linux下使用NMON监控、分析系统性能
- linux 性能分析工具整理
- free命令具体解释——Linux性能分析
- Linux下的CPU性能瓶颈分析
- Linux 下使用 NMON 分析系统性能
- Linux系统性能分析