Memcache 内存分配策略和性能(使用)状态检查【转】

前言：

一直在使用Memcache，但是对其内部的问题，如它内存是怎么样被使用的，使用一段时间后想看看一些状态怎么样？一直都不清楚，查了又忘记，现在整理出该篇文章，方便自己查阅。本文不涉及安装、操作。有兴趣的同学可以查看之前写的文章和Google。

1：参数

#!/usr/bin/perl
#
# memcached-tool:
#   stats/management tool for memcached.
#
# Author:
#   Brad Fitzpatrick <brad@danga.com>
#
# Contributor:
#   Andrey Niakhaichyk <andrey@niakhaichyk.org>
#
# License:
#   public domain.  I give up all rights to this
#   tool.  modify and copy at will.
#

use strict;
use IO::Socket::INET;

my $addr = shift;
my $mode = shift || "display";
my ($from, $to);

if ($mode eq "display") {
undef $mode if @ARGV;
} elsif ($mode eq "move") {
$from = shift;
$to = shift;
undef $mode if $from < 6 || $from > 17;
undef $mode if $to   < 6 || $to   > 17;
print STDERR "ERROR: parameters out of range\n\n" unless $mode;
} elsif ($mode eq 'dump') {
;
} elsif ($mode eq 'stats') {
;
} elsif ($mode eq 'settings') {
;
} elsif ($mode eq 'sizes') {
;
} else {
undef $mode;
}

undef $mode if @ARGV;

die
"Usage: memcached-tool <host[:port] | /path/to/socket> [mode]\n
memcached-tool 10.0.0.5:11211 display    # shows slabs
memcached-tool 10.0.0.5:11211            # same.  (default is display)
memcached-tool 10.0.0.5:11211 stats      # shows general stats
memcached-tool 10.0.0.5:11211 settings   # shows settings stats
memcached-tool 10.0.0.5:11211 sizes      # shows sizes stats
memcached-tool 10.0.0.5:11211 dump       # dumps keys and values
WARNING! sizes is a development command.
As of 1.4 it is still the only command which will lock your memcached instance for some time.
If you have many millions of stored items, it can become unresponsive for several minutes.
Run this at your own risk. It is roadmapped to either make this feature optional
or at least speed it up.
" unless $addr && $mode;

my $sock;
if ($addr =~ m:/:) {
$sock = IO::Socket::UNIX->new(
Peer => $addr,
);
}
else {
$addr .= ':11211' unless $addr =~ /:\d+$/;

$sock = IO::Socket::INET->new(
PeerAddr => $addr,
Proto    => 'tcp',
);
}
die "Couldn't connect to $addr\n" unless $sock;

if ($mode eq 'dump') {
my %items;
my $totalitems;

print $sock "stats items\r\n";

while (<$sock>) {
last if /^END/;
if (/^STAT items:(\d*):number (\d*)/) {
$items{$1} = $2;
$totalitems += $2;
}
}
print STDERR "Dumping memcache contents\n";
print STDERR "  Number of buckets: " . scalar(keys(%items)) . "\n";
print STDERR "  Number of items  : $totalitems\n";

foreach my $bucket (sort(keys(%items))) {
print STDERR "Dumping bucket $bucket - " . $items{$bucket} . " total items\n";
print $sock "stats cachedump $bucket $items{$bucket}\r\n";
my %keyexp;
while (<$sock>) {
last if /^END/;
# return format looks like this
# ITEM foo [6 b; 1176415152 s]
if (/^ITEM (\S+) \[.* (\d+) s\]/) {
$keyexp{$1} = $2;
}
}

foreach my $k (keys(%keyexp)) {
print $sock "get $k\r\n";
my $response = <$sock>;
if ($response =~ /VALUE (\S+) (\d+) (\d+)/) {
my $flags = $2;
my $len = $3;
my $val;
read $sock, $val, $len;
print "add $k $flags $keyexp{$k} $len\r\n$val\r\n";
# get the END
$_ = <$sock>;
$_ = <$sock>;
}
}
}
exit;
}

if ($mode eq 'stats') {
my %items;

print $sock "stats\r\n";

while (<$sock>) {
last if /^END/;
chomp;
if (/^STAT\s+(\S*)\s+(.*)/) {
$items{$1} = $2;
}
}
printf ("#%-17s %5s %11s\n", $addr, "Field", "Value");
foreach my $name (sort(keys(%items))) {
printf ("%24s %12s\n", $name, $items{$name});

}
exit;
}

if ($mode eq 'settings') {
my %items;

print $sock "stats settings\r\n";

while (<$sock>) {
last if /^END/;
chomp;
if (/^STAT\s+(\S*)\s+(.*)/) {
$items{$1} = $2;
}
}
printf ("#%-17s %5s %11s\n", $addr, "Field", "Value");
foreach my $name (sort(keys(%items))) {
printf ("%24s %12s\n", $name, $items{$name});
}
exit;
}

if ($mode eq 'sizes') {
my %items;

print $sock "stats sizes\r\n";

while (<$sock>) {
last if /^END/;
chomp;
if (/^STAT\s+(\S*)\s+(.*)/) {
$items{$1} = $2;
}
}
printf ("#%-17s %5s %11s\n", $addr, "Size", "Count");
foreach my $name (sort(keys(%items))) {
printf ("%24s %12s\n", $name, $items{$name});
}
exit;
}

# display mode:

my %items;  # class -> { number, age, chunk_size, chunks_per_page,
#            total_pages, total_chunks, used_chunks,
#            free_chunks, free_chunks_end }

print $sock "stats items\r\n";
my $max = 0;
while (<$sock>) {
last if /^END/;
if (/^STAT items:(\d+):(\w+) (\d+)/) {
$items{$1}{$2} = $3;
}
}

print $sock "stats slabs\r\n";
while (<$sock>) {
last if /^END/;
if (/^STAT (\d+):(\w+) (\d+)/) {
$items{$1}{$2} = $3;
$max = $1;
}
}

print "  #  Item_Size  Max_age   Pages   Count   Full?  Evicted Evict_Time OOM\n";
foreach my $n (1..$max) {
my $it = $items{$n};
next if (0 == $it->{total_pages});
my $size = $it->{chunk_size} < 1024 ?
"$it->{chunk_size}B" :
sprintf("%.1fK", $it->{chunk_size} / 1024.0);
my $full = $it->{free_chunks_end} == 0 ? "yes" : " no";
printf("%3d %8s %9ds %7d %7d %7s %8d %8d %4d\n",
$n, $size, $it->{age}, $it->{total_pages},
$it->{number}, $full, $it->{evicted},
$it->{evicted_time}, $it->{outofmemory});
}

./memcached-tool 10.211.55.9:11212    --执行
#  Item_Size  Max_age   Pages   Count   Full?  Evicted Evict_Time OOM
1      96B     20157s      28  305816     yes 95431913        0    0
2     120B     16049s      40  349520     yes 117041737        0    0
3     152B     17574s      39  269022     yes 92679465        0    0
4     192B     18157s      43  234823     yes 78892650        0    0
5     240B     18722s      52  227188     yes 72908841        0    0
6     304B     17971s      73  251777     yes 85556469        0    0
7     384B     17881s      81  221130     yes 75596858        0    0
8     480B     17760s      70  152880     yes 53553607        0    0
9     600B     18167s      58  101326     yes 34647962        0    0
10     752B     18518s      52   72488     yes 24813707        0    0
11     944B     18903s      52   57720     yes 16707430        0    0
12     1.2K     20475s      44   38940     yes 11592923        0    0
13     1.4K     21220s      36   25488     yes  8232326        0    0
14     1.8K     22710s      35   19740     yes  6232766        0    0
15     2.3K     22027s      33   14883     yes  4952017        0    0
16     2.8K     23139s      33   11913     yes  3822663        0    0
17     3.5K     23495s      31    8928     yes  2817520        0    0
18     4.4K     22611s      29    6670     yes  2168871        0    0
19     5.5K     23652s      29    5336     yes  1636656        0    0
20     6.9K     21245s      26    3822     yes  1334189        0    0
21     8.7K     22794s      22    2596     yes   783620        0    0
22    10.8K     22443s      19    1786     yes   514953        0    0
23    13.6K     21385s      18    1350     yes   368016        0    0
24    16.9K     23782s      16     960     yes   254782        0    0
25    21.2K     23897s      14     672     yes   183793        0    0
26    26.5K     27847s      13     494     yes   117535        0    0
27    33.1K     27497s      14     420     yes    83966        0    0
28    41.4K     28246s      14     336     yes    63703        0    0
29    51.7K     33636s      12     228     yes    24239        0    0

解释：

列	含义
#	slab class编号
Item_Size	chunk大小
Max_age	LRU内最旧的记录的生存时间
pages	分配给Slab的页数
count	Slab内的记录数、chunks数、items数、keys数
Full?	Slab内是否含有空闲chunk
Evicted	从LRU中移除未过期item的次数
Evict_Time	最后被移除缓存的时间，0表示当前就有被移除
OOM	-M参数？

4，总结

实际应用Memcached时，我们遇到的很多问题都是因为不了解其内存分配机制所致，希望本文能让大家初步了解Memcached在内存方便的分配机制，虽然redis等一些nosql的数据库产品在很多产品中替换了memcache，但是memcache还有很多项目会依赖它，所以还得学习来解决问题，后续出现新内容会不定时更新。

5，参考文档

Memcached内存分析、调优、集群

memcache内存分配、性能检测

memcached的基础


理解memcached的内存存储

memcached的删除机制和发展方向

memcached的分布式算法

memcached的应用和兼容程序

Memcached二三事儿

转自

Memcache 内存分配策略和性能(使用)状态检查 - jyzhou - 博客园 https://www.cnblogs.com/zhoujinyi/p/5554083.html