How to monitor memcached

There are several ways to retrieve stats for your memcached daemon, and to confirm all is well with how it works. Most of the time you'll want to monitor cache HITS and MISSES, so we'll primarily focus on this aspect.

Pick your preferred monitoring tool

Whether you don't want to install any additional package or fancy more detailed statistics, your preferred monitoring tool will vary.

Let's make a few assumptions first to better understand the context of this post:

  • Your database server is running Ubuntu/Debian.
  • You have one or several web nodes and at least one database server (alternatively you could have a primary/replica).
  • memcached is installed on the remote database server.
  • The database server's IPv4 address is 203.0.113.12
  • memcached is listening on its default port (11211)
  • netcat is installed

Using netcat

In most Ubuntu-based distributions and Debian, netcat will be available by default. This allows to run commands such as:

$ echo stats | nc 203.0.113.12 11211
STAT pid 71  
STAT uptime 6949  
STAT time 1486379640  
STAT version 1.4.21  
STAT libevent 2.0.21-stable  
STAT pointer_size 64  
(snipped)
END  

This gives you extensive stats (uptime, version, HITS, MISSES, connections, evictions...) about the memcached daemon. If you wish to retrieve HITS in real-time, you can build upon this command and do the following.

$ watch -td '(echo stats ; echo quit) | nc 203.0.113.12 11211 | grep get_hits '
STAT get_hits 1987  

The number will then increase as soon as there's data coming in through the memcached daemon.

Using memcstat

Installing the libmemcached-tools package will give you access to the memcstat command, which displays the operating status of a single or group of memcached servers. Usage is very simple.

$ memcstat --servers=203.0.113.12 11211 | grep get_hits
     get_hits: 1207

Once again we've filtered stats out to retrieve memcached HITS only. You get the point.

Using telnet

Telnet is usually installed on Ubuntu-based distributions but not on Debian. It's a network protocol that can help you with interacting with the memcached daemon.

Since we're primarily interested into high-level memcached monitoring here, we'll simply use the stats command here.

telnet 203.0.113.12 11211  
Trying 203.0.113.12...  
Connected to 203.0.113.12.  
Escape character is '^]'.  
stats  
STAT pid 71  
STAT uptime 10791  
STAT time 1486383482  
STAT version 1.4.21  
STAT libevent 2.0.21-stable  
STAT pointer_size 64  
STAT rusage_user 0.356000  
STAT rusage_system 0.652000  
STAT curr_connections 12  
STAT total_connections 804  
STAT connection_structures 13  
(snipped)

Using memcached-top

memcached-top was originally hosted on Google Code and now lives on Github. It's a perl file that allows you to watch the traffic and stats in real-time. It wasn't updated in a while but still seems to work fine.

$ perl memcache-top --instance=203.0.113.12
memcache-top v0.7    (default port: 11211, color: on, refresh: 3 seconds)

INSTANCE              USAGE HIT %   CONN    TIME    ITEMS   EVICT/s READ/s  WRITE/s  
203.0.113.12:11211    5.3%  90.7%   10      0.6ms   867     0.0     9207    118.2K  
AVERAGE:              5.3%  93.4%   10      0.7ms   867     0.0     9831    114.9K  

TOTAL: 3.4MB/64.0MB                 10      0.6ms   867     0.0     8999    117.2K  
(ctrl-c to quit.)

Using mcstat

I find mcstat more interesting because it's also more advanced. It wasn't updated in a while so it might behave inconsistently as memcached evolves.

It reports on memcached daemon statistics, but in real-time, and arbitrarily returns only the stats that really matter to you when evaluating if memcached is doing its job or not. When you exit it, it immediately returns session statistics.

$ mcstat 203.0.113.12 11211
Memcache version 1.4.21  
Max Memory Size: 64.0M  Pointer size: 64 bit

    time  cnct    used    gets    hits    miss    sets    evic   miss%   evic%      in     out
10:23:23    10    2.7M    1.3k    1.2k      87    6.1k       0    6.72    0.00   24.8M   11.0M

10:23:28    10    2.7M       0       0       0       0       0    0.00    0.00       7    1.1k  
10:23:33    10    2.7M     209     206       3       7       0    1.44    0.00   20.0k    1.4M  
10:23:38    10    2.7M     202     178      24      37       0   11.88    0.00   95.8k    1.6M  
10:23:43    10    2.7M     209     175      34      43       0   16.27    0.00  181.9k    2.2M

Session Statistics:  
             Statistic             Total           Session
                uptime         1h 5m 12s               20s
              libevent     2.0.21-stable                 0
           rusage_user             0.24s             0.01s
         rusage_system             0.45s             0.03s
      curr_connections                                  10
     total_connections               644                10
 connection_structures                12                 0
          reserved_fds                20                 0
               cmd_get              1.3k               620
               cmd_set              6.1k                87
             cmd_flush                 0                 0
             cmd_touch                 0                 0
              get_hits              1.2k               559
            get_misses                87                61
         delete_misses                 3                 1
           delete_hits                18                 1
           incr_misses                 0                 0
             incr_hits                 0                 0
           decr_misses                 0                 0
             decr_hits                 0                 0
            cas_misses                 0                 0
              cas_hits                 0                 0
            cas_badval                 0                 0
            touch_hits                 0                 0
          touch_misses                 0                 0
             auth_cmds                 0                 0
           auth_errors                 0                 0
            bytes_read             24.8M            297.7k
         bytes_written             11.0M              5.2M
       accepting_conns                 1                 0
   listen_disabled_num                 0                 0
               threads                 4                 0
           conn_yields                 0                 0
      hash_power_level                16                 0
            hash_bytes            524.3k                 0
     hash_is_expanding                 0                 0
          malloc_fails                 0                 0
                 bytes              2.7M             74.2k
            curr_items                             603-649
           total_items              6.1k                87
     expired_unfetched                 0                 0
     evicted_unfetched                 0                 0
             evictions                 0                 0
             reclaimed                 0                 0
     crawler_reclaimed                 0                 0
     lrutail_reflocked                 0                 0
Connection closed  
Exiting.  

As you can see, there are a lot of tools available for you to easily monitor memcached. Not all are created equal so it's really up to you what's the best tool for the job. My advice is: keep it simple until you need to dive deeper.

Aurelien Navarre

Read more posts by this author.

Lyon, France