Paging, Memory and I/O Delays
How to tune AIX versions 5.3, 6.1 and 7 for increased performance
Editor’s Note: This is the first of a two-part series on AIX tuning. Part one covers paging, memory and I/O delays and part two will focus on disk I/O and the network.
It’s been a couple of years since I wrote an article on tuning AIX, so with the advent of AIX 7, this is a good time to review some of the basic tuning that may need to be done on your AIX systems. There have been many technology levels (TLs) released and some of the recommendations may have changed. In this article, I’ll share AIX tuning information looking at tunables in AIX 5.3, 6.1 and 7.
I’ll focus on the key areas of I/O, memory and network. By default AIX 6.1 and 7 do a fairly good job of tuning for memory and there are only a few small tweaks needed. AIX 5.3, however, needs a lot more tuning in this area. Figure 1 shows the different tunables with their default settings. The fourth column is the recommended value for those settings at the most recent TLs for all three versions.
One key point is: A fresh AIX 6 or 7 install will automatically install the new defaults for memory. If the system is migrated from AIX 5.3, then any tunables set in AIX 5.3 will be migrated across. Prior to performing a migration, it’s suggested you make a note of all of the tunables that have been changed (take a copy of /etc/tunables/nextboot) and then reset the tunables to the defaults. After migration, check nextboot and make sure there’s nothing in it. Now, go ahead and set the tunables that need to be changed for AIX 6 or 7.
Best practice suggests there be multiple paging spaces, all be equal in size and on different, non-busy hard disk drives (hdisks). All page spaces should either be mirrored or on a raided (1 or 5) storage area network (SAN). Unless your database requires it, there’s normally no need to have twice the page space as you have memory. I have run large Oracle databases running AIX with 250 GB of memory and three 24 GB page spaces. The key is to use technologies such as concurrent I/O (CIO) to avoid paging, as they have page spaces available in case you need them.
By default, AIX creates one page space (hd6) in rootvg that’s too small. If rootvg is mirrored, then the page space will also be mirrored. I normally add a couple of custom-sized logical unit numbers (LUNs) from the SAN for the additional page spaces. Don’t add extra page spaces to the same internal disks (or SAN LUNs) that your current rootvg page spaces are on. Having page spaces on the same hdisks can slow down paging.
When a virtual I/O server (VIOS) is first built, it’s automatically configured with two page spaces, both on hdisk0. hd6 will be 512 MB and paging00 will be 1,024 MB. I always swap off paging00 and remove it and then increase hd6 to 4,096 MB. As mentioned, it’s bad practice to have two page spaces share the same hdisk.
Page Steal Method
The page_steal_method is set to 0 in AIX 5.3 default settings. This affects how the least recently used daemon (LRUDs) scan for pages to free. With lru_file_repage=0, a strong recommendation is made to the LRUD to favor executables and to always try to steal from filesystem (persistent) pages. It’s far less expensive to steal from persistent pages than to steal from working storage pages, as the latter causes paging to/from page space. With 100 GB and five memory pools, memory is broken into five pools of roughly 20 GB a piece and each LRUD handles around 20 GB (this is a very simplistic view). Based on the numclient value in Figure 2, you can assume about 45 percent of our memory is being used for filesystems, which would be around 45 GB; the other 55 GB is working storage.
With page_steal_method=0, the LRUDs have to scan all of the memory pages they control when looking for free pages, even though it will most likely only free the persistent pages. Setting page_steal_method=1, the LRUDs move to a list-based page-management scheme. This means the LRUDs split memory into a list of persistent pages and a list of working storage pages. When the LRUDs are searching for pages to free from the filesystem cache, they will search only the persistent page lists. In the example in Figure 2, this should reduce the scan-to-free rate by more than a half, which will reduce overhead. The scan rate and free rate both show in the output from a "vmstat -I 2 2".