A Deeper Dive into Enterprise POWER8
2014 is the year of POWER8. In April, the scale-out servers were announced and, on October 6, the enterprise servers were announced with availability starting November 18. This announcement means that a full POWER8 line is now available for consideration. Specifically, the replacements for the POWER7 770 and 780 (E870 and E880 1-2 node) will be GA November 18 and the replacement for the POWER7 795 (E880 3-4 node) will be available by June 2015. So what was actually announced?
IBM announced two models of enterprise servers—the E870 and the E880. Both of these servers go into a 19" IBM rack and are built in a similar fashion to the modular design used with the POWER7 770 and 780 with the addition of some of the architectural features of the 795. One key difference from POWER7 is the addition of an external midplane. In the 770 and 780, the primary node (first node) and the second node housed the redundant service processors and systems clocks. For the new Enterprise servers, the reliance on the primary and secondary nodes is removed by the addition of the midplane—all nodes are now equal. Additionally, the slots in the servers are now PCIe Gen3 slots, which are capable of providing significantly improved I/O bandwidth.
The first major change is the move for all nodes to the 19" form factor. Although the 770 and 780 were 19", the 795 was not. Everything in the POWER8 line now goes into a standard 19" IBM rack. There are 32 DIMM slots, which can contain a total of 4TB (1TB per socket). Cores are added as single chip modules (SCMs) and there are up to four of those per node. All SCMs in a server must be identical. Additionally, each node or drawer has redundant power and fans. Each server also provides for capacity on demand (CoD) with the minimum requirement being the activation of eight cores and half the memory.
E870 nodes have two processor options—a 32-core 4.02ghz node or a 40-core 4.19ghz node. The E880 initial offering is a 32-core 4.35ghz node. Each server can have 1 or 2 nodes. The E880 will get more nodes and an additional core option in 2015.
The system control unit (SCU) or midplane is a critical component to the E870 and E880. Each system has one SCU and it’s a 2U drawer that contains the service processors, Hardware Management Console (HMC) ports, master system clocks, vital product data and an optional DVD. The SCU must be adjacent to the system nodes and all of the nodes must be contiguous in the rack. The SCU is powered from the system nodes so doesn’t plug into the power distribution units (PDUs).
Each node or CEC drawer is 5U, has four power cords and has 8 x PCIe LP (low-profile) PCIe Gen3 x16 slots for adapters. These slots use new low-profile blind swap cassettes and are hot pluggable. Each slot can handle Gen1, Gen2 or Gen3 low-profile adapters or can be used to connect to a PCIe Gen3 I/O expansion drawer.
Unlike the 770 and 780, the POWER8 E870 and E880 don’t include disk bays or media bays in the nodes. To provide for DVDs, either a DVD drive must be added into the SCU, which requires a PCIe USB adapter in the CEC or I/O drawer, or a DVD must be added along with an external I/O enclosure.
The new PCIe Gen3 I/O drawer is a 4U drawer that provides for 12 full-height hot-plug PCIe Gen3 slots that use the same blind-swap cassette as the POWER7 I/O drawers (5877, 5802, etc.). The I/O drawer connects to the E870 or E880 using an optical cable and uses two PCIe slots in the node to connect. In 2014, only 0 or 2 I/O drawers can be attached per node with a total of 0, 2 or 4 drawers per system. Thus a fully loaded 2-node E8* can have up to two drawers with a total of 36 PCIe slots available.
If there’s a requirement for internal disk, then it’s necessary to add a disk I/O drawer. The EXP24s (#5887) I/O drawer can be used as long as a PCIe adapter is provided for connectivity.
It should also be noted that IBM Manufacturing will always build and test the E870 and E880 in a 7014-T42 19" IBM rack. This is done to ensure thorough testing after buildout and cabling and also ensures the rules around only using horizontal PDUs in the rack are adhered to. IBM now offers a de-racking chargeable feature if you want the servers to be shipped to be installed into your own rack. There are also stringent requirements for any OEM (other equipment manufacturer) racks. IBM also requires a special lift tool with the server as the system nodes weigh 167 pounds. The IBM lift tool is the only tool certified and tested for installation.
Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.
comments powered by