A follow-up on an earlier post about building machine rooms.
One of the additional difficulties in designing high-capacity server space is the problem of heat transfer. I’m not a mechanical engineer (and really, most people who read this blog aren’t interested in learning a couple college semesters worth of thermodynamics and practical HVAC engineering concepts, not to mention industry standards). In a nutshell, if you’re trying to cool down something that’s really hot, you have limited options.
There are a few products out there that attempt to help you solve this problem. Emerson/Liebert produces a contained server enclosure called the 😄, in two configurations (25kW and 17kW). Rittal Corporation produces a modular refrigerator/server cabinet system called the LCP+ that can be configured in a variety of ways. Of course APC has its own solution called InfraStruXure that also handles cooling in an integrated fashion.
None of these solutions is low cost, on the face of it. On the other hand, buying a pair of XDs (or a row of LCP+ units with the Rittal cabinet enclosures) is much more practical for the purposes of chilling a few racks (or even a smallish datacenter) than trying to retrofit an existing building. Using one of these solutions makes it pretty easy on your facilities manager -> bolt the sucker to the structure of the building, hook up a 3/4″ chilled water pipe and return, and run a big power circuit into the room. This (as expensive as it may be) is probably still going to be significantly cheaper than trying to build out a small datacenter in a converted closet in the back corner of your leased office space. And, you can take it with you when your lease is up, which you probably won’t bother to do if you’ve stuck a 35 ton refrigeration unit in a retrofitted room.
We’re building a new building at my place of employment, and I’ve been working on fitting 25 racks with a design parameter of 25kW per rack into a space that’s about 1,000 square feet. We’ve looked pretty exhaustively at both the Rittal and the Emerson packages, and all other things being equal, here’s my considered opinion.
If you’re building a data center from scratch, both of these solutions are pretty damn good, and they both have minor advantages and disadvantages in their design. In my opinion (again, no engineer here), the Emerson product is slightly better engineered not to fail as a stand alone unit, but the Rittal product is designed to fail more modularly and gracefully. What this means for your organization is dependent upon your reliability requirements. Both of them enable you to fit high-power compute clusters in a very small space. The Emerson product is less flexible than the Rittal product since it is self contained, but the obvious flip side to that is that the Emerson product is much easier to add-on in a smaller increment; if you’re planning on adding compute capacity on a 1-rack every 6 months basis, it’s easier to buy one Emerson 25kW unit every six months than to buy 1 row of Rittal cabinets with LCP+ units every year and a half. If four years pass, and you need to upgrade your coolant capacity because you just swapped out 42 1U dual core 2.6GHz machines with 42 1U ten-core 4.8 GHz machines, you can buy one or two more LCP+ units and tack them onto your enclosure. You can’t really do that with the Emerson solution.
They both work in particular scenarios, depending upon your maximum power load available, your chilled water supply, and the number of machines you want to power up (and how often you want to replace them, or add more). I have a slight preference for the Rittal units because the heat exchanger is on the side instead of the bottom. With the Emerson 😄 solution, your 42U rack is elevated about 14″, which means racking stuff up at the top of the rack requires a lift or a platform; or an employer willing to violate OSHA regulations and an employee strong enough to lift a 100 lb server over his or her head. On the other hand, if square footage is your constraint, you can fit more XDs in the same space that you’d have to dedicate to the LCP solution. Both sets of sales guys were excellent, friendly, and fairly responsive when it came to getting me information.
If you’re blog-searching looking for more information about these sorts of solutions, drop me a comment.