Modius Data Center Blog

Computational Fluid Dynamic (CFD) software provides modeling of data center airflow and quick identification of hot spots.  A CFD system’s three-dimensional, multi-colored thermal maps are downright sexy, and, if you’ll pardon the pun, extremely cool.  When changes are made to the data center intentionally, CFD analysis can be repeated to detect the introduction of new thermal problems.  So far, so good.

But what happens when the data center changes unintentionally?  Today, CFD users require real-time thermal imaging of hot spots that could result from contingencies like equipment failure, blockage or cabinet overloading.  Furthermore, users want more than just problem visualization – they want recommendations for problem mitigation.  They want a CFD model with some muscle – in effect, a CFD on steroids.

 

What is a CFD on Steroids, and more importantly, why do we need it?

The CFD on steroids works in real-time by collecting and synthesizing all available sensor data within the data center.  It leverages wireless, wired, server-based and return/discharge air-temperature readings to determine not only the immediate problem, but also the immediate impact.  This high-fidelity monitoring system renders a thermal topology map and also sends immediate notification to operations personnel stating what temperature has been registered, where it is located, and that urgent action is needed.

Really pumping you up

The next level of growth in temperature control is temperature-based reaction.  Data Center operators are now looking not only at identification but also action automation through demand-driven cooling directly to the cabinet.  By leveraging Variable Frequency Drives (VFD) in cooling units, remote commands can adjust cooling at the point of demand.  This can reduce power costs substantially and can prevent a cabinet meltdown.  Automated actions can be taken with the existing Building Management System (BMS) without having to rip out and replace the entire system.  Integration of CFD can make the BMS smarter - processing and synthesizing a vast array of data, encoding commands in building-management language, and passing reliable information to the appropriate destination so that the secure communication infrastructure can be fully maintained.  Modius OpenData is currently being leveraged by customers to pump up their BMS, leverage the current infrastructure, prevent cooling related outages, and save money in power-related cooling.

... Or is there a cheaper alternative?

The latest advent in data center cooling is intelligent networked HVAC systems.  The HVAC systems are intelligently managed to allow remote sensors to provide feedback so that the HVAC system can tune cooling to meet the dynamic demand of the IT infrastructure.  The systems are “intelligent” in that they can change the speeds/frequency of the fan (VFD) to provide more or less air to the cooling zones and cabinets supported by the cooling system.  Further, they can auto-engage the economizer (for ambient cooling) and control water valves to provide greater efficiency to powering air-conditioning units.  They are also on a network so that they can be controlled in total rather than only independently, with one turning up while another could be throttling down. 

All very, very, cool stuff and can greatly influence one of the largest data center cost, powered cooling.  Ok, now the downside, wow.. is it really $1M to do it.  In most cases, the answer is yes. The cooling system manufacturers are hoping that you will replace your existing system and allow them to generate a services engagement for them to spend the next year turning up and tuning the system. 

So here is the question … Is there any way to make my existing HVAC smarter and NOT spend the $1M??  Glad you asked and yes there is.  Before spending that cash, there are three steps you can take in making your existing more efficient and they include:

1. Installing Variable frequency drives
2. Unifying data from temp/humidity monitoring at the cabinet
3. Compute, measure, and integrate into the BMS

 

Step 1. Install Variable Frequency Drives for controlling airflow

As discussed previously in earlier blogs, VFD’s will provide the throttle necessary to achieve energy efficiency.  Several states, including California, are providing rebated for installing VFD’s and pay for nearly 60% of the cost of the equipment (for more information on this topic, contact us at info@modius.com, and we can help put you in touch with the right people).  But remember … VFD’s are only as good as the control procedures you put in place to in order to modulate the cooling as required at the rack level.

Step 2. Unify data from a broad cross-section of temperature and humidity instrumentation points

In order to get the best possible data about what is actually happening at the rack level, there are several practical ways to extend your temperature and humidity instrumentation across your environment.  This may include not only deploying the latest generation of inexpensive  wirefree environmental sensors, as well as unifying data that is already being captured by existing instrumentation from wired, wireless, power strip-based or server-based instrumentation.  

The most cost effective way is to leverage the environmental data  the new servers are already collecting (often referred to as chassis-level instrumentation).  The new servers from the leading three vendors register both the server inlet and exhausted temperature.  Depending on the deployment architecture, this can provide you with a lot of fidelity including front/rear, min, max, average, and standard at the bottom, middle and top of the cabinet. 

In most cases, this is enough information to provide equipment demand for direct cooling.  Where you don’t have newer servers that support temperature, wireless sensors are the next best option.  There are several vendors on the market that make these products and are nice in that they are easy to set up and you can place just about anywhere.  If you have data being generated from power strips or wired sensors, incorporate those as well (the more information, the better).

Step 3. Compute, measure, and integrate into the BMS

Building management systems are traditionally very good at controlling systems such as VFDs and recognizing critical alarms.  What they are not good at is being easy to configure, integrate or extend across the network.  This is where you need to be able to provide a booster to how data is collected and synthesized. 

Modius OpenData is used to collected real-time data across the network into potentially hundreds of new devices and thousands of newly collected points.  Once the data is collected from servers, wireless sensors, pdu’s and wired sensors the data is correlated against key performance metrics then fed to the building management system so that it may adjust the VFD’s, water flow, and economizer.  Example metrics might be:

• Rack-by-rack temperature averages for inlet and outlet
• Row-by-row averages with alarm thresholds for any racks which exceed the row average by a particular margin
• Delta-T with alarms for specific thresholds

These types of computations can be based off of unified data from a variety of sources (sensors, strips, servers, etc.), all of which can be used to make your existing HVAC system smarter.  The most important point is to continually measure as you go and make a series of small or incremental optimizations based off of verified data.  The best news is that this architecture is the fraction of the cost of what new HVAC infrastructure costs and leverages your existing building management system.

Just read about a new innovative way to address the cooling requirements within the data center worthy of mention here. As no surprise, the data center energy management challenge has many parts to it, and as we all are seeing, MANY different new solutions will be required and combined over time to fully embrace the REALM OF WHAT'S POSSIBLE. Oh sure, everyone will have their favorite 'energy saver' technology. We saw this happen with Virtualization, and we saw it happen with Variable Frequency Drive controllers for data center fans.

Well, what if we take a look WITHIN the servers themselves and consider the opportunities there? Does the WHOLE server generate heat? NO. Key parts do, like the CPU, chipset, VGA chip and Memory & controllers. So why do we have to BLOW SO MUCH air across the entire motherboard, using bigger expensive to operate fans? Wouldn't it be better to SPOT COOL just where the heat is? Reminder, the goal is to just move the heat away from the chips that generate heat. We don't need to move large volumes of air just for the thrill of air handling....

I have seen two competing advances in this space. One maturing approach has been adopted in 'trials' by some of the biggest server vendors. They offer liquid based micro heat exchanger equipped versions of some of their commercial server product lines. This means these special servers have included PLUMBING/cooling pipes into the server chassis themselves, and the circulating fluid moves the heat away from the server's heat-generating chips. Take a look right next to the LAN port and power plug in the back, and you'll see an inlet/outlet fitting for liquid! Basically fluid based heat removal. Humm, harkens back to the 80's when big IBM 390s were using water cooling when everyone else went to air. (As a note, fluid cooling is making a resergence as liquid cooling becomes popular once again...).

So now I see a new approach... 'solid state' air jets. Air jets? Yes really small air movers that are essentially silent, have no moving parts, and consume tiny bits of power. Turns out at least one vendor has created really small 'jets' which have proven that you can move LOTS of air without any moving parts. Yes, they are also really silent, and can magically create large amounts of air movement in really small spaces. Using this technology, you can target just the chips that need cooling with relative 'hurricanes', and then simply use small standard fans to carry this (now easily accessible) hot air out of the box.

What results in savings does the spot jets achieve? In their published test, they reduced the standard high power fan speed from 9000 rpm to 6500 rpm, going from 108watts originally to only  62watts. Add back into this an estimated 10% energy cost for the air jets themselves, and the net savings for fans inside the box is about 30%. Remember, FANs account for nearly 47% of a data centers' entire cooling energy consumption, so reducing FAN speeds inside AND outside the boxes is critical to long term power savings.

Lastly, how do you know all your effort has paid off??? Monitor FAN speeds! I'll say it a million times, monitoring FAN speeds is very important. The slower the run, the less they consume. Monitor, Monitor, Monitor!!!