Received date: October 30, 2015 Accepted date: December 10, 2015 Published date: December 14, 2015
Citation: Oró E, Vergara A, Salom J (2015) Dynamic Energy Modelling for Data Centres: Experimental and Numerical Analysis. J Fundam Renewable Energy Appl 6:197`. doi:10.4172/2090-4541.1000197
Copyright: © 2015 Oró E, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Fundamentals of Renewable Energy and Applications
The total energy demand of data centres has experienced an important increase in the last years. Due to their unique nature, data centres demand enormous amounts of energy. Therefore, they are ideal candidates for implementing actions to reduce the energy consumption and thus improve their ecological footprint while at the same time reduce their operational costs. The aim of this work is to develop and to validate with experimental data a dynamic energy model of a real data centre located in Barcelona. The dynamic energy model is then used first to characterize the energy consumption and the energy efficiency of the infrastructure and second to see the benefits of the implementation of different energy efficiency strategies into the data centre cooling system portfolio. The results show an average Power Effective Usage (PUE) of 1,74 while some of the proposed strategies can achieve important energy reductions, up to 21%, in the cooling energy consumption. Therefore, the combination of them can achieve important reductions in the overall data centre energy consumption. The validated energy model can then be used to study the benefit of the implementation of different energy efficiency strategies in other data centres.
Data centre; Dynamic energy model; Experimental analysis; Energy efficiency; Energy consumption
Data centres are continuously growing in size, complexity and energy demand due to the increasing demand for storage, networking and computing. These unique infrastructures run 24 h a day, the 365 days of the year and they are up to 100 times more energy intensive than conventional office buildings. Nowadays, 40% of the total energy consumption is attributed to cooling and therefore the development of effective and efficient strategies to reduce cooling demand is required. Recently the data centre industry [1,2] has taken consciousness of the need of the implementation of energy efficiency strategies and the use of renewable energy sources in data centre not only to show their environmental commitment, but also to reduce the operational cost. In that sense, Oró et al.  presented a literature review on the implementation of energy efficiency strategies and the integration of renewables into data centres portfolio.
In conventional data centre the cooling infrastructure is divided into cooling production mainly air cooled chillers which produce chilled water and cooling distribution which distributes chilled water to Computer Room Air Handling (CRAH) units. Therefore the evaluation of the cooling system performance in data centres should be focused on cooling production and air management which has the objective to keep Information Technology (IT) equipment intake conditions within the recommended ranges with the minimum energy consumption. Due to the recent increase of data centre industry many researchers have been focusing on the implementation of energy efficiency strategies into the cooling system. In that sense, a number of case studies reveal that the fan energy savings in 70-90% range and chiller energy savings in 15-25% range are achievable with effective air management . Lu et al.  evaluated the air management and energy performance of the cooling system of a data centre in Finland. They investigated for that specific facility the possibilities of energy savings (mainly the reduction of the fan speed) and heat reuses for space heating and hot water. Similarly, Lajevardi et al.  analysed a small data centre located in the Gresham City Hall (United States) over a period of six weeks proposing different energy efficiency and thermal management issues. Choo et al.  evaluated experimentally and numerically the energy efficiency performance of a medium size data centre at the campus of the University of Maryland. They also assessed energy conservation measures such as eliminating unnecessary CRAH units, increasing the return set point temperature, using of cold aisle containment and implementing free cooling. To evaluate data centre air management performance, Computational Fluid Dynamics (CFD) modelling of IT equipment and indoor temperature distribution is required regards the air distribution system and IT server’s operation requirements. Even though CFD analysis provides valuable inputs for data centre air management system, they are useless for real time data centre cooling management infrastructures and for dynamic energy systems evaluation. Therefore, other modelling techniques are needed to overcome these problems, reaching a compromise between time and cost required by the simulation tool and reliable information. The use of dynamic energy model using Transient System Simulation program (TRNSYS)  can overcome this problem. Kim et al.  studied the feasibility of the integration of a hot water cooling system with a desiccant-assisted evaporative cooling system for data centre air conditioning using TRNSYS. Recently, Depoorter et al.  developed a dynamic energy model using TRNSYS to assess the potential of direct air free cooling in the data centre portfolio around Europe.
The aim of this paper is to develop a dynamic energy model for data centre facilities in order to evaluate the benefits of the implementation of different energy efficiency strategies in the cooling system. The dynamic model is validated with experimental data from a real data centre located in Barcelona (Spain) and is also used to characterize the energy consumption of the data centre.
The American Society of Heating, Refrigerating, and Air- Conditioning Engineers (ASHRAE) thermal guidelines  define recommended and allowable temperature and humidity ranges for four environmental classes, two of which are applicable to data centres. The recommended envelope (Table 1) defines the limits under which IT equipment would most reliably operate while still achieving reasonably energy efficient data centre operation. However, it is acceptable to operate outside the recommended envelope for short periods of time without risk of affecting the overall IT equipment reliability.
|Equipment Environment Specifications|
|Class||Product Operation||Product Power Off|
|Dry-Bulb Temp. range||Humidity range||Dry-Bulb Temp. range||Humidity range|
|A1-A4||18 to 27 ºC||5.5°C DP to 60% RH and 15°C DP|
|A1||15-32||20% to 80% RH||5-45||8% to 80% RH|
|A2||10-35||20% to 80% RH –12°C DP and||5-45||8% to 80% RH|
|A3||5-40||8% RH to 85% RH||5-45||8% to 80% RH|
|A4||5-45||8% RH to 90% RH||5-45||8% to 80% RH|
Table 1: ASHRAE environmental classes for data centres .
Data centre characteristics
The data centre is an operative facility with an IT capacity of 115 kW which is used to provide computing and information services for the Polytechnic University of Catalonia (Spain). The data centre was constructed in 2006 and started operations in 2007. It has an IT room area of 285 m2 and the whitespace is located on a second basement surrounded by other refrigerated areas. Figure 1 shows the scheme of the data centre and the equipment distribution. The facility is composed by 70 racks of data and 12 racks of communication equipment. The theoretical maximal power consumption is 4 kW per rack but at the present moment the real average power consumption is between 1.5 and 2 kW per rack. Some racks are distributed in cold and hot aisle containment and some other racks are placed with no containment in the whitespace. Hot and cold aisle containment is an effective strategy to ensure a properly air management inside an aisle of racks. However, if proper installation is not done air inefficiencies can also be present. The racks which are not enclosed are directly mounted over the perforated tiles of the raised floor or they have a frontal air tide. This situation reduces the airflow velocity and also forces some of the air to directly bypass the rack without exchange heat with it.The infrastructure is connected to the main grid but there is also an emergency generator in case of energy supply failure. Figure 2 shows schematically the main electrical and mechanical components of the data centre. The installation has N+1 redundancy in the chillers, and 2N in the power distribution units (PDU) and the power supply units (PSU). Notice that all the electrical components are located inside the whitespace and thus contribute to the IT room thermal load.
The power consumed by the IT equipment and the electrical losses of the equipment is converted into heat  and therefore reliable thermal management is essential to provide an adequate environment for IT devices . The refrigeration system is composed of five CRAH units which use chilled water from three water-air chillers. A raised floor is used to distribute the chilled air to the bottom of the racks and the exhaust warm air leaves the room direct to the CRAHs as it is shown in Figure 1. The 3-way valve of the CRAH controls the cooling exchange between the chilled water and the exhaust air. This valve actuates (opening or closing) in function of the return air temperature from the whitespace. Each unit keeps a constant air flow rate to the IT room of 8500 m3/h while the total chilled water flow rate is 39.9 m3/h. There is also a 1 m3 buffer tank after the chillers to prevent water temperature fluctuations. The main characteristics of the cooling equipment are listed in Table 2.
|Vapour Compression Chiller|
|Cooling capacity [kW]||77.7|
|Water flow rate [m3/h]||13.3|
|Model||Uniflair TDCR 1200A|
|Cooling capacity [kW]||37|
|Energy consumption [kW]||2.5|
|Air flow rate [m3/h]||8500|
|Energy consumption [kW]||3|
|Water flow rate [m3/h]||42|
Table 2: Specifications of main equipment
Dynamic energy model
To evaluate the effect of airflow efficiency improvement in the data centre, a dynamic energy model using TRNSYS has been developed. The model is based on a component-by-component approach. Information from the equipment manufacturers, data centre operators and data collected directly in the facility was used to build the energy model. This paper is focused on analysing the cooling system and therefore the whitespace characterization is assumed as a black box. This black box calculates the return air temperature in function of the current IT load, miscellaneous loads, and the air inlet conditions following Equation 1.
Q̇ cooling=ṁair·Cp,air·(Tair,return−Tair,supply) E q .1
Where mair is the air mass flow rate, cp, air is the air specific heat capacity, Tair, supply is the air inlet temperature to the whitespace and Tsupply, return is the air return temperature. Notice that the total cooling demand (Q̇cooling) is defined as the IT load (Q̇IT) plus the additional loads (Q̇add):
Q̇add=Q̇CRAC,e+Q̇el ectrical+Q̇miscel laneous−Q̇loss Eq.3
Where Q̇CRAC,e is the electrical loss inside the white space of the CRAH units, Q̇electrical is the electrical loss inside the whitespace (it are) by means of PDU, wiring, UPS, filter, electric board, Q̇miscellaneous is the miscellaneous losses inside the whitespace such as the lighting, ventilation, working people, etc. and Q̇loss is the heat loss through the wall to the environment.
The whitespace is cooled by CRAHs which were modelled with a cooling coil using a bypass approach (type 508). In order to control the chilled water flow rate to the CRAH and the humidification ration a proportional controller (type 1669) acts on a flow diverter (type 11) and an evaporative cooling device (type 507). The water is cooled by three chillers which are simulated using the well-known type 655. The energy model also consists of other main system components including single speed fan (type 112), pipes (type 31), etc. which are available in the TRNSYS library. Those components were connected according to the system configuration already described and shown in Figure 3. The typical meteorological year (TMY2) data was used to obtain the weather conditions for Barcelona. The TMY2 data sets are the typical values of meteorological elements for a one-year period from Meteonorm .
For operational data, the water return temperature (Tw.return) was used as input of the model. The model was validated by comparing the simulation results with the operational data during 12 hours. In the validation, the supply chilled water (Tw.supply), the return water temperature (Tw.return), the cooling consumption (Pcooling) and the total consumption (Ptotal) were selected to validate the model. The actual chiller set points are different between them. Chiller #1 starts when the return water temperature is above 9.5ºC, when it is higher than 12.5°C then chiller #2 also starts and chiller #3 does not start till the return water is above 13.5ºC. Moreover, the outlet water set point is set at 6ºC. For the CRAH units, the return air temperature set point is set at 25.5ºC. Figure 4 shows the modelled and the real temperature and power values over a period of 12 hours. According to these results, the predicted values by the model make good agreement with the actual operational data and therefore the consistency of the dynamic model is demonstrated.
Data centre energy characterization
Once the dynamic energy model proposed has been validated with experimental data from the data centre, it is used to evaluate the energy characterization of the installation over an entire year. Figure 5 shows the disaggregate energy consumption of the data centre after one year of operation . As expected the main consumers are the IT equipment and the chillers, being 82% of the total energy consumption. The cooling system which takes into account the consumption of the chillers, water pumps, CRAHs and ventilation represents up to 34% of the total energy consumption. Therefore, a potential of energy saving due to the implementation of better operational strategies is possible. The Power Usage effectiveness (PUE) metric is the most common metric used in the data centre industry. It measures the energy efficiency of the installation by dividing the total energy consumed by the facility with the IT energy consumed. In the present data centre the average PUE value is 1,74 which is in line with the self-reported PUE’s values from the Uptime Institute 2013 data centre industry survey .
Cooling management strategies
The dynamic energy model allows the estimation of the data centre energy consumption under different energy management strategies. Different scenarios have been analysed:
- IT room air inlet temperature increased.
- Modification in the chillers working sequence.
- Conditioning of the chiller air.
- Increasing water chilled temperature.
IT room air inlet temperature rise: Increasing the IT room supply temperature has been suggested as the easiest and most direct way to save energy in data centres. However, as Patterson  noted, just implementing a higher inlet air temperature while still relying solely on mechanical cooling, may not improve the efficiency of the cooling system. That conclusion is confirmed by the results of the simulation, where an increase in the air inlet temperature has negligible results in the cooling energy savings of the infrastructure. Increasing the inlet air temperature from 18ºC to 27ºC and maintaining constant the chilled water temperature, a cooling energy reduction of only 2% was observed. Since the total heat that has to be removed from the IT room is the same and the temperature difference between air inlet and outlet is constant since the air volume flow is also constant, thus the fans does not experience less consumption. On the other hand, the chiller water pump operation is highly affected for this measure reducing its energy consumption drastically but it does not affect at the overall picture since its consumption is not significant.
Modification in the chillers working sequence: It is well known that most chillers operate more efficiently in partial load using variable speed compressors and pumps, resulting in a reduced water and refrigerant flow. Therefore, when using chillers at partial load, the Energy Efficiency Ratio (EER) will increase. However, an optimum exists at a certain partial load, depending on the characteristics of the chiller and below this the energy efficiency starts to decrease again. Moreover, chiller sequencing control is an essential function for multiple-chiller plants that switches on and off chillers in terms of data centre instantaneous cooling load. It significantly affects both inlet air temperature control and data centre energy consumption [15,16]. In the present installation a chiller sequencing control is implemented which switch on or off chillers according to a direct indication of the system, the return water temperature. Therefore, firstly chiller #1 is activated when cooling is needed; if more cooling is needed then chiller #2 is activated and just in the case that a high level of cooling is needed chiller #3 is switched on. Table 3 shows the initial and the modified water set point temperatures for each of the chillers. Notice that in the reference configuration when the return water temperature was between 9.5 and 12.5ºC only chiller #1 was activated (Figure 6). In the new configuration the activation temperature for the first compressor of chiller #2 and #3 has been reduced while the set point temperature for the second compressor has been enhanced. Notice that in the reference situation a small increase in the inlet water temperature activated the full load of each chiller. This modification in the current configuration allows the chillers working at partial load while maintaining the supply water temperature.
|Chiller #1||Chiller #2||Chiller #3|
|Twater.IN (Return water temperature)||Reference||9.5||12.5||15|
|Twater.IN (Return water temperature)||New version||9.5||11||12|
Table 3: Return water temperatures to activate the chillers.
This new sequence configuration was implemented in the control strategy of the data centre model and was evaluated over a period of one year. Figure 7 shows the energy consumption of each of the chillers of the installation and the energy reduction. The total energy reduction was around 12%, representing more than 4% in the overall data centre energy consumption. Therefore an annual energy savings of 50.4MWh are expected. Notice that chiller #3 increased its energy consumption since in the reference case it is normally switched off.
Conditioning of the chiller air: It is well known that lower dry bulb temperature of outdoor air enhance the EER air-cooled chiller systems . This outdoor air cooling can be achieved using adiabatic cooling process which reduces heat through a change in air humidity rate; this phenomenon happens due to in the process of absorbing water the air uses its enthalpy reducing then its temperature. Figure 8 shows the ambient air temperature during 7 days of summer at Barcelona and the air temperature once it is conditioned by adiabatic cooling. Notice that the conditioned air is plotted for total saturation at 100% relative humidity and at 90% relative humidity. In peak hours the temperature decrease can be up to 5ºC. The model has adopted total air saturation during the entire year. The results show that for Barcelona an energy reduction of 4% in the cooling system can be achieved. Therefore an annual energy savings of 17.62 MWh are expected. However, the implementation of this process besides the adiabatic cooling system also needs to account for water consumption. With no implementation of optimization process in the system; so the system always adds water to the air in order to reach total saturation even though the temperature reduction potential is really low i.e. during nights the temperature reduction can be less than 1ºC, the yearly water consumption is 330 m3. This water consumption obviously reduces the overall economical savings of the implementation of this strategy. Moreover, an additional water pump must be installed but its energy consumption will not affect the overall energy consumption of the system.
However, the overall potential of this strategy increases when it is applied to locations with higher dry bulb temperature and low humidity thanks to its ability to reduce of reducing temperature by absorbing water. This happens for instance in continental climate zones like Madrid. The model has been used to see the results of this implementation in Madrid showing a potential energy reduction of the cooling system of 6%.
Increasing chilled water temperature: In this scenario the authors wanted to study the effect on the energy consumption of the infrastructure when the chilled water temperature is raised from 6 (reference case) to 11ºC while the inlet air temperature is kept constant at its maximum. In this scenario the water temperature increases is selected following ASHRAE and chiller manufacturer recommendations and while the maximum air inlet temperature is set to 27°C based on the ASHRAE recommendations. When increasing the supply air temperature, the IT room environment will obviously be changed. This strategy needs to be studied in detail in order to avoid hot spots due to air management inefficiency. Figure 9 shows the energy consumption of the chillers under different scenarios (supply air temperature at 27ºC while enhancing chilled water temperature). With proper system management conditions annual cooling energy savings up to 21% can be achieved. Therefore an annual energy savings of 88.20 MWh are expected. At a certain point, even though the water chilled temperature is increased the energy consumption is almost constant and it is expected to decrease. This phenomenon occurs since the heat transfer between the water and the air decreases (lower temperature difference between them) and thus the water flow rate has to be increased in order to supply the same amount of cold to the IT room.
However, the dynamic nature of IT equipment cooling fans and semiconductor leakage current in the CPU due to higher temperatures may diminish or even negate the cooling system gains due to an enhancement of the air inlet temperatures. Moss and Bean  highlighted the importance of undertaking a holistic analysis of the data centre energy consumption taking into account not only cooling energy reduction by the implementation of energy efficiency strategies such as free cooling or higher inlet air temperatures but IT equipment performance due to higher inlet air temperatures. Figure 10 shows the increase of energy consumption (percentage in function of the energy consumption at 18ºC) of 3 different servers in function of the air inlet temperature. While the air inlet temperature does not play a significant role in the energy consumption of some architecture (server X) in other servers can increase the overall energy consumption up to 25%. Therefore, further research should be done in order to optimize the management system minimizing the overall data centre energy consumption and taking into account particular IT equipment consumption.
In the last years, the increasing energy demand of the data centre industry has put the sector under pressure to limit its environmental impact and reduce energy costs. Industry and researchers have made a lot of effort to overcome this situation and many energy efficiency measures have already been investigated and implemented. Efficient cooling production and air management in data centres plays an important role reducing the overall cooling energy consumption. This paper aimed to develop a dynamic energy model of a real 115 kW IT data centre located in Barcelona (Spain). The proposed model is then validated with experimental data collected during 2 weeks under normal operation. First, data centre energy characterization was studied being the cooling system energy consumption up to 34% of the total energy consumption of the facility. Moreover the PUE value was also calculated being 1.74. Second, the already validated dynamic model was used to study the benefit of the implementation of different energy efficiency strategies. The strategies proposed and studied were an increase of the inlet air temperature, a modification of the chiller working sequence, an analysis of the improved energy performance of the air-cooled chiller with cooled air and an increase of the water chilled temperature. The results show that the only increase of the IT room supply temperature has no significant decrease in the energy savings. This strategy should be implemented in parallel with others such as the use of air free cooling but not when operating with solely mechanical cooling. The actual chiller working sequence (one chiller after the other) can be improved maintaining the IT room temperatures. This strategy has the potential of reducing the cooling energy consumption up to 12%. Adiabatic cooling can help improving the energy performance of air-cooled chiller system. The implementation of this strategy in the studied infrastructure can reduce the cooling consumption up to 4%. However, this strategy aims for an initial investment and operational costs (water consumption and maintenance). Notice that the location of the data centre in Barcelona (high ambient relative humidity) reduces the overall potential of this strategy which will be much more efficient in continental climate zones such as Madrid. Moreover, for the correct implementation of this strategy, a control algorithm to optimize the use of cooling water reducing the annual water consumption while enhancing the benefit of the air saturation. Finally the chilled water temperature increase while the inlet air temperature was kept as its maximum (27ºC from ASHRAE recommendations) was also studied. The results show that with proper system management control annual cooling energy savings up to 21% can be achieved. It is worth to highlight that at certain point the cooling energy consumption keeps almost constant and it is expected to increase due to an increase of the water pumps. This strategy should also be studied in detail since due to the dynamic nature of IT equipment (mainly for variable velocity internal fans and semiconductor leakage current in the CPU) can diminish or even negate the cooling system gains due to an enhancement of the air inlet temperatures. Therefore, further research should be done in order to optimize the management system minimizing the overall data centre energy consumption and taking into account particular IT equipment energy consumption.
The research leading to these results has received funding from the European Union’s Seventh Framework Programme FP7/2007-2013 under Grant Agreement nº 608679 - RenewIT.