The climatological mean (1975–2005) surface air temperature for June– September (JJAS) from Indian Meteorological Department (IMD) observations and MMM are shown in Fig. 1a, b, respectively. The IMD climatology for the historical period shows that the highest temperature occurs over the northwestern region of the country, and locates over the axis of the monsoon trough inclined with a northwest–southeast orientation (Fig. 1a). It can be seen from Fig. 1b that the MMM captures the observed pattern to a great extent, however, there are biases in magnitude. As can be seen from Fig. 1c, the temperatures are severely underestimated over the northern and northeastern parts of India with biases of around 10–15 °C, which was also previously noted by Basha et al.14 On the other hand, the temperatures over the axis of monsoon trough are overestimated by 2–5 °C, with higher biases towards the northwestern flank of the trough. Over southern and southeastern India, the biases are relatively low with the magnitude of around 1–5 °C. As mentioned above, in order to compare the models with observations, all the models were regridded to the observational grid. To verify if the severe underestimation over the northern parts of India could be an artifact of interpolation, in Fig. 1d, e, we show the example of one of the models (MIROC-ESM), with temperatures at its native coarse resolution of 2.8° × 2.8° and its re-gridded version at 0.25° × 0.25°. As noted from Fig. 1d, e, this underestimation is not an artifact of regridding, but is a real bias that can be seen in the native resolution as well.

Fig. 1 JJAS climatological mean (1975–2005) temperatures from a IMD, b MMM, c MMM–IMD, d MIROC-ESM model at 2.8° resolution, e MIROC-ESM model regridded to 0.25° resolution, f MIROC-ESM at 2.8°, g MIROC5 at 1.4°, h MIROC4h at 0.56°, i CCSM4 (best model), and j GISS-E2H (worst model) Full size image

One of the obvious reasons one may think to explain the cold bias in the MMM is the inaccurate representation of orography over the Himalayan and Tibet region in the models due to their coarser resolution. In order to check this, we have analyzed a modeling system (MIROC) for which data is available from the CMIP5 archive at multiple resolutions (see Fig. 1f–h), however, there are differences in their physics packages as well, so the difference in simulations can only be partially attributed to the difference in spatial resolution. Analyzing the corresponding figure panels, it can be seen that counter-intuitively enough, the biases over northern India systematically increases with the increase in spatial resolution. Hence, one may conclude that the coarser resolution of some of the CMIP5 models included in the MMM may not be playing a significant role in causing the cold bias seen in surface air temperature over the northern parts of India.

Since the MMM used in this case is an unweighted (each model gets the same weight) mean of all models, one may wonder if assigning the same weights to the better and the worse models could lead to large biases. In order to verify this we compute the root mean square error (RMSE) and the pattern correlation coefficient (PCC) of the individual models (see Table 1), and then choose the best and the worst performing models based on a simple ratio of PCC by RMSE (the best model would have the highest value and the worst model would have the lowest value). It is noted that the CCSM4 is the best model, while the GISS-E2H turns out to be the worst model (see Fig. 1i, j). Although the cold bias over northern India is somewhat larger in the worst model, it is not too different than the best model. Thus, one may conclude that the cold bias in surface air temperature seen over northern India in the MMM is a systematic bias in all models, and increasing the model resolution does not help to alleviate this problem. The large difference between MMM and observations over the Himalayan region could either be due to the inaccuracies in the observational dataset over this region or there is something missing in the models physics/dynamics that is causing this large bias, which is common across all models and resolutions, and it needs targeted numerical experiments to identify the reasons behind this.

Table 1 RMSE, PCC, and ratio of PCC by RMSE for JJAS surface air temperature and precipitation Full size table

It is to be noted that the value of the ratio of PCC by RMSE is ~0.15 for the MMM (Table 1, with north box). For 16 out of 28 models, this ratio is less than MMM, whereas there are 12 models for which this ratio is equal to or even higher than MMM. We have also examined the biases in these 12 individual models and it is found that some of them, for example, CCSM4, have biases similar to MMM (figure not shown). This, therefore, suggests that though MMM is better than several individual models, however, there are still few individual models, which perform as good as MMM. It is previously shown that the biases over the northern India are common across individual models and therefore we also compare the performance of the MMM after removing the north box. It is noted that though the values of this ratio for MMM and most individual models increase after removing the north box, however, for this selected domain too, the performance of best individual model is equivalent to the MMM. Therefore, the general notion that the MMM performs better than individual models is not valid for climatological seasonal mean temperature over the Indian land region.

In Fig. 2, we analyze some other important aspects of surface air temperature characteristics. Figure 2a shows the annual cycle of the area-weighted spatial mean of pentad surface air temperature over the Indian land. The annual cycle is shown for IMD observations and the MMM. Since there exist extremely large cold biases in the MMM over northern India, the annual cycles from both observations and MMM are plotted for the whole Indian land in one case (blue lines), and by excluding the northern box (Indian land between 33 and 38.5°N) in the other (red lines). From Fig. 2a, one may conclude that (i) the MMM captures the annual cycle of temperature over Indian land, (ii) there is an overall negative bias in surface air temperature in the MMM throughout the year, (iii) the negative bias is small during JJAS, but large in other months, and (iv) excluding the northern box reduces the total bias in all months, but the MMM temperatures are still lower than that observed throughout the year. Our results, thus, show that the cold bias in the annual mean throughout the year is not entirely because of the lower than observed temperatures simulated over north India.

Fig. 2 a Annual cycle of all-India weighted area average pentad surface air temperature from IMD and MMM with (blue) and without (red) the north box (Indian land between 33–38.5°N), b PDFs (%) of daily surface air temperature from IMD and mean of the distributions of individual models, without the north box. PDFs for MIROC-ESM and MIROC-4h are also shown. Spatial distribution of JJAS surface air temperature trends (°C per decade) from c IMD and d MMM. Grid points with a trend value significant at the 90% level have been stippled Full size image

Since the daily distribution of surface air temperature is important for various societal applications of climate information, we next analyze this aspect and show it in Fig. 2b. As noted above, there are large biases over north India, so in order to be fair to the models, we exclude this box of large negative biases while computing the frequency distribution. Since taking a MMM before computing the distribution will lead to the elimination of extremes due to the out-of-phase occurrence of daily extremes in individual models, we first compute the distribution for each of the models and then take the mean of all distributions. It is to be noted that the computation of the probability distribution function (PDF) for individual models does not involve any spatial or temporal averaging. Figure 2b thus shows that, even without the northern box, the occurrence probability of temperatures in range 0 to +15 °C is overestimated in the MMM. In the range of 15–35 °C, there is a good agreement between the MMM and observations in the distribution of daily surface air temperatures over the Indian land. However, in the temperature range above 35 °C, the MMM overestimates the occurrence probability as compared to the observations. To check the improvement in the occurrence of temperature extremes with increasing model resolution, we also show the PDF for MIROC-ESM (one of the coarsest resolution model) and MIROC4h (one of the finest resolution model) at their native resolutions in Fig. 2b. The PDFs for both the models (blue and green lines) are quite similar with large over-estimation on both sides of PDF. No improvement is noted from MIROC-ESM to MIROC4h thereby suggesting that the simulation of temperature extremes in these models may not improve just by increasing the spatial resolution.

Since trends provide some of the most valuable information in the context of climate change, we analyze how well the models perform in simulating the observed trends. The JJAS trends from observations and the MMM are shown in Fig. 2c, d, respectively. To calculate the MMM trend at each grid point, first the MMM time series is calculated and then the trends are estimated using Theil–Sen method (which is more robust than linear trends) and the significance is determined using Mann–Kendall test. Only the grid points where the trend is significant at the 90% level have been stippled. As can be seen from observations, there is a net warming trend of 0.05–0.30 °C per decade over almost whole Indian region, except east coast of India where there is a cooling of around 0.05–0.15 °C per decade. Note that the temperature trends are not significant over the entire Indian region in observations. However, the MMM show significant warming over the entire Indian land and captures broad features of the trend distribution with warming over the northwest and peninsular India. Thus, one may conclude that the MMM captures the magnitude of the trend in surface air temperature for some locations but the trend is not significant in observations.

Figure 3a, b shows the climatological mean (1975–2005) precipitation for JJAS from observations and MMM, respectively. The IMD climatology for the historical period shows that the highest rainfall occurs over the Western Ghats and northeast India (Fig. 3a). From Fig. 3b, it can be seen that the MMM captures the observed pattern to some extent (such as the maxima over the Western Ghats and northeast India), however, it fails to capture the local and regional scale features and there are large biases in the magnitude. Figure 3c shows severe dry bias over most parts of Indian land, especially over the Western Ghats, whereas there is a wet bias over the leeward side of Western Ghats. A similar exercise as that done for temperature was repeated to investigate if some of the large biases, such as those over peninsular India, are due to the interpolation while re-gridding. It is confirmed from Fig. 3d, e that the biases are not an artifact of regridding as similar biases can be seen in the native resolution as well.

Fig. 3 JJAS climatological mean (1975–2005) rainfall (mm day−1) from a IMD, b MMM, c MMM-IMD, d MIROC-ESM at 2.8° resolution, e MIROC-ESM regridded to 0.25° resolution, f MIROC-ESM at 2.8°, g MIROC5 at 1.4°, and h MIROC4h at 0.56° resolution Full size image

To investigate the resolution dependence of the model bias, a similar exercise as that done for temperature was repeated (Fig. 3f–h). It is worth noting that increasing the spatial resolution of a model leads to a more accurate representation of the bottom boundary (such as the orography, coastlines, land-use, land-cover, etc.), however, if the physical parameterizations are not satisfactory, the final simulation may turn out to be worse even though the resolution is high. The major improvement in such a case would be seen over the locations where the bottom boundary plays a critical role. The MIROC5 model has a spatial resolution of 1.4° × 1.4°, whereas MIROC4h model has a spatial resolution of 0.56° × 0.56°. Some aspects such as rainfall over the Western Ghats are better simulated in MIROC4h due to its higher resolution. As shown above, the increase in model resolution does not show any major improvement in the temperature biases, however, for rainfall, some of the biases get significantly alleviated due to the increase in spatial resolution. We have also checked the PCC by RMSE ratio for each individual model and the MMM (Table 1). This ratio is ~0.17 for the MMM and for the individual model it varies from 0.04 to 0.20. There are two models, viz. EC-Earth and MIROC5, for which this ratio is higher than MMM. For the EC-Earth model, the PCC is higher and RMSE is lower than MMM and the magnitude of regional biases is similar to the MMM, thereby suggesting the good performance of this model over the MMM.

We next look at other aspects of rainfall that are equally important for many applications of the climate information provided by models. Similar to the analysis done for temperature, in the following we analyze the annual cycle, occurrence probability, and trends in rainfall over the Indian land. It can be seen from Fig. 4a that the MMM captures the annual precipitation cycle over the Indian land quite well, but there is a dry bias during June–August, and wet bias during November–January. Since the distribution of rainfall on sub-monthly timescales is equally important for many climate applications (e.g., agriculture), we next look at the PDFs of daily precipitation from observations and the MMM. Given the large range in rainfall values (as compared to temperature values), we use a finer bin width (0.2 mm day−1) for the 0–20 mm day−1 range, and a coarser bin width (5 mm day−1) for the 20–500 mm day−1 range, and show the PDFs in two separate panels. The PDFs for the precipitation are computed using the same methodology as the PDFs for the temperature to avoid issues due to the out-of-phase behavior of the individual models in simulating rainfall over a given grid point. It can be seen from Fig. 4b, c that the MMM overestimates the occurrence probability in the low to moderate precipitation range (0–20 mm day−1) and fail to capture the heavy precipitation. In the MMM, the maximum daily precipitation values are ~60 mm day−1, whereas in observations it goes beyond 500 mm day−1. The underestimation of extreme events in CMIP5 models may be attributed to the coarse resolution of the models, and to the issue of high-frequency low-intensity drizzle that prevents the moisture to build up in the atmosphere to a level where it can cause extreme rain events. This could also be substantiated from Fig. 4b, c which shows that the PDFs for MIROC-ESM model (shown by the green line) and MIROC4h model (shown by the blue line) at their native resolutions. The MIROC-ESM model overestimates the low to moderate precipitation intensity (0–20 mm day−1) but severely underestimates the high precipitation intensity >20 mm day−1. This model also fails to capture the precipitation intensity >120 mm day−1. In contrast to this, the fine resolution model, MIROC4h, is very close to the observations and captures the extreme events realistically.

Fig. 4 a Annual cycle of all-India weighted area average pentad rainfall (mm day−1) from IMD and MMM. PDFs (%) of daily rainfall from IMD, mean of the distributions of individual models, MIROC-ESM and MIROC4h for b 0–20 mm day−1 range with bin width of 0.2 mm day−1 and c 20–500 mm day−1 range with bin width of 5 mm day−1. JJAS rainfall trends (mm day−1 per decade) from d IMD and e MMM. Grid points with a trend value significant at the 90% level have been stippled Full size image

Finally, we analyze the fidelity of the MMM in reproducing the observed spatial pattern of trends in the last few decades. Figure 4d shows that the trend in IMD precipitation is less consistent in space as compared to temperatures. The trend values are significant (stippled) over a very limited region of Indian land and over this limited region, there is a drying trend in precipitation with values from −0.3 to −0.6 mm day−1 per decade. However, during the same period, the MMM shows a weak positive trend over the monsoon trough zone, and the values are not significant over most of the grid points. Thus, one may conclude that the MMM fails to capture the observed rainfall trend over the Indian land.