An accurate measure of inflation is very important for accurate national statistics. Measured inflation is used to adjust nominal GDP to real GDP. Changes in Real GDP over time are used to gauge improvements or declines in living standards. If inflation statistics are incorrect and overestimate the level of inflation, then assumptions based on real GDP changes can be incorrect.

Measuring price changes across products to estimate inflation can be extremely difficult. In the U.S., the Consumer Price Index (CPI) measure of inflation attempts to track price changes across a representative basket of products for an average consumer. Arriving at a representative baket of products for an average consumer is difficult, and mistakes could change measured inflation if categories that are declining in price are underweighted, or assigned too small a percentage of the basket. Each category of goods in the CPI is assigned a percentage of the entire basket of the CPI based on the Consumer Expenditure Survey.

The CPI adjusts measured price changes for quality changes; a product with quality improvement and no price change is adjusted to account for its quality improvement. For example, if computers as a category became much faster and had a larger hard drives while selling at the same prices, the computer category would be deflating, which is negative inflation.

The CPI basket had an annual inflation rate of about 1.03% between September 2012 and September 2016, meaning that prices on average increased around 1% per year. However, different categories in the CPI basket experienced different rates of inflation. For example the telephone hardware, calculators, and consumer information devices category, which is .091% of the CPI index, had an annual inflation rate of -8%. A rate of -8% implies that this category has either gotten 8% cheaper every year or 8% higher quality, or some combination of both. The .091% value indicates that according to the Consumer Expenditure Survey, consumers on average spend .091% out of their total yearly spending on this category (the denominator includes spending on housing and trasportation). A related category in the CPI basket is wireless telephone plans, which was 1.74% of the index in September 2016. Between September 2012 and September 2016 it had an annual inflation rate of around -2.3%.

Deflation, or negative inflation, can be a sign of productivity improvements and technological progress. When things get cheaper or better, consumers benefit. The CPI generally attempts to reflect quality improvements in its measures of inflation. However, deflation due to quality improvements can be very hard to estimate, particularly when new models frequently replace old versions.

Smartphones have improved rapidly, while their cost has generally stayed the same or declined slightly. If CPI statistics are accurately measuring quality improvements, the telephone hardware category should have a large negative inflation rate (or deflation rate). A simple hypothetical is to imagine a smartphone introduced five years ago and to picture what that phone would sell for if it were introduced brand new today. For example, the iPhone 5 was introduced September 2012 with a price of $650 for the base model. An inflation rate of -8%, which is the figure from the CPI, implies that such a phone introduced new today would sell for around $430. A larger negative inflation figure, such as -15%, would imply that the phone would sell for less — around $340.

Some economists have found that in several product categories the CPI does not fully account for quality improvements in new products and therefore overstates inflation. Typically the CPI overstates inflation when products are discontinued and replaced by new versions: for example a 2015 model year Toyota Camry replacing the 2014 model year version, or the iPhone 7 replacing the iPhone 6S.

The economist Mark Bils has an informative paper on this topic. Bils finds that existing car models gradually decline in price over their life span, while new car models are more expensive than the models they replace. He determines that new cars are more expensive mostly because they are are higher quality, and not because of inflation, whereas the CPI considers the higher prices of new models to be inflation. Higher prices because of better quality should not be considered inflation; depending on the dollar value of quality improvements higher prices could co-exist with deflation if the quality improvements more than offset the price increases. Bils reaches his conclusion about new car models because he finds that consumers shift their purchases to those car brands with new car models. Consumers buy fewer cars from brands and lines with old models. Assuming consumers are rational and not overly swayed by advertising, consumers would only switch from cheaper older models to more expensive newer models if the price differences were “worth it.”

Bils uses sophisticated techniques to reach his results, though his conclusion reflects the straightforward intuition that most products are getting better over time, and usually price increases in new versions of products are due to better quality and more features. Bils has to use advanced techniques to estimate quality improvements in new car models, because usually new car models do not overlap with older versions of the same model. The old model is usually discontinued immediately after a new model is introduced.

Price measurement would be much simpler if older models continued to be sold and received a discount at the exact same time that a new version of the same product was introduced. Assuming that manufacturers discounted old models so that they were still attractive to consumers, the discount could show the quality improvement in the new version. If a manufacturer followed this practice for several model versions in a row, the price decreases could be chained together to estimate quality improvements, and the inflation or deflation rate.

Luckily, iPhones seem to follow this model fairly closely. iPhones are discounted by Apple only once, at the introduction of the following model. They are discounted but still sold as brand new phones, intended for more budget minded consumers. They are usually but not always discounted by $100 from an introductory price of $650. They are discontinued one year after discounting (two years after introduction). New models follow the same pricing pattern as the old models did when introduced — they start at $650 for the base storage capacity. Each model has followed this pattern since 2011. My argument is that a $100 discount implies that the quality improvement of a new iPhone is “worth” $100 — the new model sells for $650 and the old for $550. Such a quality improvement for the base model would be approximately equivalent to a -15% inflation rate ($100/$650).

The CPI lists a deflation rate from September 2012 to September 2016 for telephone hardware of around –8%. This figure seems low. I use these months as cut off dates because the iPhone 5 was introduced September 2012, and the iPhone 7 was introduced September 2016.

Based on my constructed index of iPhones, which I created by chaining together iPhones starting with the iPhone 5 and ending with the iPhone 7, I arrive at an annual rate of inflation around -16%, significantly lower than the CPI figure. Thus the CPI is overstating inflation, and understating deflation, for this product class. An error in one product category, if the category is large enough, could affect the accuracy of the entire measure. I arrive at my deflation figures by averaging the price declines for each year, with some slight adjustments for the iPhone 5C and the 6S, since those models changed slightly one year after introduction. I estimated an average price decline of -10.75% for 2013 (iPhone 5 to iPhone 5S and 5C), -17.7% for 2014 (iPhone 5s to iPhone 6), -13.43% for 2015 (iPhone 6 to iPhone 6S), and -21.93% for 2016 (iPhone 6s to iPhone 7. These figures are probably conservative, and are based on some assumptions and adjustments. They are different each year partly due to adjustments and also because of the introduction of the Plus models in 2014. I weight each model equally and average the price changes. Chaining together the price decreases follows logically because of the yearly pattern with each model discounted one year after introduction.

My conclusions from iPhone prices rest on a few assumptions, 1.) that consumers are largely rational and not overly influenced by advertising campaigns, 2.) that a “fashion effect” demand for the latest iPhone does not account for the higher prices of new models, and 3.) that Android phones follow similar quality improvements as iPhones (which seems likely). A fashion effect occurs when new items are priced higher than older versions for fashion reasons, and not because they are better. Consider the latest designer clothing, which may have identical quality to last year’s designer line, but a significant markup (and last year’s line might sell a discount). I do not find evidence for a fashion effect with iPhones.

Note that the CPI category for telephone hardware also includes calculators and personal information items. However, non-smartphone items are insignificant compared to the huge size of the smartphone market, and therefore unlikely to affect these figures.

You may think this CPI error is relatively minor since telephone hardware is such a small part of the CPI. However, smartphones should actually be a larger part of the CPI. Based on internet statistics which show that the smartphone market is around $52 billion annually in the U.S., the telephone hardware category should be around .45% of consumer spending. I emailed the Bureau of Labor Statistics (BLS) about the discrepancy between their figure of .091% from the Consumer Expenditure Survey. The BLS responded that it was possible that smartphones were miscounted as part of the wireless plans category, since many consumers may report their monthly cell phone payment as part of their wireless plan bill. The BLS was clear that this was not intentional and that it had instituted policies to try to fix the issue, but was probably was not completely successful. The CPI might therefore be further biased because the wireless plans category could be too large of a percentage of the index based on actual consumer spending, since the wireless plans category includes some spending on smartphones. As noted above, the wireless plans category is 1.74% of the index, or 19 times larger than the value for spending on smartphones. The wireless plans item has an inflation rate around -2.3%.

In conclusion, I believe the BLS does important and difficult work measuring prices monthly with the CPI. Properly adjusting thousands of different goods with constant substitutions and quality changes, while updating the basket of goods to reflect current spending habits is a daunting task. However, we should not take all CPI statistics at face value, and we should attempt to bring some domain knowledge to test the accuracy of the statistics in specific areas. With smartphones, it seems the CPI has not captured the high level of quality improvement in this category. Further, the CPI does not give enough weight to smartphones in its index, as the percentage of the telephone hardware category of all spending suggests an implausibly low level of yearly spending on smartphones. This would further bias the inflation rate. An overstated inflation rate means that we are not fuly capturing the increase in living standards.