ASSESSING THE ACCURACY OF RECENT HOTEL OUTLOOK FORECASTS

Gaining Confidence Using Econometric Forecasts

DRAFT SUMMARY REPORT - JANUARY 2005
MADE AVAILABLE FOR 2005 ALIS CONFERENCE

WHAT IS HOTEL OUTLOOK AND WHO PROUCES THE FORECASTS?

Hotel Outlook is a product offered to hotel capital suppliers, managers, and others with interests in hotel markets that presents to clients quarterly forecasts of average daily rate (ADR) and occupancy for six years. These forecasts of hotel room revenue (aka revenue per available room - RevPAR) result from combining (1) historical room revenue data assembled by Smith Travel Research (STR), (2) historical economic data from Economy.com, (3) forecasts of macroeconomic performance measures developed by Economy.com, and (4) a state-of-the-art econometric model that finds relationships between hotel market and economic performance and then generates predictions. With one exception noted later in this report, Hotel Outlook presents forecasts in an entirely objective manner. Current hotel market coverage is as follows: 

• United States Market - At this time, forecasts of hotel room revenues are produced for the U.S., but not for other nations. The U.S. hotel market is defined as 51 of the largest hotel markets in the nation.
• Two Market Segments - Hotel Outlook includes Full-Service and Limited-Service room revenue forecasts. These segments represent the two dominant business models in the hotel sector. Forecasting at finer market segment delineations requires highly specialized techniques and analysis because of the overlapping demand among market sub-segments (e.g., upscale and upper upscale sub-segments of the full-service segment).
• Metropolitan Statistical Area (MSA) Markets - Room revenue forecasts cover the 51 largest MSA hotel markets. Most of the focus of Hotel Outlook clients is on the financial performance of hotels located in the largest MSAs in the U.S., such as New York, Chicago, and Orlando. Because of the complex interactions between gaming and hotel businesses in Las Vegas and Atlantic City, room revenues in these MSAs are not forecast at this time.
• Affiliated Hotels - Room revenue forecasts are provided for hotels that STR defines as having either ownership by or a contractual relationship with a nationally recognized brand.
• Annual and Quarterly Forecasts - Hotel Outlook includes forecasts on both a quarterly and an annual basis for each market for six years into the future.

In summary, Hotel Outlook comes to clients online in a convenient EXCEL format at quarterly frequency and includes room revenue forecasts by quarter and by year for the upcoming six year period. These forecasts of room revenues for affiliated hotels cover each of 51 MSA markets and the sum of the room revenues in these markets which defines the U.S. market. Separate forecasts are provided for full-service, limited-service, and the combined total of these two market segments.

Profile of PKF Hospitality Research (www.pkfc.com)

Headquartered in Atlanta, PKF Hospitality Research is the research affiliate of PKF Consulting, the international consulting and real estate firm specializing in the hospitality industry. Along with PKF Consulting and the PKF Capital Markets Group, the company has offices in New York, Indianapolis, Philadelphia, Washington DC, Atlanta, Houston, Dallas, Los Angeles, and San Francisco.

Profile of Torto Wheaton Research (www.twr.com)

Boston-based Torto Wheaton Research, a wholly-owned subsidiary of CB Richard Ellis, is the premier provider of commercial real estate forecasting, analysis, and consulting services for office, industrial, retail, multi-housing, and hotel property types. The firm provides unrivaled market knowledge through a full suite of research products and specializes in commercial real estate risk management through strategic debt and equity consulting. Highly sophisticated and reliable forecasting models, along with proven analytical expertise, have earned the company international recognition.

HISTORICAL BACKGROUND

The motivations to begin offering econometric forecasts for U.S. hotel markets came from expressions of interest by subscribers to TWR's Outlook series covering other property types to have a companion product for hotels and from the unsatisfied demand for objective, MSA-level hotel market forecasts observed by PKF among its clients. Following the completion of initial modeling research by Wheaton and Rosoff (1998) and the acquisition of historical data from STR, PKF and TWR formed a joint venture late in 2000 to create Hotel Outlook. Further model development and testing occurred during the first eight months of 2001 culminating in the initial release of Hotel Outlook (Fall 2001 edition) on September 6, 2001. Following the catastrophic events of September 11, 2001, the Fall 2001 edition was recalled and subsequently re-released on September 28, 2001. As shown in Table 1, Hotel Outlook is in its 14th edition at the time of this report.

The second half of 2001 was an incredibly difficult time to forecast hotel market performance. The economic recession in the U.S. beginning in early 2001 impacted hotel demand by the summer of 2001. The unprecedented events of September 11, 2001 created fear of air travel never before experienced in the world. In addition, it became clear during early 2002 that the U.S. would mount a war effort in the Middle East of an unknown scale.

Economists at PKF and TWR produced various revisions of the Hotel Outlook econometric model for the Fall 2001 through the Summer 2003 editions to incorporate the effects of the following influences on hotel revenues:

• The timing of the decrease in hotel revenues following the decline in general economic activity. Hotel revenues historically react quickly to shifts in general economic conditions, but reactions do not occur instantaneously. Moreover, the effects of recessions are differentially experienced across MSA markets and hotel market segments.
• The initial impact on hotel occupancy and ADR in all markets from the fear-of-travel stigma introduced by the terrorist attacks of September 11, 2001.
• The dissipation rate of the initial fear-of-travel effect as the stigma wore off at varying rates across metropolitan markets and hotel market segments.
• The anticipated effects of the U.S. war effort prior to the Afghanistan initiative and Iraq War, the actual effects of these military actions, and the dissipation rates of hotel market impacts as these actions approached conclusions.

The confounding influences of the exogenous event sequence experienced from mid-2001 through mid-2003 created so much volatility in hotel room demand that it became nearly impossible to develop accurate forecasts. Psychological factors that cannot be captured by the economic variables in econometric models dictated market movements during this period. The revisions to Hotel Outlook made during this period resulted in forecasts that showed the correct future directions of RevPAR changes, but the models could not accurately predict the magnitudes of change. Markets, such as San Francisco and Boston for example, experienced declines of unprecedented size during 2001 and 2002.

MAPPING HOTEL OUTLOOK ACCURACY

At the completion of the military action to remove the Iraqi government from power, the psychological influences on business and leisure traveler behavior all but disappeared from U.S. hotel markets. During the summer of 2003 it became clear that hotel market movements were again directed by economic activity, and coincidently these markets had moved to a state of recovery as the economies in most MSAs turned upward. For the reasons presented above, we begin assessing the accuracy of Hotel Outlook forecasts with the Fall edition issued in September 2003 using STR historical data through 2003 Q2. As shown in Table 2, beginning with the Fall 2003 edition allows us to compare two annual and fifteen quarterly Hotel Outlook forecasts against actual hotel financial performance.

Table 3 presents additional information about the scope of this accuracy assessment. Note that in parentheses next to the edition label in every exhibit in this report are two identifiers - the last quarterly period for which STR data were available for forecasting (e.g., 03Q2) and release date for that edition (e.g., 9/03).

Accuracy Measurement

Forecasting accuracy may be assessed in a variety of ways. Given the short history of Hotel Outlook, our assessment at this time focuses on MSA cross-sections by market segment. Furthermore, we assume that clients prefer specific comparisons of forecast-to-actual results as opposed to relative comparisons to naïve forecasts using methods such as moving averages and other extrapolation techniques. Thus, we rely on the percent variance of each forecast from the actual result as the principle method of assessment. Percent variance for each measure is computed as follows:

V_it = (F_it-A_it) / A_it

where V_it is the percent variance of the i^th forecast relative to the actual outcome during period t, F_it is the forecast of the i^th hotel market measure in period t, and A_it is the actual outcome of the measure.

The average of the absolute values of percent variances (i.e., mean absolute percent variance) across all MSAs serves as a summary measure. We report the number of cases when the forecast was greater than or equal to the actual outcome and less than the actual outcome. The averages for these cases also are reported.

ASSESSMENT OF ANNUAL FORECASTING ACCURACY

Because this analysis was completed approximately six weeks prior to the release of year-end 2004 data, assessment of annual forecast accuracy is confined to calendar year 2003. As shown in Table 2, we perform this assessment from the Fall and Winter Hotel Outlook editions. This means that the annual forecasts for 2003 were produced with full information about hotel market performance from the first half and first three quarters of 2003, respectively. Future accuracy assessments will include more annual assessment opportunities over longer forecasting horizons.

ASSESSMENT OF QUARTERLY FORECASTING ACCURACY

The quarterly assessment map presented as Panel B of Table 2 shows that a total of 15 quarterly forecasts can be assessed from Hotel Outlook editions beginning with Fall 2003 and ending with Fall 2004. For each edition, the first quarterly forecast was produced during the middle of that forecasting period.

SUMMARY OF FORECAST ACCURACY

The information contained in this section represents a 'snapshot' of the results from assessing Hotel Outlook accuracy to this point in time. In Table 4, we present annual and quarterly RevPAR accuracy measures for all editions released since Fall 2003.

The table shows accuracy for the components of RevPAR - ADR and occupancy - and by market segment in addition to all hotels (i.e., sum of markets). The accuracy measure for the U.S. is the percent variance between forecast and actual results. The mean absolute percent variance summarizes the accuracy across MSA markets. This measure is computed by taking the absolute value of each MSA percent variance (i.e., removing the negative percent differences) for the performance measure and then calculating the arithmetic average of the absolute values.

Accuracy of U.S. Hotel Market Forecasts

Forecasts of financial performance for larger markets over short forecasting horizons generally have higher levels of accuracy than forecasts of smaller markets and over longer horizons. Thus as expected, the near-term Hotel Outlook forecasts for the U.S. hotel markets are highly accurate. All of the annual forecasts are within one percent of the actual results. Forecasts of ADR fall just below actual ADRs and the model slightly over-predicts occupancy. The negative ADR percent variances and positive occupancy percent variances have an offsetting effect on the RevPAR forecast accuracy. The accuracy of annual forecasts for all U.S. hotels, U.S. full-service hotels, and U.S limited-service hotels is nearly the same.

The accuracy of quarterly U.S. hotel market forecasts follows a pattern similar to the pattern of the annual forecasts. Nearly all forecasts of ADR, occupancy, and RevPAR come within two percent of the final numbers released by STR. Most forecasts of ADR fall short of actual results. Some forecasts of occupancy lie below actual results which has a compounding effect on RevPAR accuracy when the model also under-predicts occupancy. As with annual forecasts, no discernable differences can be observed between the accuracy of full-service and limited-service hotel performance measures on a quarterly basis. Finally, we expect accuracy to decline as the forecast horizon is extended, but contrary to this assumption, the forecasts for periods four quarters into the future are no less accurate than forecasts one and two quarters into the future.

Accuracy of MSA Hotel Market Forecasts

Hotel Outlook forecasts MSA hotel market performance with a high level of accuracy. The accuracy of annual forecasts falls in a range of one-to-two percent, and in most instances, the accuracy of quarterly forecasts comes within five percent of market performance reported by STR. As with forecasts of U.S. hotel market performance, no discernable differences can be observed between the accuracy of MSA full-service and limited-service hotel performance measures on a quarterly basis. The forecasts for periods four quarters into the future are no less accurate than forecasts one and two quarters into the future.

CONCLUSION

This report previews the results of a comprehensive assessment to be released soon of Hotel Outlook accuracy since the Fall 2003 edition. The summary table (i.e., Table 4) indicates that at best RevPAR forecasts for the U.S. are within one percent of actual results, and at worst, within 2.5 percent. Across the MSAs, the best RevPAR forecasts fall within three percent of STR reported numbers and at worst six percent. Forecasts of hotel market performance for any particular market (e.g., Atlanta limited service) can vary considerable from the absolute mean percentages presented in Table 4. These results will appear in the final report on Hotel Outlook accuracy.

REFERENCES

Wheaton, W. and L. Rosoff. 1998. The Cyclic Behavior of the U.S. Lodging Industry. Real Estate Economics 26(l): 67-82.