A major problem with using statistics is that today it’s much easier to count things than it is to decide exactly what to count, or exactly why to count. The answers we obtain when analyzing economic and commercial real estate data may reflect the real world, but there’s no guarantee that a set of questions are the right ones. In data science or statistical analysis, the quality of an answer entirely depends on the quality of the design of the question asked.
The way they describe this problem in the computer science world is: garbage in, garbage out. And out it comes indeed: when you ask the wrong question (or a question lacking in the right detail) the answers you get will come pouring out just as plentifully and convincingly as when you ask the right question.
The Retail Closures Question
As e-commerce continues to radically reshape the retail ecosystem, disrupting decades of assumptions about physical space, parking and real estate value, it’s perfectly reasonable to notice that a growing number of once-venerable retailing brands have closed, or are threatening to close, or are pointedly denying they will close.
In such a world, it’s reasonable to wonder what effect all this change is having on retail rents generally. That’s a good general question to put to statistical analysis, but incomplete in its basic form — you have to define exactly what retail rents are, and you have to decide what makes a good relationship between closures and those rents.
This is what Barbara Byrne Denham, an economist in the research department of Reis, Inc. has done. Her best effort to keep garbage out was to get a good handle on what rents were in metros across the country. She included data on rent growth, ranking metro areas by their growth in rent rates, such data coming from within her Reis data warehouse. From her piece “Impact Of Large Chain Closures On Retail Rents” published last week in NREI:
Few, if any, have analyzed the impact of these store closures on real estate statistics. Having property- level retail real estate data, analysts at Reis have been tracking store closures for the larger, more high-profile brands across the country. In short, the Reis database includes 280 store closures in 59 of the 80 primary retail metros that Reis tracks, totaling 12.8 million sq. ft. of closed stores across the United States. The major brands include Wal-mart, Kohl’s, Sports Authority, Pathmark, Superfresh, A&P, Waldbaums, Haggen and Kmart. Many of these closures were concentrated in a handful of metro areas, including Chicago, Central New Jersey, Northern New Jersey, Philadelphia, Long Island, San Diego and Los Angeles—all of which had more than 400,000 sq. ft. of store closures from 2015 through July of this year.
The report looks at the percentage of inventory that store closures account for and the change in rent growth rates by metro. The purpose of this analysis is to see if and how these store closures have affected rent growth rates. In short, the closures may have impacted these metros, but there is no overall conclusion that can be drawn from the data. It should be noted that this detailed data does not include details on whether or not the store spaces have been re-leased to other users. Some may have been in the interim.
The conclusions are carefully drawn in Denham’s work, as she meticulously spells out the limitations of the analysis, highlighting where and how it could differ from the real world. It’s not glamorous or provocative to be complete and correct about what a study has found, nor to be scrupulously above board concerning the work’s assumptions. The business world wants plain and actionable insights, validated by “crunching the numbers”. This isn’t that. Denham’s study suggests that the impact of sizeable retail closures on rent growth was one of many factors that contributed to declines in rent growth, and perhaps not even the strongest factor.
The study is a helpful look into a use of statistics to ask the right questions, to avoid garbage-in-garbage-out and to be scrupulous in never confusing correlation (stuff that happens nearby something else happening) with causation (stuff that happens because something else is happening). In an age marked by oceans of “big data” and thousands of software tools to work through it, it’s of growing importance that we all focus on the quality of the questions before we accept the significance of the answers.
Read the entire article at NREI here.