Posted on 04/15/2016 12:53:26 PM PDT by MichCapCon
Background
Previous articles in this series have documented the secrecy surrounding what have been called budget justification reports purchased through no-bid contracts by the Michigan Economic Development Corporation from a company called Longwoods International. (See Pure Michigan Puffery Parts I and II as well as Puffery or Proof?) Longwoods produces very similar products for agencies in at least eight other states, claiming high returns from taxpayer-funded tourism ads.
The MEDC is this states agency in charge of business subsidy and assistance programs. Last year it got a $33 million appropriation to run its Pure Michigan ad campaign. The agency would like to get another $33 million or more this year.
However attractive you think the Pure Michigan ads may be, they amount to pure corporate welfare for the travel and leisure industry. This reality, and the fact that even in Lansing $33 million is real money, means that during appropriations season legislators and the public may need some convincing.
Thats where Longwoods comes in. The most recent report it did for the MEDC purports to show a $6.87 return for every dollar spent on Pure Michigan ads in 2014. Whether this is accurate or even plausible cannot be determined because Longwoods refuses to reveal all the details of the methods it may have used to generate the claim.
The MEDC paid almost $150,000 for this product, but agency officials havent shown much curiosity regarding their hand-picked vendors claims. To the contrary, the MEDC has offered a dubious defense of the secrecy.
Who Watches the Watchers?
Such unanswered questions are not helpful for budget justification purposes. (Thats how Longwoods characterized research it produced for another states tourism agency. The statement was made on a page that has since been taken down from the companys website.)
So it is interesting that in 2011, when pressed on similar questions about earlier Longwoods reports, the then-head of Michigans travel promotion bureau pointed to a validation study for which the MEDC paid a different contractor $5,000. While this was supposed to confirm that the Longwoods methodology was valid, it also suggests the agency knew these reports had a credibility problem.
Moreover, even this second vendor said Longwoods had a lack of total transparency, and acknowledged that reviewing its process would require costly replication to totally confirm that its claims were valid. But even if cost were no object, Longwoods secrecy would make confirming its claims impossible.
Dodgy Methods Acknowledged
We do know some things about the method, although the details provide little comfort. Bill Siegel, founder and CEO of Longwoods, has said his company uses a hybrid system based on what are called conversion and advertising tracking studies. The former are said to measure how many people responded to a travel ad by later visiting the destination.
Nevertheless, traditional conversion study methods appear to have been what the validation study authors used for their work. Conversion studies have been widely criticized by academic economists, who consider them highly dubious. In his book, Tourism Analysis: A Handbook, author Stephen Smith writes that especially with tax-funded travel promotion agencies Conversion studies are increasingly popular as a tool to justify budgets. Because of survival motivation, many conversion studies fall short of full adherence to rigorous scientific procedures. Smith wrote this was especially true of tax-funded agencies that promote tourism interests.
Even Siegel (writing with colleague William Ziff-Levine) said in 1990 that, The main value of a conversion study would appear to be to get diagnostic information on a fulfillment piece, not to provide hard numbers on the bottom-line return on-investment of a campaign. More recently, he said that this hybrid approach uses the parts of the conversion study system that he believes are its most useful and effective.
And then there is the secrecy. While you can read an online copy of the survey Longwoods used to produce its 2014 report, it is meaningless without a detailed explanation of how the company used the survey to derive its claim of a 587 percent return from an ad campaign. And such an explanation is exactly what you wont find on any Longwoods website or published document.
The MEDCs validation contractor was supposed to evaluate the legitimacy of Longwoods claim. Notwithstanding Longwoods lack of transparency, this second outfit chose to attempt its own version of a conversion study.
Process Fails Smell Test
As if doubts about the underlying contract werent enough, there are many reasons to question the practice of paying a second contractor to validate the claims of a budget justification contractor.
To begin with, the MEDC did not issue a request for proposals, which could have given it a number of validation contractors to choose from. Instead, agency officials hand-picked a contractor, which at the very least limited the scope of possible perspectives. Also, its hard to ignore that the co-author of the validation study is a former vice president of Longwoods International, the company being evaluated.
That revolving-door relationship is part of a pattern. The MEDC official who mentioned the validation study in 2011 was George Zimmerman, whose title was vice president of the MEDC's Travel Michigan office, which oversees state promotion efforts. Mr. Zimmerman has since been hired by Longwoods International as its chairman.
The questions multiply when details of this second contractors work are examined. For instance, the survey research it used involves asking respondents to report the degree to which the state of Michigans advertisements influenced their decision to visit. This introduces the possibility of response bias a pitfall of survey research which would influence the output. Moreover, the authors simply lump in two categories of responses at different rates before making their calculations. They claim this is a common technique but offer no supporting evidence. Curiously, one result derived by the validation authors apparently matched the Longwoods International conclusions to the penny ($3.42).
Confirmation Study Confirms Nothing
Perhaps not surprisingly, the validation study authors so thoroughly hedged and qualified their report that they appear to doubt whether return on investment claims like Longwoods can actually be measured. In one place they wrote, The real conversion rate in [sic] unknown and next to impossible to derive given the many variables involved, many beyond the control of the researcher.
Elsewhere, contractor No. 2 admitted, that the possible confidence interval for our estimate is wide, and that our bottom line estimate will likely be inflated to the degree that other Michigan advertisers were also engaged in these markets at this time at a level beyond the precision of this estimate. Translation: They came up with a ballpark estimate in a very large ballpark. Still the contractor suggests that even if theyre off by 25 percent the result is still positive.
The outfit that was paid $5,000 to prepare this report can at least be commended for admitting they dont know what they dont know.
Perhaps capping all the preceding, 2011 press reports describe former MEDC and current Longwoods International official George Zimmerman repeatedly pointing to the validation study to support his companys claims about MEDC ad spending. However, the validation study contract was entered into in 2007 to cover a Longwoods report on a 2003 tourism ad campaign. Published reports indicate that the state had validated Longwoods technique but failed to mention that the validation work involved ads unrelated to the Pure Michigan campaign.
Conclusion
To sum up:
MEDC officials have granted a series of no-bid contracts to Longwoods International, a company that has basically advertised itself as a budget justification assistance service. The agency asked the company to quantify a positive return from tax-funded tourism ads, not to discover whether the spending generates a positive return meaning it simply assumed a positive return. (See Puffery or Proof? for details.) The MEDCs close ties to Longwoods are mirrored by a tie between Longwoods and the firm hired to confirm the validity of its claims. Independent verification of Longwoods methods is impossible, however, because the company refuses to reveal the details of those methods. Recognizing it may have a credibility problem, the MEDC hired a second firm to examine Longwoods claims about a particular ad campaign. The second contractor all but admitted that the thing it was hired to do verify Longwoods extraordinary return on investment claims was impossible. All of this amounts to an ongoing campaign by officials in a government agency to justify continued spending on a corporate welfare program thats a major component of their annual budget, with no credible evidence that taxpayer dollars arent being flushed away.
Yet policymakers and taxpayers are expected to presume that these officials are acting solely in the public interest rather than their own interest and that of the industry benefiting from this corporate welfare. The events described in this series make this presumption exceedingly difficult to grant.
Indeed, it has become almost impossible to avoid the conclusion that it all amounts to a self-serving campaign of self-promotion by an agency whose continued existence and legitimacy are the focus of growing skepticism from lawmakers and taxpayers.
Bookmark
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.