Gartner's Magic Quadrant (MQ) for email archiving is not as scientific as Gartner would have you believe.
According to ZL Technologies Inc. v. Gartner Group Inc. and Carolyn DiCenzo, ZL Technologies makes allegations of defamation, trade libel, false or misleading advertising, unfair competition, and negligent interference with prospective economic advantage against Gartner and Carolyn DiCenzo. ZL Technologies seeks US$1.3B in punitive damages, $396M in damages, and other fees and charges.
You can see the actual suit document itself here. Among other things, it spells out problems with the archiving Magic Quadrant.
Since its inception, Gartner has consistently placed Symantec in the "Leader" quadrant of the MQ. Other archiving vendors such as ZL Technologies have for years struggled to get positive placement on the MQ. Getting placed in the Leader quadrant of the archiving MQ has become somewhat of a holy grail for archiving vendors, with Ms. DiCenzo the gatekeeper. In the suit, ZL outlines the many ways it has tried to convince Gartner that its technology is on par or superior to that of Symantec. According to the filing, "at the core of this action are not only the reckless statements made by Gartner as an influential member of the media, but an economic model championed by [Gartner] that elevates marketing puffery over serious technology."
Interestingly, Gartner's bio of Ms. DiCenzo suggests that she left Gartner in August. Whether her departure is connected with this case is unclear; she may have simply retired.
We fully recognize that at Ferris we, too, are analysts. That said, this case illustrates some important points:
- Any analyst firm is simply expressing an opinion, hopefully an educated one.
- No one knows the future. We can only make our best predictions, based on past experience.
- Any analyst vendor rankings need to be transparent concerning the underlying data. How was a decision made, and why?
- The link between monies spent by a vendor with the analyst, and how well the analyst evaluates the vendor and its products, should be minimal. Vendors commonly note that if they spend a day or two educating the Gartner analyst (and paying handsomely for the privilege), they often move to a better position in the MQ.
- Gartner MQs profess to give a quick feeling for how well products compare, in terms of several key indicators, notably excellence of technology and market share. However, when you look closely at them, they don't really hold up. Given MQ parameters, some products need to appear at two points in the quadrant: Gartner shoehorns them into a single point.
- Customers evaluating vendors should always look at the products themselves, and not rely solely on analyst opinion.
- Bigger is not necessarily better in the analyst world. At Ferris we pride ourselves on years of cumulative and deep experience in the fields we cover, but we're far smaller than Gartner.
... David Ferris
One Comment
David,
Excellent post – and thanks for the links/important points as they are very useful in helping to understand the challenge of “rating vendors”.
Best/Rob Robinson
Our chum and industry colleague Andrew Lochart has been commenting on this: See https://www.lochart.com/lochart/Blog/Entries/2009/9/18_Dont_Try_This_at_Home.html
–David
David:
Insightful report, thank you. I have watched Gartner gobble up its competitors over the past two decades until it has become one of the largest and most influential analyst firms in the market, and one of the hardest to work with as a vendor or PR professional. I should hope that size does not equivocate with influence. Their reporting is no better than many other analyst firms, and I agree that they don’t have the depth of knowledge in certain market segments and smaller firms who specialize. I will be curious to see if this suit reveals desperation on the part of ZL Technologies or will demonstrate that the Gartner Emperor has no clothes.
Cheers,
Tom Woolf
Oct 23, 2009. Just received this from one of our readers:
David – here’s the official statement following today’s hearing:
“In today’s hearing on Gartner’s motion to dismiss ZL’s complaint, the court did not come to a decision.
Though Gartner’s marketing collateral insists that its research is “objective, defensible and credible,†Gartner continues to assert that its research reports are effectively pure opinion, namely, opinions not based on fact. ZL believes this is clearly misleading to enterprises which expect better from Gartner than mere opinions-not-based-on-fact to resolve serious IT problems.”
David,
As usual, an insightful post. I do fully understand ZL’s position, however. It is absolutely clear that, valid or not, the MQ has enormous market sway. There is also some feeling in the industry that hiring Gartner is a necessary step to achieving better MQ placement. It would be interesting to know, and perhaps this suit will allow us to discover, the correlation between high placement and payments to Gartner.
Full disclosure: I am a former analyst with Ferris…
-ben-
I’m a former Gartner analyst — in fact, I worked in the same group as Ms. DiCenzo — and I’ve been very interested in reading about this case.
Andrew Lochart makes several good points in his entry — namely, that it’s important to have an ongoing relationship with Gartner, and that user views are very important. Gartner interviews such users year-round, not just associated with Magic Quadrants.
I didn’t work on Magic Quadrants personally — I worked for Datapro, which was one of the companies Gartner acquired — but I was aware of the process.
There’s nothing wrong with being a ‘niche,’ just like — to use Mr. Lochart’s analogy — not every movie has to be a blockbuster.