Celebrity CMS Deathmatch – The Aftermath

There’s a lone soldier on the cross, smoke pourin’ out of a boxcar door,
You didn’t know it, you didn’t think it could be done, in the final end he won the wars
After losin’ every battle.
- IDIOT WIND

Part 1: The Beginning
Part 2: The Meme Spreads
Part 3: The Aftermath

Right, it is time to draw this chapter to a close. If you don’t know what this is all about, please read Part 1 and Part 2 mentioned above first. There has been a great response, and there is talk that an ECM focused meme will start soon too. This has been reported by Julian Wraith, who also did an excellent job keep track of all the responses. Thanks! Google Bertrand Delacrétaz’s Meme ID to find everything there is to find: 9c56d0fcf93175d70e1c9b9d188167cf.

For the record, the 24 vendors that responded (with links to their responses) are: Jahia, Ektron, Hippo CMS , Magnolia, EPiServer, Nuxeo, GX, Midgard, Knowledge Tree, infopark, Day, Alfresco, CoreMedia, Sitecore, OpenText, Alterian, dotCMS, Vignette, Autonomy/Interwoven and Escenic. Not yet in the table: eZ Systems, e-Spirit, Enano, Sense/Net.

Below is my summary of the results. Click the image for a larger version. Please note that everything here is extremely subjective. You’d be insane to use this as part of any vendor selection exercise!

Summary of the Scores

Notes on the scoring:

• Vendors shown with a blue background did not score themselves. So I scored them based on what I think their answers meant.
• GX gave themselves a 0 and a 1, where they meant a 1 and a 2. So their score rises from 40 to 42.

The Overwhelming Yes Questions

These questions had an average score of 2.8 and above:

• Amazingly, everyone thought their Software Does What It Says it did. That’s great news for the CMS buyer. Everyone is honest . Most don’t even try to justify this, although Ektron justify their reply with the fact that they have “more WCMS implementations than any other vendor in the marketplace”. I’d be interested to see the list/research on which this is based.
• They all claim to have a Free SDK. I wonder if Kas Thomas had a vendor in mind when he put that question onto the list?
• The No Reboot topic caused a bit of controversy, with three vendors claiming it isn’t important. I disagree with this. A reboot doesn’t cause any downtime in a load balanced environment, but I feel it is architecturally wrong for the CMS to go so close to the Operating System to warrant one.
• Alfresco were the only ones to admit that they need their technical presales engineers to help the Sales Guys. Maybe the other vendors included these techies as part of the sales team so gave themselves full marks.
• Again, Alfresco were the only ones that confessed their Sample Site wasn’t great. I’ve seen a lot of these sample sites, and some of them are really quite dire. I think a few more vendors should have given themselves a “Sort Of” here. But I guess the question did not ask for a Good sample site, so they are technically honest. They all have a Tutorial, most of which are good.
• Three vendors admitted that they didn’t have a Full Installer. dotCMS lost some points here, but at least it is on their roadmap. I do think that the Hippo were a bit hard on themselves. I prefer a standard EAR/WAR deployment to an install Wizard, especially in a large, clustered environment. A One Click Update question would have brought some interesting answers as upgrades are often much hard than installs.

The Differentiator Questions

These questions had an average score below 2.8:

• About half the vendors struggled on the No English challenge. The smaller US based vendors normally do worse here as many of their clients are single language. The European based vendors live and breath multi-language every day.
• Interwoven is the only vendor that does not offer a Download. Six others offer one with conditions attached.
• Most vendors eat their own Dogfood, apart from SiteCore, who drink their own pre-release champagne. Quite a few aren’t on the latest version though. Escenic gave themselves a 1 for this, while many other vendors gave themselves a 2. And I’m taking bets on when Vignette’s site is going to be running V7.6. KnowledgeTree gave themselves a very kind 3 seeing as they don’t do WCM. But their score does prove that most of the questions are noting to do with WCM and could apply to almost any software vendor.
• Escenic were the only ones that admitted their Price List is able to “adapt to a large variety of customers”. I believe that all the major vendors do this, and that the price can vary enormously. Maybe the answers refer to list price as opposed to the actual golf-course price, but I think Escenic showed honesty here that some of the others could have done. Vignette did also admit their model is complex. Half claim a 5-year old could understand it. But I hope this five year old can also drink a lot in the pub and negotiate a good discount or he is going to get screwed.
• Surprisingly, the question with the lowest average score was the Raise Issues From Product one. All vendors have an issue logging system, so this would be extremely simple to implement. It could just be a link from the admin screens to the support screens.
• The most controversial question was probably the All Help Files And Documentation are Part of the Install. Now I’m going to disagree with Kas on this and side with some of the vendors. I prefer online documentation to local installation for a number of reasons: It is kept up to date and continuously enhanced, it can include user submitted contributions, it can be powered by an advanced search engine, it can contain offsite links, and I don’t like having extra items installed on the servers. I do like to have a local of the SDK, but this should be embedded in the IDE and be an optional extra download as part of the SDK. And if people do still work when they are offline, the ability to download all the documentation would be a bonus.

The Jon Vendor Meme Awards 2009

Any great competition needs an awards ceremony. So, in the spirit in which this whole contest was conducted, I’m honoured to be able to announce:

• The Jon Award For Transparency goes to Escenic. Sure they came stone last, but I trust the guys.
• The Jon Award For Agility Above And Beyond Expectations goes to Vignette. I really didn’t expect them to respond, yet they were the first ECM player to do so.
• The Jon Award For The Best Product Name goes to Hippo CMS. Of course.
• The Jon Award For Anti-Cheating goes to Nuxeo, who miscounted their score on the low side. I’ve added a point to their reported score.

And thanks once again to Kas Thomas for his “A reality checklist for vendors“, and Day for starting this party, giving CMS geeks like me something to smile about. It’s been real.

UPDATE: 28 March 2009 – Added KnowledgeTree and Ektron.

UPDATE: 03 April 2009 – eZ Systems have responded. I’ll update the chart when I have more time.

UPDATE: 08 April 2009 – e-Spirit set themselves up for a fall by giving themselves a perfect score. Anyone know enough to see if they’re being cheeky? Not a vendor I know much about sadly. 45/45 seems a bold claim to make. They didn’t publish any contact details for question 15. I notice @espirit_news joined Twiter 15 minutes ago. Just in time. Well done, e-Spirit! Thanks for playing.

UPDATE: 09 April 2009 – Adding Enano (seeing Julian included them in his list) and Sense/Net.