In the end, my answers always follow the same patterns, so here you are my tips on choosing the right web application. What should you evaluate when you choose a web application?
- Vision and people
- Standards and integration
I know, the KISS (Keep It Simple, Stupid) unix paradigm would suggest to have a web application which only does a single task - a best of breed, if you wish. However, while we do have software management tools in Windows or Linux which can somehow keep application proliferation at bay, we don't have such tools when it comes to web applications. Managing 5 or 6 web applications is a real burden when compared to handling just one: updates, dependencies and so on, everything (or almost everything) has to be managed by hand. And don't even make me start with application servers. So, definitely go for
You should not stop ad data sheets or feature lists: install the product if you can, or try it on a virtual machines (most vendors are now shipping demo virtual appliances) and try to build up an idea of the actual features. There's always some gap between what's on paper and what's in the code!
A strong community is really a must have for any web based product, no matter if it is open or closed source. In the opensource world, having a strong community means the product will unlikely die out, while for a proprietary apps it means you will have more than just the official documentation.
A community also means addons and plugins which almost any modern web product will support. Just watch out, because you should evaluate addons one by one in the same way you evaluate your application.
Browse the forums, noting the date of the last posts, try to find active sites or blogs about the products and take note of who's using it. Do not trust the "BigCompanyX is using our product" claims, since especially in opensource software, given a large enough company any software is bound to be used in some department.
Security is always an issue when it comes to web applications. You should check the application history (try for a Google query like "YOUR.APP.HERE exploit" or "YOUR.APP.HERE security advisory") to get a rough idea of how many issues the application had in the past. Note that no issue is maybe even worse than a lot of issues: it likely means that no independent reviewer ever took a look at the application...
Take care, when you assess a CMS: you will find a lot of security issues on external or third party components, and it's often hard to tell if they are there due to the lack of security of the component or of the platform itself.
Nonetheless, a OneBugAMonth history is definitely a warning sign..
This is very important when it comes to large projects, like Content Management Sytems. Where is the software going? What the company will do with it? This is the most hard part to judge, since you will most likely not have any resource. My advice: go for the people. Search the web for the coders, take a look at their tweeter accounts and their other projects, if any.
Should you need a way to retrieve some email address, try FOCA: it will automatically fetch email addresses from any document on the company's website.
Look for proper standards. "We're using Java so we are standard" is one of the most widely heard of claims, and doesn't make any sense. Look for actual interoperability: open source code, web services, open and well documented data format.
Even the most well guarded, proprietary application can have well documented standards, saving everything in XML and thus being much more easier to integrate in your environment. Remember integration is a must.
Last but not least, look at the requirements and the technology the solution has been built around. Deploying a Zope based solution today might not be a great idea from a strategic perspective, no matter how much I like Zope. On the other hand, using a technology you're completely unfamiliar with can slow down deployment and makes management much more difficult. This said, you should reiterate the analysis on the underlying component as well: maybe the application will require Tomcat, maybe Oracle Application Server.
It really does make a difference: choose the wrong technology you and you will end in a cul-de-sac in no time.