Tuesday, March 16, 2010

Don't over architect your application integrations!

Simple and agile should be your foremost design considerations in integration. Trying to get applications with different functionality and schemas to talk to each other via different protocols or APIs is complicated  enough, you don't need to add to it. Focus on the business reason why you're integrating the applications. If in doubt, ask your business user or CIO.

Keep these guidelines in mind as you embark on your next integration project:
  • If point-to-point works and its all you need, don't spend time and effort designing a hub-n-spoke model and an all encompassing uber canonical intermediary model. Unless you're the fortune 100 (50?) and have multiple identical applications across your enterprise, most of your integrations will be point-to-point. That's true even if you force a messaging bus (queue) in the middle. Don't confuse your underlying software/hardware architecture with your actual integration architecture. Everything may flow through a single hub (or server) but you can still just have a bunch of point-to-point integrations. If object A from application A only needs to be integrated with object A of application B, then stop there. If you have an integration platform worth it's salt, you can always come back and evolve your integrations.
  • Avoid custom code like your job depends on it, which it will. Custom code is the path to the dark side and the famous Gartner Spaghetti diagram. It's not point-to-point integration that gets you in trouble, it's the lack of a unifying integration platform that leads to Spaghetti. Using an integration platform will give you consistency across design and management of your integrations. If you're being tempted by the dark side to use custom code, remember that running tail on a log file doesn't consitute management and monitoring. If you're getting tempted to build your own awesome integration platform, go back to your business user or CIO and ask them what they wanted again.
When selecting off-the-shelf integration software, keep the following in mind:
    • Pick an integration platform that can deliver 80% of what you need out-of-the-box. Most enterprise application integration problems fall into the 80% category. The other 20% edge cases may need a different solution. Platforms and vendors that focus on those edge cases to differentiate themselves tend to provide the same model for solving all problems, with great pain and difficulty!
    • Make sure what ever solution you choose has solid coverage for the 3 basic types of application integration:
      • Data Synchronization - One time migrations or ongoing synchronizations.
      • Process Integration - Data syncs only get you so far, being able to write integration flow logic tying different business processes together is also important.
      • UI Integration (aka Mashups) - Perhaps the most under rated method of integrating applications. If all you need is visibility to data in another system, why synchronize the data. If you don't need to use the data in the target application for processing or reporting, then don't synchronize it. Most applications these days provide browser-based interfaces with the ability to embed iframes or other methods of incorporating external data sources. "... mashups can deliver an 80% solution at 20% the cost (or less) " (source: http://blog.programmableweb.com/2009/07/23/enterprise-mashups-continue-to-gain-momentum-as-part-of-enterprise-20/).
  • Apply standards when they make sense and avoid them when they only serve to complicate solutions. Having adopted SOA doesn't mean that every thing needs to be exposed as a service. It certainly doesn't mean that you should force adoption of overly complicated WS* "standards". Stick with the basics that you really need (vs what would be cool). You'll find varying levels of WS* "standards" adoption in the application market place. SOAP-based or REST-ful? What ever works. You'll find that you'll have to be able to support all kinds of different "standards". Make sure your integration platform is flexible.