Enterprise Initiatives

A few days ago I had a chance to catch up with iTKO's co-founder and "Chief Scientist" John Michelsen to discuss SOA Testing. For those of you unfamiliar with iTKO, they produce one of the industry leading SOA testing products known as Lisa.

I have been writing about SOA testing a lot lately and the folks at iTKO contacted me after I wrote Six Questions to Consider before Building a SOA Testing Team on CIO.com. Since this article was fresh on my mind, I asked John some of the same questions that were posed in the article.

My first question to John was to identify some of the characteristics of successful SOA implementations he has seen from his customer base. Without hesitation, John pointed to customers that integrated testing into the development cycle. Successful SOA implementations tend to shy away from old waterfall methodologies and involve QA early in the life cycle. In many cases, testers work side by side with developers. John mentioned that "development builds things from scratch, QA does not". He pointed to this three step process which is the trademark of successful SOA testing:

1. Development builds the testing framework
2. Development builds initial test cases
3. QA enhances the test cases

Steps 2 & 3 is an iterative process and requires good collaboration between development and QA. This advice is similar to what I recommend in my article SOA Skillsets: PArt 4 - The Testers. In this article I recommend that a Test Architect be part of the EA team and design the testing framework and strategy. This person needs to be extremely technical and have a deep understanding of SOA concepts. The test leads also need to have a thorough understanding of SOA while the remainder of the testing team does not need to be as strong technically. However, they do need to be able to write some code as the developers hand over their test cases for refinement.

I asked John how technical the testers need to be. He stated that "most QA folks are still business analyst types and it should stay that way. If QA gets too technical then they lose the focus on testing". I agree which is why I advocate having the Test Architect as a member of the EA team focusing on building out the framework, evangelizing, and training the QA team.

I asked John about the importance of testing tools. He stated that the "Testing tools need to let less technical people perform technical tasks". This is a good point because your testers need to more focused on validating that the business and technical requirements are met than figuring out how to master a new tool.

This last point led us to a discussion about iTKO's Lisa products. John mentioned that they have just released version 4 which has many new features. In addition to the core product, Lisa Enterprise, the Lisa Server product has two new important modules, Continuous Validation Service (Lisa CVS) and Virtual Service Environment (Lisa VSE). Lisa CVS addresses the challenges of integration testing, deployment issues, and validation of governance compliance. Anybody with experience in SOA will tell you that the distributed and loosely coupled nature of SOA adds a whole level of complexity in the areas of builds and deployments.


Lisa CVS helps manage and automate some of those complexities and takes a more proactive approach to management and governance by allowing you to "revalidate expected behavior". Lisa VSE takes advantage of virtualization technologies so that you can create images that can simulate behavior of your heterogeneous environment. They accomplish this by installing agents on your various environments (mainframes, database servers, web servers, etc.) which allows VSE to monitor and emulate performance by creating a server side model of your production environment. This model can then be deployed in a virtual testing environment and be tweaked to simulate various scenarios such as abnormal spikes in traffic, simulating new customer loads, or whatever scenario you wish to entertain.

John was real excited when he talked about version 4.6 that is due out towards the end of the year. In this release, Lisa will add functionality to allow customers to test stateful services. This has been the missing link in SOA testing up to this point. John also mentioned that Lisa is now integrated with all major registry and repository tools. In addition, Lisa is also being used for Web 2.0 testing due to the work they have done with web services and WS-* standards.

In summary, testing is a critical component in the quest to launch and implement a successful SOA. By creating a well thought out testing strategy that includes tools to automate and simplify testing, organizations can focus more on the business side of SOA and less time focusing on the technology side of the equation.