Enterprise Initiatives

I have discussed the SOA Evangelist, the Architects, and the Developers. Today I will discuss the role of the Testers and the characteristics required to contribute to a successful SOA implementation.

One of the most important roles and one that I probably should have included in the Architects post is the Testing Architect. As I have written on CIO.com in my article called Six Questions to Consider Before Building a SOA Testing Team, SOA testing requires a much deeper knowledge of technical skills, including development skills. It might be unrealistic to expect your entire testing team to possess the desired technical skills required to successfully test SOA, but your test architect and your leads should be able to understand concepts like statefullness, distributed computing, and data services, to be able to properly validate the underlying architecture. They also need to be able to take developer test harnesses and update them with their own test scripts.

A successful SOA testing strategy starts with the test architect. This person must have a very in depth knowledge of SOA and should work closely with the EA team. I actually recommend that this person is a member of the EA team, but every business and culture is different. The goal of the test architect should be to set up a framework and a core group of policies and procedures (part of IT Governance) so that the rest of the test team has the tools and the guidance to successfully test SOA. Without an established testing architecture, the company will have to rely heavily on expert knowledge from the entire testing team. I have seen three scenarios through my personal experiences and through my research.


Scenario 1: Underestimating SOA
The first scenario is out right failure caused by not having the tools and knowledge required. This is caused by a company not realizing that their current methodology and internal personnel are not well equipped for testing SOA. These companies do not invest enough in tools, training, and governance and usually can only test the presentation layer and possibly the interfaces. By not understanding the concepts of SOA, they are unable to validate the architecture which leads to poor performing and fragile services.

Scenario 2: Paying through the nose
The second scenario I have seen is relying too heavily on "expert" consulting firms for testing. In this scenario the company bets the farm on an experienced SOA consulting firm and pay rates that far exceed $100/hr. This model cannot be sustained for any length of time unless the company is willing to burn huge amounts of capital (which is not a popular thing to do these days).

Scenario 3: Good balance of internal/external expertise
A more desirable scenario is to train or hire a SOA testing architect, build a solid testing platform tailored for the needs of the organization, and govern the testing process while training the other members of the team. Companies should hire one or more experienced SOA tester, find an experienced consultant or two, or train an experienced and credible internal candidate to take the lead for creating the testing architecture. At the same time the testing experts are charged with transferring knowledge over to the rest of the internal team members. This is critical because a highly experienced SOA tester is in great demand and is a flight risk since they are a rare breed. It is critical to grow the knowledge base internally.

Testing needs
The needs can be broken down into these categories: People, Tools, Governance. So what are the characteristics of a successful SOA tester. The answer is dependent on the architecture that is implemented, which is related to the tools and policies that are put in place. Below is a diagram I often use to discuss the typical layers of an architecture.

From SOA Slides

As I mentioned in the previous discussion about the developers, I see the need and desire to specialize within the layers. The same holds true for testers. Work within the layers happens simultaneously in development. I recommend an iterative development and testing approach which means there should be testers working within the layers simultaneously as well.

Here is what I would strive to put in place, keeping in mind that these are roles and some people may fill multiple roles:

SOA Test Architect
Courageous leader with extensive working knowledge of SOA, distributed computing, integration testing experience, coding/scripting experience, and understands the business. It is likely that you may have to go outside to hire this person or bring in a consultant to assist your top level tester.

SOA testing leads
This person or persons must understand all layers of the architecture (let's not forget security either) and should be able to design test plans that validate both the architecture and the governance model. They should understand how to perform black, white, and gray box tests. Testing abstracted services requires extensive testing in the areas of security, performance, and regression. Throw in the versioning capabilities of services and the fact that service consumers can use services in ways that were not anticipated and the permutations of test cases start skyrocketing. The test leads need to practice risk based testing and balance risks, timelines, and costs. There is just so much more at stake here than in the traditional n-tier architectures.

Business process testers
The business processes should be modeled within some tool and will likely call one or many services. The process flow requires a series of decision points based on variables and constants and can trigger events such as notifications, alerts, other processes, error handling routines, services, and a variety of other possibilities. The testers need to validate the business process as a stand alone entity. For example, if the business process is "validate credit card", the tester needs to ensure that this process handles the inputs correctly, properly runs the validation process, and generates the appropriate output. At this point, the tester need not be concerned with any other processes or services. These testers must work closely with the business and/or business analysts and do not need the breadth of technical knowledge that the leads and architects must have (although it would help). They should be approaching the testing from a business standpoint.

Integration testers
These testers must be much more technical and understand how to work with XML, SOAP, WSDLs, networking & telecommunication concepts, statefullness, and various platforms and technologies (Java vs .Net, Windows vs Unix/Linux vs. mainframe, etc.).

User Interface testers
The company most likely already has an abundance of people who can test the UI. If your company is leveraging mashups, wireless devices, or consumer facing UIs, the complexity of the testing will be greatly increased. These testers may be need to become familiar with AJAX, RIA, portal technology, RSS, and a variety of social software.

Data services testers
These testers must understand concepts of data modeling, database CRUD operations, transformations, security and roles, authentication, and much more. Everything starts with the data and if errors are introduced in this layer, everything else is doomed to fail. You must have a very strong testing lead working in the data services layer since data is the foundation of any successful implementation.

Other areas of focus
The name of the game is speed to market and test automation is a critical component for making that a reality. Performance testing is extremely critical and organizations should practice simulations to try to anticipate future performance of all services. Validating the security model and the governance model should also be part of the overall test strategy. Whoever is responsible for designing the security test plan should read (and fully understand) this book.

Closing comments
I am by no means a testing expert (but I did stay at a Holiday Inn Express last night). I do read a number of testing blogs listed below:

• Randy Rice's Software Testing & Quality Blog
• Frank Cohen's blog
• The ITKO Lisa Soapbox: SOA testing, validation & virtualization
• SOA Testing
• Mike Kelly's testing blog

Make sure your organization funds the necessary training and tools for testing. From my own personal lessons learned, make sure you invest as much effort in testing and security up front as you do in architecture and development. On my project a few years ago, I wore my developer hat too much in the research phase and did not properly budget for what was necessary for testing. Do not make this same mistake. As hard as it is to sell the business on SOA, it is much harder to go back for more money, especially before any business value is realized!

Startups and Non-profits

Startups and non-profit organizations typically are constrained by limited capital budgets. Startups need to quickly launch their products and services at low costs. Cloud computing allows startups to rapidly deploy into a “leased” infrastructure supplied by a cloud computing vendor without having to procure a ton of hardware, hire system administrators, and build out a data center. In addition, the cloud computing option allows startups to pay as they grow as opposed to having to pay for maximum capacity out of the gates. Startups also have the luxury of starting with a blank sheet of paper as opposed to having to migrate their legacy applications to the cloud.

Many non-profits, like the United Way and Goodwill, are funded by donations. It is in everyone’s best interest that the majority of these donations are used to benefit the people who need help from these organizations. Building large data centers and hiring people to look over them is not a great way to achieve this goal. Instead, non-profits can now move their data centers to the clouds and spend more of its donated funds on worthy causes instead of infrastructure.

Small and Midsize Companies

Small and midsize companies have to stretch modest IT budgets to meet the demands of the business while providing 24x7x365 capable systems. Some of these companies cannot afford to support multiple data centers to protect against disasters and to provide business continuity. Others have a robust main datacenter and a scaled down backup datacenter that would allow the company to limp along for a short period of time if an unfortunate event was to occur. But the bigger challenge is to focus enough resources on revenue generation. The more resources these companies have keeping the lights on, the less demand from the business they can support. These companies often have to make some compromises in technology upgrades, architectural decisions, and infrastructure purchases to allow them to meet the needs of the business. I have worked in a few midsize companies over the years and witnessed this first hand. Every year we were asked to do more with less money and we often had to provide “good enough” solutions because we could not afford to build the level of robustness into the infrastructure and applications that we would have liked.

With cloud computing, small and midsize companies immediately get the benefits of world class infrastructure without having to procure, implement, and administer it. This frees up resources to focus on innovation and business demands. In addition, these companies can leverage multiple datacenters located in different geographical locations to meet their disaster recovery and business continuity needs. As the demand for computing resources increase, staff can add additional servers and disk on the fly by using tools provider by the cloud computing vendor without having to purchase any hardware. They simply just tap into some more computing resources into the cloud and pay for what they use.

Large Companies

Large companies with billion dollar IT budgets do not have a lot of the issues that the startups, non profits, small and midsize companies have. Many of these companies are watching from the sidelines as cloud computing matures, but still think it is too risky to jump in. The large companies that do adopt cloud computing will be gaining competitive advantages over their competition. Here are some of the reasons why.

Going green – Many billion dollar companies are under intense pressure to be more environment friendly. Replacing thousands of servers and other equipment is an extremely expensive and time consuming undertaking. Large companies can start by migrating their non-mission critical applications to the cloud. This allows them to go green quicker, cheaper, and get a chance to “kick the tires” on cloud computing at low risk.

Reduce costs – Energy costs for running hardware and cooling can be greatly reduced by moving applications to the cloud. In addition to reducing the amount of hardware to buy, this could prevent or at least prolong plans to acquire additional floor space, buy additional generators, or install more cooling systems.

Agility – Prototyping is another opportunity for using cloud computing. The business wants things faster from IT. IT can now quickly create a prototype by leveraging virtual infrastructure in the cloud and experiment with various configurations without having to go through long procurement cycles. Cloud computing can also be used to set up and tear down multiple test areas with ease and without having to free up or buy and configure hardware.

Security/Privacy - Large companies’ biggest fear is security and privacy. That is why some cloud computing vendors offer private clouds for their customers. Companies can enjoy all of the benefits of cloud computing within their own private VLAN and configurable firewalls to protect their data and privacy.

Conclusion

Regardless of the size of a company and its IT budget, cloud computing provides many benefits that just cannot be ignored any longer. For startups, non-profits, and small and midsize companies, cloud computing is a no brainer. Large corporations are being cautious right now. It will only take a few key success stories from a competitor who drastically reduces costs and improves its speed to market to get more large companies on board. In the mean time, large companies should look at piloting a non mission critical application in the cloud and continue to monitor the vendors as cloud computing continues to mature.

Keeping the enterprise secure is a challenge these days. Security specialists are concerned with a variety of threats from external issues (worms, viruses, rootkits, identity theft, etc.) to internal issues (lost/stolen laptops, data breaches, voluntary or involuntary exposure of confidential information, etc.). Unfortunately, many IT shops look at security as a technology issue and forget to address the business side of security. Some shops lock down their enterprise to the point where they impact the business's ability to be successful. While researching this topic I came across an excellent abstract from the Software Engineering Institute written by Richard A. Caralli and William R. Wilson. This abstract is a must read.

What is the goal of an organization's security strategy?
Security experts must understand that everybody in the organization is their customer. Many IT shops act more like a dictatorship and continue to put policies and technologies in place that make IT a hindrance to the business. Caralli & Wilson remind us that...

..."the ultimate benefactor of the security activities that an organization undertakes should be the organization itself."

..."the industry’s affinity for technology-based solutions alienates the “business people” in the organization."

..."anything that impedes assets and processes from doing their jobs potentially derails the organization’s ability to be successful."

They suggest that the CSO (Chief Security Officer) reports to someone from the business. I don't disagree, however, if the IT department is business focused, I see no reason why the CSO can't report to the CIO. The same argument is often made for business analysts, project management, and even the Chief Architect. This is a direct result of IT becoming out of touch with the business and taking a technology first approach to all problems instead of a business first approach. The authors go on to say...

"Managing security in the context of the organization’s strategic drivers, provides both advantages and conflict. On the one hand, this approach ensures that the goals of security management are forged from and aligned with the high-level goals of the organization. On the other hand, the strategic drivers and needs of the organization are often in conflict with the actions required to ensure that assets and processes remain productive. In addition, as the organization is exposed to more complexity and uncertainty (because of the increasing use of technology and the pace at which the organization’s risk environment changes), keeping security activities and strategic drivers aligned becomes more difficult. In the end, finding the right balance between protecting the organization’s core assets and processes and enabling them to do their job becomes a challenge for security management—and a significant barrier to effectiveness."

Examples of IT security becoming a barrier
Security has similar issues as governance, standards, best practices, and enterprise architecture when the strategy is too technology focused. A great example is IT's resistance to adopt collaboration technologies like social networking, instant messaging, and even wireless access. Security professionals are often so insecure, that they lock down the enterprise to the point where they stifle innovation and productivity. All of the fears that I keep hearing related to collaboration technologies are the same fears I heard when companies where looking into providing Internet access years back. To me, it is the fear of the unknown. Instead of trying to live in the mainframe era of the 60's and 70's where everything was centrally controlled and nothing was acceptable unless it went through the all-powerful administrators, IT people need to accept the fact the world revolves around the business and not IT. We need to change our old ways of thinking and acknowledge that as technology continues to change at a rapid pace our strategies need to change with it.

How do we solve this problem?
Whether the CSO (or whatever the title of the security leader is) reports into the business or IT is not relevant to me.What is important is, this person must fully understand the overall business strategy and the related IT strategy. A too rigid security strategy can hinder IT from doing their jobs as much if not more then it can impact the business. The armies of people in IT building systems and keeping the lights on are customers of the security office as well. The security strategy should be one that is created with the collaboration of representatives in both the business and IT. Within IT, the strategy needs input from more then just security and infrastructure departments. Caralli & Wilson recommend...

"In pursuit of addressing the challenges noted herein, the first obstacle that an organization must confront is to determine what they are trying to accomplish with their security activities. In essence, the organization must ask what benefits they get from “doing security.” The organizational perspective is essential to determining these benefits and for setting appropriate targets for security."

"A resilient approach transforms the basic premise of security - that of locking down an asset so that it is free from harm—to one that positions security as a contributor to strengthening the organization’s ability to adapt to new risk environments and accomplish its mission. Aiming to make the organization more sensing, agile, and prepared provides a clearer purpose, direction, and context for security management. Looking beyond security (to resiliency) may provide the change in perspective that organizations need to balance security and risk management with the organization’s strategic drivers."

Next steps
If you are a business owner or a leader within IT, you should read the abstract and ask yourself, "is our security strategy just another mechanism for IT to say NO to its customers or is it keeping us secure while still allowing the enterprise to meet its goals". If it's the former, I suggest that you start a conversation with the leaders within company and take a fresh look at security. After all, Insecurity should not be the driver for your security strategy.

Day 2 of the LZA Bootcamp proved to be another solid day packed with great information about SOA. Today we covered abstraction, SOA and legacy systems, data services, XML, and security.

Here are some of the key take aways:

Abstraction - The business doesn't care how the service is built or where it is located. All that matters is that it works. Business services abstract both data and application functionality.

Legacy - SOA impacts Legacy in three ways.

1. Migration - You can use SOA to migrate off of legacy by abstracting the interfaces and then gradually replacing the pieces on the backend.
2. Enablement - Expose legacy capabilities and data as services.
3. Rejuvenation - Leverage legacy as an active SOA participant. This is common with mainframe systems today. Many companies have fully functional mainframe systems that need to be connected with each other and presented with new rich interfaces.

Data Services - One of the biggest problems integrating systems is dealing with data issues across systems. abstracting data as a service allows you to hide the complexity of the physical implementation of data from the other layers. Here is an example of its use. Let's say you have four different sources of the same data entity that has four different data structures and leverages four different technologies. The data service layer hides all of this from business services. Then let's say you have a goal to reduce the data sources down to a single source. If your business services are calling a data service that represents the logical representation of the data, then you can eliminate the other three data sources without making a change to your business service. In other words, your business service does not care how or where your data is stored.

XML - XML allows for standardization and maximum flexibility, but it sacrifices performance. XML is a widely accepted standard and leveraging it gives you the ability to connect more easily with external services. However, since XML contains huge amounts of metadata and is Ascii, it can consume large amounts of network bandwidth. This can be addressed with hardware and software solutions that compress and accelerate XML messages.

Security - SOA increases the risk of security breaches. In addition to the normal dangers (SQL injection, denial of service, buffer overflow, and trojan horse), XML adds XML injection, WSDL scanning, schema poisoning, and other threats. In the traditional distributed computing world, systems were closed and API's were proprietary. With SOA, we have open systems and distributed APIs. A much larger effort is required by architects to keep the "bad guys" out.

Day three focuses on governance and funding. I'll be back tomorrow for a quick recap on those topics. Here is the recap from day 1.

Many of the folks I know who have issues with Linux, lean on unproven myths and perceptions to prove their point. I prefer to back arguments with research and real data. In doing so I stumbled across a well-researched article that answers many questions about the Windows vs. Linux security debate. It is a rather long article so I'll highlight a few sentences out of it.

While discussing the myth, "Windows only suffers so many attacks because there are more Windows installations than Linux, therefore Linux would be just as vulnerable if it had as many installations", the article states:

...according to Netcraft, 47 of the top 50 web sites with the longest running uptime (times between reboots) run Apache. None of the top 50 web sites runs Windows or Microsoft IIS. So if it is true that malicious hackers attack the most numerous software platforms, that raises the question as to why hackers are so successful at breaking into the most popular desktop software and operating system, infect 300,000 IIS servers, but are unable to do similar damage to the most popular web server and its operating systems?

Malicious software is so rampant that the average time it takes for an unpatched Windows XP to be compromised after connecting it directly to the Internet is 16 minutes -- less time than it takes to download and install the patches that would help protect that PC.

While discussing the myth, "Open source is inherently dangerous because the code is readily available", the article states:

Windows Server 2003 has experienced the most severe security holes. Microsoft's own classification of the flaws shows that 38% of the patched programs are rated as Critical. If we apply the metrics outlined in the previous sections, we would have to raise that to between 40-50%.....In sharp contrast, of the 40 vulnerabilities listed by Red Hat, only 4 are rated as Critical by our metrics (Red Hat does not list a severity rank for its alerts). That means 10% of the most recent 40 updates are of Critical severity.

There is a lot more good information in this article so feel free to read more...


read more