Enterprise Initiatives

I have discussed the SOA Evangelist, the Architects, and the Developers. Today I will discuss the role of the Testers and the characteristics required to contribute to a successful SOA implementation.

One of the most important roles and one that I probably should have included in the Architects post is the Testing Architect. As I have written on CIO.com in my article called Six Questions to Consider Before Building a SOA Testing Team, SOA testing requires a much deeper knowledge of technical skills, including development skills. It might be unrealistic to expect your entire testing team to possess the desired technical skills required to successfully test SOA, but your test architect and your leads should be able to understand concepts like statefullness, distributed computing, and data services, to be able to properly validate the underlying architecture. They also need to be able to take developer test harnesses and update them with their own test scripts.

A successful SOA testing strategy starts with the test architect. This person must have a very in depth knowledge of SOA and should work closely with the EA team. I actually recommend that this person is a member of the EA team, but every business and culture is different. The goal of the test architect should be to set up a framework and a core group of policies and procedures (part of IT Governance) so that the rest of the test team has the tools and the guidance to successfully test SOA. Without an established testing architecture, the company will have to rely heavily on expert knowledge from the entire testing team. I have seen three scenarios through my personal experiences and through my research.


Scenario 1: Underestimating SOA
The first scenario is out right failure caused by not having the tools and knowledge required. This is caused by a company not realizing that their current methodology and internal personnel are not well equipped for testing SOA. These companies do not invest enough in tools, training, and governance and usually can only test the presentation layer and possibly the interfaces. By not understanding the concepts of SOA, they are unable to validate the architecture which leads to poor performing and fragile services.

Scenario 2: Paying through the nose
The second scenario I have seen is relying too heavily on "expert" consulting firms for testing. In this scenario the company bets the farm on an experienced SOA consulting firm and pay rates that far exceed $100/hr. This model cannot be sustained for any length of time unless the company is willing to burn huge amounts of capital (which is not a popular thing to do these days).

Scenario 3: Good balance of internal/external expertise
A more desirable scenario is to train or hire a SOA testing architect, build a solid testing platform tailored for the needs of the organization, and govern the testing process while training the other members of the team. Companies should hire one or more experienced SOA tester, find an experienced consultant or two, or train an experienced and credible internal candidate to take the lead for creating the testing architecture. At the same time the testing experts are charged with transferring knowledge over to the rest of the internal team members. This is critical because a highly experienced SOA tester is in great demand and is a flight risk since they are a rare breed. It is critical to grow the knowledge base internally.

Testing needs
The needs can be broken down into these categories: People, Tools, Governance. So what are the characteristics of a successful SOA tester. The answer is dependent on the architecture that is implemented, which is related to the tools and policies that are put in place. Below is a diagram I often use to discuss the typical layers of an architecture.

From SOA Slides

As I mentioned in the previous discussion about the developers, I see the need and desire to specialize within the layers. The same holds true for testers. Work within the layers happens simultaneously in development. I recommend an iterative development and testing approach which means there should be testers working within the layers simultaneously as well.

Here is what I would strive to put in place, keeping in mind that these are roles and some people may fill multiple roles:

SOA Test Architect
Courageous leader with extensive working knowledge of SOA, distributed computing, integration testing experience, coding/scripting experience, and understands the business. It is likely that you may have to go outside to hire this person or bring in a consultant to assist your top level tester.

SOA testing leads
This person or persons must understand all layers of the architecture (let's not forget security either) and should be able to design test plans that validate both the architecture and the governance model. They should understand how to perform black, white, and gray box tests. Testing abstracted services requires extensive testing in the areas of security, performance, and regression. Throw in the versioning capabilities of services and the fact that service consumers can use services in ways that were not anticipated and the permutations of test cases start skyrocketing. The test leads need to practice risk based testing and balance risks, timelines, and costs. There is just so much more at stake here than in the traditional n-tier architectures.

Business process testers
The business processes should be modeled within some tool and will likely call one or many services. The process flow requires a series of decision points based on variables and constants and can trigger events such as notifications, alerts, other processes, error handling routines, services, and a variety of other possibilities. The testers need to validate the business process as a stand alone entity. For example, if the business process is "validate credit card", the tester needs to ensure that this process handles the inputs correctly, properly runs the validation process, and generates the appropriate output. At this point, the tester need not be concerned with any other processes or services. These testers must work closely with the business and/or business analysts and do not need the breadth of technical knowledge that the leads and architects must have (although it would help). They should be approaching the testing from a business standpoint.

Integration testers
These testers must be much more technical and understand how to work with XML, SOAP, WSDLs, networking & telecommunication concepts, statefullness, and various platforms and technologies (Java vs .Net, Windows vs Unix/Linux vs. mainframe, etc.).

User Interface testers
The company most likely already has an abundance of people who can test the UI. If your company is leveraging mashups, wireless devices, or consumer facing UIs, the complexity of the testing will be greatly increased. These testers may be need to become familiar with AJAX, RIA, portal technology, RSS, and a variety of social software.

Data services testers
These testers must understand concepts of data modeling, database CRUD operations, transformations, security and roles, authentication, and much more. Everything starts with the data and if errors are introduced in this layer, everything else is doomed to fail. You must have a very strong testing lead working in the data services layer since data is the foundation of any successful implementation.

Other areas of focus
The name of the game is speed to market and test automation is a critical component for making that a reality. Performance testing is extremely critical and organizations should practice simulations to try to anticipate future performance of all services. Validating the security model and the governance model should also be part of the overall test strategy. Whoever is responsible for designing the security test plan should read (and fully understand) this book.

Closing comments
I am by no means a testing expert (but I did stay at a Holiday Inn Express last night). I do read a number of testing blogs listed below:

• Randy Rice's Software Testing & Quality Blog
• Frank Cohen's blog
• The ITKO Lisa Soapbox: SOA testing, validation & virtualization
• SOA Testing
• Mike Kelly's testing blog

Make sure your organization funds the necessary training and tools for testing. From my own personal lessons learned, make sure you invest as much effort in testing and security up front as you do in architecture and development. On my project a few years ago, I wore my developer hat too much in the research phase and did not properly budget for what was necessary for testing. Do not make this same mistake. As hard as it is to sell the business on SOA, it is much harder to go back for more money, especially before any business value is realized!

I have discussed the role of the SOA Evangelist and the importance of the architects. Today I will discuss the different developer roles and the characteristics that make a successful SOA developer.

In a traditional 3-tier architecture it was common that there was a presentation layer, a middle tier or application layer, and a data layer. In some cases, a single developer role was responsible for all three layers. In larger companies, there may have been specialized roles such as a UI developer, an application developer, and a database developer. In an SOA, the concept of an application is no longer relevant except from the standpoint of integrating to existing applications. With SOA, we should be building business services that are application independent. The following chart is one I have used in previous posts which shows a typical breakdown of some of the development roles required for SOA.

From SOA Slides

So let's talk about business services. A business service is the compilation of all of the work done by the collection of developers within each layer. For example, let's discuss a "Shopping Cart" business service like the one we use on Amazon.com. It is most likely made up of services and/or components that live within each of the various layers of the architecture. The presentation layer obviously contains the physical implementation which the user sees and interacts within the browser. The business process layer contains the logical flow of steps that walk the user through the checkout process from start to finish. The business rule layer contains rules about tax codes, discounts, memberships, etc. and the underlying data elements and structures are handled in the data layer. In many instances, companies have multiple data structures that serve similar functions due to mergers, acquisitions, years of legacy systems, 3rd party application purchases, and many other reasons. To abstract these different data structures and present a common data interface that hides the complexity of the physical implementation of the data, the data services layer exists (think master data management).

So to develop this shopping cart business service, all of the people working within the different layers of the architecture must collaborate to deliver the service in a way that it both meets the business requirements and the technical requirements as defined by the SOA Governance model that the company has adopted. Examples of the technical requirements might be...

• Adheres to WS-* security standards
• Data encryption policies
• Platform neutral implementation
• Meets specific performance requirements

Why do I bring all of this up? Because developers that will succeed developing in a service-oriented architecture need to have the following characteristics:

• Flexible, open to change
• Collaborative
• Able to work in a setting with peers reviewing their work
• Able to see the big picture
  
• Not married to a certain technology
• Able to withstand constructive criticism
• Innovative

Developers who are protective of their own work or of their technology of choice will likely struggle working on SOA. One of the goals of SOA is to build platform neutral software that is highly flexible, maintainable, and loosely coupled. To build software like this we must shift away from software developing and move to software engineering. In simple terms, we must move from drag and dropping to planning and modeling. We must embrace collaboration, peer reviews, and governance. If your developers don't like these requirements, they either need to change or leave. Otherwise they will be a cancer and will continously feed the forces of resistance.

Ok, off my soap box now. So let's discuss the roles. But before we do let me note that we are talking about roles and not individuals. In smaller companies, a developer may perform multiple roles. In larger companies we may actually see very specialized developers living within a single layer of the architecture. One last disclaimer: I am assuming that an architecture team exists and some form of standards and governance is in place within each layer.

UI developers
If your company can afford to specialize, this is the perfect layer for it. Developers in this layer do not necessarily have to be extremely technical. It is more important that they understand usability, UI standards, and web interface best practices. These people should be comfortable working in prototype mode where they can start with mockups and work closely with the business or business analyst to reiterate through the various visuals until an agreed upon format is approved. These developers must be able to talk to the business in business terms and understand how typical business users interact with web technology.

Business Process developers
There are two distinct roles in this business process layer. One role deals with modeling business processes, the other role deals with integrating business processes with the underlying services and systems. In some companies this role may be served by one person but more likely it will be two because of the difference in the skills required. The business process modeler may not even be an IT person. Some companies have a department that exists within the business that focuses on business process improvement and business process automation. This is the norm for companies that practice Six Sigma or Total Quality Management (TQM).

The business process integrator is a technical role which requires expert experience working with Web services, REST, JMS queues or several other technical areas of expertise. The integrator is the person who connects business processes to the backend technologies which controls the overall flow of the business services and overall composite business application, often coined orchestration.

Business Rules developer
This one is a little fuzzier. Not all architectures have a distinct business rule layer. In some instances, the business rules are controlled in the data layer. For companies who have very dynamic business rules, especially rules that are customer centric and could be updated by end users and even customers, it makes a great deal of sense to abstract the business rules. If a company has a business rule layer and implemented a tool to manage business rules, it is highly likely that you will need technicians who govern this layer much like DBAs govern the database layer. In some case this might be an additional role for a DBA but it does not have to be. Whoever it is they must understand the concepts of abstraction and be looking for ways to empower the end user to react quicker to changes in the business. For example, let's say that a loan approval process requires state specific business rules about what percentage of capital is required from a loan applicant up front before the loan application can be expected. The frequency which this value changes varies by state and must be updated as soon as possible. The best scenario is to take IT out of the process and enable certain users (or systems) with the appropriate authority to update the values as necessary. Whoever is responsible for the business rules must be able to work with the business and/or business analysts to understand the frequency of change, the rights and permissions, and the impacts of the various business rules. There is an amount of overhead associated with creating and managing rules and the person in charge must understand the trade offs.

Data Services developer
Maybe a better name for this person is an information architect. The person in charge of this function is responsible for abstracting the underlying data layer and exposing it the other layers of the architecture and even externally to other systems. For example, let's say that our shopping cart business service allows multiple forms of payment (American Express, Visa, and Pay Pal). In addition, the different products purchased (books, DVDs, clothing, etc.) are managed from different inventory systems. To hide that complexity and to allow us to add additional payment services and inventories with ease, we leverage data services to present the shopping cart service with a logical view of the data. In other words, we create a standard payment and inventory message that the shopping cart communicates with. As long as the shopping cart communicates with these standard messages, the underlying physical implementation of the different payment and inventory services are irrelevant. The shopping cart service simply communicates these messages in the standard format it knows and the translation from the standard message to the physical layout of the receiving systems or services is performed in the data services layer.

From SOA Slides

(This slide is from BEA)

Developers or architects within this layer must have exceptional technical knowledge in the area of data modeling and database design. If the company has a tool to manage this layer (which is highly recommended) then an administrator is also needed.

Database developers
This data layer is typically an area that already exists today within each enterprise. This is where the DBAs live. With SOA, the DBAs must work extremely close with the developers within the other layers of the architecture. They must understand the performance and security requirements of each layer and ensure that the underlying data model meets the needs. Since legacy integration is common in many SOA implementations, DBAs are often called upon to create views or structures or even establish new ETL processes to provide a common view of data to expose to the other layers of the architecture.

Security developers
The security experts may not actually do the development but there must be a person or group of people who understand the new security challenges that SOA introduces. SOA can introduce the following new threats:

• Expose components of legacy systems that were never designed to live outside the firewall
• Require trusts between multiple internal/external systems without additional sign-ons
• Send a single message to multiple partners each with their own distinct security requirements
• Expose credit card and other privacy related data out on the web

From SOA Slides

(This slide is borrowed from the following presentation from Mamoon Yunus of Forum Systems)

Todays n-tier security models are not enough when dealing with B2B type SOA implementations. The developers in this area must have a deep understanding of communication protocols, security best practices, web architecture, regulation (ie. HIPAA, SOX, PCI) and WS-* or other standards.

Summary
As you can see there are many distinct developer skill sets required to implement SOA properly. Trying to find a person that can master each layer is not realistic. If you have this person, it is likely that he or she is on your architecture team. The beauty of this architecture is the ability to divide and conquer the problem within each layer. A single business service can be worked on by numerous people within each layer. It requires solid governance, strong collaboration skills, and teamwork to work this way which is why it is critical that the right team of people is assembled to tackle SOA. My next post will discuss the testers and their relationship with the development team.

In part 1, I discussed the importance of the SOA Evangelist. I also talked about the shift from software development to software engineering. This shift has made the role of the architect critical for the success of any SOA initiative. First, let us talk about the various architect roles required to successfully design an SOA.

Chief Architect - This person, possibly your SOA Evangelist, should have strong technical skills, sound business knowledge, and great leadership skills. Not only should this person understand the ins and outs of SOA, but he/she should be able to explain the value of SOA to the business in business terms, to the CIO in high level technology and business terms, and to the technicians in detailed terms. Depending on the size of the company and the SOA initiative, this person may or may not actively participate in the design. Regardless, this person should be knowledgeable enough to participate in design sessions when called upon. From a leadership perspective, this person is likely the change agent who is the driving force behind establishing and implementing governance, shifting the IT mindset from programming to engineering, establishing the SOA roadmap, and other culture changing elements.

Enterprise Architect - In smaller or midsize companies, the chief architect and the enterprise architect may be one in the same. In bigger companies, there may be more then one EA. The EA has broad domain and business knowledge and works across organizational boundaries to ensure that the architecture meets both the business and IT requirements. Wikipedia says it best when it mentions....

The role of the Enterprise Architect is to take this knowledge and ensure that the business and IT are in alignment. The enterprise architect links the business mission, strategy, and processes of an organization to its IT strategy, and documents this using multiple architectural models or views that show how the current and future needs of an organization will be met in an efficient, sustainable, agile, and adaptable manner.

Domain Architect - The domain architects specialize in specific areas of technology. They are the experts of their domain. Here are a few examples of domains that require specialization:

• Application
• Information
• Security
• Infrastructure
• Business Processes
• Network
• Integration

Once again, the size of the company often dictates how many of these roles exist as a stand alone job function or if they are rolled up into a broader architect role. For example, smaller companies may combine application, information, and integration into one role and infrastructure and network into another role. The key for me is that these roles need to be accounted for regardless if its by one or many people. Each role is critical in successfully architecting a true SOA.

Solution Architect - The solutions architect has hands on experience with specific technologies and even business applications. You may have an architect who specializes in Java, .Net, backend systems, web applications, financial systems, ecommerce systems, distributed processing, etc. Once again, I am advocating that the role is addressed and whether or not a company has one or many people with this title depends on its size and budget.

The Architecture Team
As I described above, the team should consist of the following roles (not necessarily individuals):

• Chief Architect
• Enterprise Architect
• Domain Architects
• Solutions Architects

The number of people filling these roles vary from company to company. In my previous job, a shop of roughly 200 people, our architecture team consisted of myself as a Chief Architect, an EA, a domain architect, and a solutions architect (even though they all had the simple title of "Architect"). There was an Architect over the infrastructure and network that reported into a different structure and a one person security team. My architect team worked closely with the others but it would have been more effective had we all been on the same team with the same goals and objectives.

I have seen a case study at an enterprise architecture conference a few years back where a global SOA initiative for a fortune 100 company had over 100 architects on the team including numerous EAs and even multiple Chief Architects from different business divisions. There is no one way to slice this pie.

Architect Mindset
You can organize the teams anyway you like but one thing you must have is the proper mindset. The architecture team must be focused first on the business drivers that the SOA initiative has set out to solve. If the team loses sight of these drivers then SOA will become an IT project instead of an important initiative for the entire corporation. You must have the business buy-in throughout the project to reap the true benefits of SOA. Please read 8 Winning Characteristics of Successful SOA Implementations which I wrote for CIO.com that summarizes the critical success factors that wer common among the winners of the SOA Consortium/CIO.com SOA Case Study contest earlier this month.

Second, the team needs to ensure that they are not an Ivory Tower architecture team. SOA is too important for architects to be preaching philosophy from the top of their perches. The architect team must be the change agents required to move the organization away from the traditional silo driven application development approach and into a more collaborative engineering approach that focuses on the long term vision not the short term and short sighted approach that many companies practice today.

Third, this team must have an open mind. They need to evaluate current business processes, current IT practices, and current systems, and provide the necessary change and direction required to move the organization forward towards the vision established by the business and IT and reflected in the SOA roadmap. Without changing the way IT approaches development, it is likely that the team won't be able to differentiate between SOA and web services. Education and evangelism is key to understanding the proper way to architect and govern your SOA.

In part three I will discuss the developer role. In part four I will discuss the testers.

I am currently enrolled in an executive MBA program and my current class is about Leadership. As part of our group project we are working on we took the Jung Typology Test (personality test) to get our Meyer's Briggs indicators. After answering the questions on the test you get graded in four areas:

1. Extroversion vs Introversion
2. Sensing vs. iNtuition
3. Thinking vs. Feeling
4. Judging vs. Perceiving

I am a ENTJ which means I scored higher for the following (Extroversion, iNtuition, Thinking, Judging). These four letter codes then map to certain personality types which fall into four distinct temperaments:

1. The Guardians - 40-45% - administrators and conservators
   
2. The Idealists - 8-10% - mentors and advocates
   
3. The Artisans - 35-40% - entertainers and operators
   
4. The Rationals - 5-7% - Engineers and coordinators
   

Within each temperament there are four types. The four types of Rationals are:

1. Architects - Their major interest is in figuring out structure, build, configuration -- the spatiality of things.
2. Field Marshals - organize their units into smooth-functioning systems, planning in advance, keeping both short-term and long-range objectives well in mind.
3. Inventors - have an eye out for a better way, always on the lookout for new projects, new activities, new procedures.
4. Masterminds - natural brainstormers, always open to new concepts and, in fact, aggressively seeking them.

As I read through the information, I found it interesting that only 5%-7% of the world falls into the Rational personality type. And within the Rationals, only the Field Marshals prefer to manage. These are the Chief Architects which represent only 2% of the population. Masterminds tend to stay in the background and only insert themselves into leadership roles when they see others fail. Masterminds represent only 1% of the population. If you combine the Field Marshals and Masterminds you only have 3% of the population that represent the characteristics of a Chief Architect. Enterprise architects fall into the inventors and architects types which make up only about 3-5% of the population.

To make a long story short, it takes a special breed of people to fill the role of an architect. And many of these special people have no desire to take on the leadership tasks that make up a Chief Architect. Why do I bring this up? Well, so many companies are looking at investing in SOA. SOA requires an enterprise architecture which requires architects. There is already a shortage of "real" architects and SOA is going to make those resources in more demand. That's good news for architects (demand) but bad news for companies (supply). Companies that can't distinguish between architects and architect wannabees are going to be in for a rude awakening. If you want to know if your candidate is a real architect or not, read this article from Zapthink and test your candidate on his/her understanding of what SOA really is.

Take the test and see what your personality is. I am a Field Marshal.

Zapthink posted an interesting description of a role they call VP of SOA. Todd Biske argued that many of the tasks in this role belong to the Chief Architect and a dedicated role of VP of SOA is unjustified. My view is somewhere in between. As someone trying to fulfill both the role of the Chief Architect and the role Zapthink describes as the VP of SOA, I can honestly say that this can be a lot to juggle. I have fallen victim to becoming so engaged in the VP of SOA role that at times I am not totally fulfilling my duties as the Chief Architect. After all, the Chief Architect is responsible for the architecture of the entire enterprise, which SOA is a subset of. My solution is not to create another high paying role like the VP of SOA but instead give my enterprise architects more responsibility and decision making power to free me up enough to take on SOA and EA. This creates a cascading effect where the tech leads then need to step up to the plate and take on some of the enterprise architects' tasks. This creates opportunities for many people as opposed to one highly paid guru and allows you to build a stronger team over time.

I work in a 200 person IT shop so creating VP positions is harder to come by then if I worked in a 1200 person IT shop. In a larger company, especially one with a distributed IT staff, I can definitely see the justification for a VP of SOA. In a centralized IT shop like the one I work in, I think the VP of SOA and the Chief Architect should be the same person. Your thoughts?

There have been many blogs that argue whether SOA is an IT only initiative or a business initiative. Regardless of who or what is driving SOA, one way to maximize your investment is to align SOA with your company's project portfolio (hopefully you have one).

Let's think about that last statement for a second....

...align SOA with your company's project portfolio.

The value that Project Portfolio Management (PPM) brings to the table is that if done right, PPM ensures that the company focuses its assets (people, products & services, hardware & software, etc.) on initiatives that align with company strategies. Now if you have everyone working on the right projects, the next logical step is to focus your architecture on the projects that bring the greatest value. If the PMO (Project Management Office) does its job and helps the business manage its project portfolio(s), then the architecture team can align with the business to maximize the value of SOA.

I recommend that a company has more then one portfolio. I will use a fictitious company called Acme Toys to demonstrate why.

Acme Toys is a 30 year old toy manufacturer who has a new CEO. The CEO is charged with doubling revenue in the next five years and removing waste from the operational aspects of the business. The IT team has recently implemented SOA and BPM and is starting to reach a modest level of SOA maturity. They have an effective governance model in place that they are constantly refining and have a few high profile projects under their belt. The next step for IT is to figure out where to leverage SOA next.

The CEO gathered his leadership team and put forth a strategy to meet his goals of doubling revenue and improving processes. The CIO and the Head of the PMO proposed creating four separate portfolios to be funded and staffed separately. The picture below illustrates the recommended portfolios.

The growth portfolio is where the company manages the projects that enable the company to double its revenue over the next five years. Within each portfolio there are programs. Programs can be defined as a collection of projects. In other words, a program is a strategy that maps to a goal, while a project is a tactic for delivering the strategy. Here is an example of a Growth Portfolio:
You can see from this example that B2C (Business to Consumer) is a program or strategy. Underneath that program is a list of projects that help deliver the strategy.

The next portfolio is the Optimize Portfolio. In this portfolio Acme Toys focuses its resources on operational efficiencies. Below is an example of what this portfolio might look like.

Acme Toys still needs to keep the lights on so the PMO recommended an Infrastructure Portfolio where IT can drive and manage upgrades, cost reductions, and modernization of its IT services and assets, as well as, deliver compliance or regulatory type projects like SOX or PCI Compliance.
And finally, Acme Toys needs to drive innovation in new products. They have a separate Research and Development team that prototypes various new potential products. This team's charter is to prove out the viability of new product ideas before the company makes huge investments in manufacturing and IT to productionize the product.

That rounds out the four portfolios. Now the IT team has clear direction on where the company is going over the next 12-24 months and can start looking at aligning SOA with the various portfolios. For the sake of this example, let's say that the architect team has enough bandwidth to contribute to two of the four portfolios. It is likely that IT would choose to align with the CEO's goals of growth and optimization. This allows IT to focus its efforts on the programs within those portfolios. The architecture team assigns a group of solutions architects to the Growth Portfolio. They now have a roadmap of projects to start identifying candidate services. With this 12-24 month roadmap of projects provided by the PMO, the architects can then prioritize the candidate services based on business value and potential reuse. By front loading highly reusable candidate services, the architect team can reduce the amount of new development required for future projects. Here is an example.

After analyzing the list of projects for the next 12-24 months, the architect team built a SOA roadmap. They identified two candidate services that would be used in every single project. The first candidate is a product lookup service and the second candidate service pertains to authentication. These two services will be prioritized high and addressed in the first project so the work can be reused in all of the subsequent projects.

The architecture team staffs the Optimize Portfolio with another group of solutions architects. They follow the same process for identifying candidate services for the project roadmap within their portfolio. The enterprise architects provide oversight for both of these portfolios to enforce governance and to take on building or modifying any enterprise wide services that are identified within the projects. The solutions architects work on the application or process specific services.

To wrap up this long post, I hope I have made it clear how Project Portfolio Management can be a great asset to the architecture team in their quest to further entrench SOA into the enterprise. PPM provides a clear vision into corporate strategy which the architecture team can leverage to maximize the value of SOA. I would love to hear your thoughts on PPM and SOA.

We are in the middle of implementing our SOA Governance model. We have put together a ton of lengthy reference documents like the architect guidebook, SOA roadmap, developer guidebook, administrator handbook, and many others. Our biggest fear has been getting people to read these documents. In order to enforce our SOA best practices and standards, people must be aware of what they are. Giving people 200-300 page documents is not an effective way of sharing information.

Enter open source wiki tool, MediaWiki. With a little help from OpenOffice v.2.3, we took the lengthy Word documents and converted them to Wiki Markup to create easy to use, clickable, and searchable documents. Now our architects and developers can easily maneuver through pages of pertinent documentation without being overwhelmed by large sequential documents. In addition, people can collaborate by using the discussion feature in MediaWiki to help the architect team improve upon the documentation.

Our next step is to implement an open source blogging tool. One of our challenges today is to communicate both technical and project specific information pertaining to our SOA initiative in a timely manner. I have held a number of meetings with several IT teams to walk them though various presentations and white board sessions. Unfortunately I don't have the bandwidth to do this often enough. We will start to leverage blogging technology to share daily information about the initiatives. People who are interested will be able to subscribe to these blogs using an RSS feed reader. This is much more effective then pushing more email on people.

And finally, we will create an Enterprise Architecture community in our enterprise portal. This will allow us to bring all of our content together in a one stop shop approach. The community will display our blogs, our wiki, project status, tips & tricks, announcements, and various other information.

One of my lessons learned so far on our SOA project is that we do not communicate often enough. I believe a combination of monthly presentations and white board sessions, along with our wiki, blogs, and portal community provides multiple opportunities for people to keep up to date with our SOA initiative. I also think it is more effective to publish content and let people "pull" it on demand then to "push" it on them.

I just returned from a great vacation and saw that James McGovern had a few questions for me.


Question #1 (after reading Real World Open Source Solutions)

Mike Kavis believes that others should help spread the word that open source isn't just about Linux. Maybe Mike could talk about where folks get their information, such as all those advertising dollars paid by barely open source vendors and their outsourced PR departments otherwise known as industry analysts.

Answer

Getting information for open source software is not much different then getting information from closed source software. Yes, there is less conferences and less marketing/advertising, but those two sources should not be where you find all of your information. Leverage your network, read blogs, search the web for case studies where other companies have leveraged open source software to address specific areas like CRM, project management, database technology, portals, etc. I like to go to sourceforge.net and other popular sites and look at the number of downloads and the size of the community. Those two numbers can clue you in on which products to start researching in greater depth. I also use my Google Reader to follow articles on these sites: OpenSourceCommunity.org, OpenSources, OSS-Biz, NotJustLinux, OpenSourceUnleashed, and LinuxToday to name a few.

Question #2 (after reading Who's Killing SOA?)

I wonder if Mike Kavis understands the simple truth that most enterprise architects don't actually know other enterprise architects in other enterprises. Many of us are insular in nature?

Answer

I do understand and that is one of the main reasons why both James and I have called for more EA Collaboration. EA's need to "get out" more and see what the rest of the world is doing. Why reinvent the wheel from scratch? Throw some ideas out there and see where the conversation leads. Add your two cents to other bloggers' opinions and contribute to the overall good of the community. If you disagree with a viewpoint explain why. Other EA's can read both view points and decide for themselves which approach is best or maybe a combination of the two makes more sense. I know that EA's don't actually know other EAs in other enterprises because I knew none before I started blogging back in March of this year. Now I have a great network comprised of talented individuals who share their thoughts on important topics.

I have been thinking about writing this post since I wrote this article about Second Life. As I was getting my daily updates from my Google Reader I came across Nick Malik's article called

Using Massive Multiplayer Online Concepts to Build a Shared Architecture

In this post Nick writes:

What I haven't seen yet (and perhaps it is the nature of the child-like games my kids play) is a MMO game where every person plays a role to build something instead of defeating something. It is easy to tear something down. Divide and Conquer. Building something up is much harder.

Welcome to Second Life. If you still are not up to speed on how Second Life is being used in the business world you must read this article called Alternate Universe. As I continue to follow the success of Second Life I often wonder why is it that Linden Labs can create such a successful architecture that runs a virtual world with over 8MM users world wide while most companies struggle to create an architecture to support the real world?

So I did a little homework and as I suspected, Linden Labs leverages open source technologies like MySQL, Apache, Squid, Mono, uBrowser (for online help) and Debian as its choice of operating systems to run its grid technology. Linden Labs provides developers with APIs that are simple REST style web services. They have their own proprietary XML format (called LLSD) that supports Perl, PHP, Python, and Ruby. They have a very robust portal and wiki which uses Mediwiki, another open source product.

The beauty of this architecture though is not the open source technologies that they have chosen but instead is the use of open standards which allows them to create a platform for others to build from. People can contribute to this platform as C++ developers, they can use Second Life's scripting language, they can leverage a host of third party tools and services, or they can simply use the tools provided by the user interface. This is a very open environment, much like many of the successful open source projects that exist today.

Now back to the topic of architecting the real world. As architects within an enterprise, we should take the time to review the benefits of the Second Life architecture and apply it to our work. Here are some of the obvious benefits:

1. Open; supports multiple development platforms
2. Easy to use; simple interface & documentation is easy to find on the wiki
3. Scalable; scale up by adding more nodes to the grid
   
4. Cost effective; use of open source reduces high cost of software licensing
5. Enables users; users are free to innovate and are not restricted by the architecture
6. Extendable; other companies can leverage platform to drive more revenue inward

Isn't this what we should be delivering in the real world? Do you think they spend a lot of time arguing over process? Do you think they spend a lot of time gathering user requirements? Or do they have a vision and know how to anticipate user needs? Have they tried to handle every possible exception in the world or did they create a robust set of standards that do not restrict usability and functionality?

There are still many people who are unaware of what is happening in the virtual world and others who are not taking this movement seriously. We just may be looking at the future of business where individuals and businesses do most of their commerce in virtual worlds. The architecture will definitely support it. Whether virtual worlds are the future or not, we should at least study the underlying architecture and the interactions that take place in their open development community.

I posted my opinion on CIO.com (The Role of the Enterprise Architect). Here is a summary...

The EA should know how to code and should have spent many years developing, designing, and managing technical projects. So when I say that the EA should not be coding, that does not mean the EA doesn't know how to code.

With that said, the EA's focus should be on strategy and vision. The architects below the EA should turn the strategy and visions into reality.

I have read many blogs that argue both sides of this question. James McGovern's article states:

You probably have ran across architects that don't code. Hopefully you aren't one of them...

I am hoping that he truly means architects and not Enterprise Architects (EA).

Colin White has a good article called Architects don’t code, they whiteboard. I tend to agree with his stance on this topic.

Here is another article by Simon Brown who has a similar opinion.

On Bill Barr's blog, Agile Enterprise Architecture, Bill phrases it slightly different. He says that EAs should not write production code but...

When it comes time to talk to people about a new idea, get feedback and even sell the idea, that's when a page or two of some clearly written examples are worth a thousand powerpoint slides.

What do you think?

In a previous post (Another Benefit of SOA - Career Path) I talked about how SOA can create a career path that consists of specialized technical jobs.


We just launched the requirements and development stage of our BPM/SOA implementation. We partnered with my buddy Eric Roch and company to assist in the implementation. We brought in specialists for each layer of the architecture. The specialists include a Web Designer for the presentation layer, a Process Analyst for the business process layer, SOA architects, a project manager, and my own architects (Software, Data, Network, and Test) who know the business and the existing infrastructure.

The first day was for introductions, general admin and housekeeping tasks, and a kick off. Today was day two. The first draft of the UI and the business process model was completed today! This was accomplished because we specialized or to put another way, we had experts assigned to specific technologies. My architects are incredible talents but they are not experts at UI development or business process modeling. Had I tried to tackle this internally it would probably have taken us 2 weeks instead of 2 days.

Another reason for the success is that the resources are 100% dedicated to the project and live in a "war room" where they have access to all of the people they need. We bring users and technicians into the war room when needed and never have to worry about meeting schedules. We will also follow an iterative approach and deliver weekly builds so the users can touch and feel the application early and often. This allows us to make changes on the fly and keep the users involved throughout the life cycle.

All that in the first 2 days! I will recap our progress each week and share the good, bad, and ugly as we encounter it.

I have read and posted several articles about the many business and IT benefits of SOA. I haven't seen much talk about the benefits it creates for the individuals within IT. This article focuses on the potential career path opportunities that a Service Oriented Architecture can create.

Many IT shops that do not have a well defined architecture have a limited career path for those folks who want to stay technical and not go into management. There are typically a few levels of development (programmer, programmer/analyst, sr. programmer, etc.) followed by an architect role. There is only room for a few architects so most people with over 10 years of experience get stuck in the sr. programmer role for long periods of time. People stuck in this role tend to do one of the following:

1) Go into management soley for the purpose of getting promoted. Many of them return to development, leave, or do not perform at previous levels.

2) Leave due to lack of opportunity.

3) Become miserable and unmotivated

Enter SOA. Companies moving towards SOA may develop a layered architecture like the one in the following picture:

Within each layer you can see specialized roles and job responsibilities. Some of the roles (DBA & Configuration Management) will likely already exist in your organization. Some of the others may be new or the job responsibilities may change dramatically. The point here is that for those organizations with many talented IT professionals who are stuck at a dead end technical career path, SOA might be the answer to your prayers.

Some of these roles (Process Analyst and Enterprise Architect) are best suited for technical folks with great communication skills. These people will interact often with the business. For those who would rather not be exposed to the business, there are many other roles for you (Configuration Management, Object Librarian, Information Architect, etc.). Your more junior developers or people who prefer web development can live in the presentation layer. They will be extremely productive since most of the business rules and services will be available for them to assemble into the UI's. The folks who just love to bang out code can live in the business rule layer where they can develop components and services. The more seasoned developers will build enterprise components and services (i.e. encryption, authentication, error handling, etc.) while the others can focus on application specific components and services (i.e. create order, print invoice, etc.) .

Most IT shops will have pockets of people who don't like change. They may not be interested in these new roles. Those folks can focus on maintaining the legacy systems.

In summary, SOA has many benefits. For IT professionals, many new roles will be created and your company's IT career path will most likely be extended. Over the next few years, more companies will be moving towards SOA and the demand for SOA professionals will be great. So embrace SOA if your company is implementing it. SOA will create many opportunities for those who take advantage of it.

Nick Carr created one of the longest running questions I have ever seen when he wrote his book Does IT Matter back in 2004. There has been so much discussion arguing for both sides of the answer including this article from Harvard's Andrew McAfee. I believe that a key differentiator for a company is the business processes. For IT to Matter, IT must create an architecture that supports the business's need to dynamically change and customize their processes. This allows the business to be more self sufficient and agile, which leads to a better user experience and faster time to market.

This is why the BPMS and SOA vendors are cashing in right now. Many IT executives are starting to understand that IT needs to be more of a partner to the business then a cost of doing business. BPMS solutions allow IT to provide tools for the business to rapidly deploy new systems that can have a huge impact on the bottom line by:

• reducing costs through process reengineering
• identifying bottlenecks and providing what-if analytics for ongoing process improvement
• providing robust, AJAX & web enabled user interfaces
  
• providing visibility into performance metrics with built in business intelligence tools
• providing self service capabilities for the end users, thus reducing the dependency on IT resources

The smart IT executives are leveraging SOA to allow the BPMS tools to talk to the existing legacy systems by providing a service layer that acts as a bridge between the user interface and the backend systems. This allows IT to rapidly deploy new systems without having to throw away the huge investments made over the years on their existing systems.

So to make a long story short. Does IT Matter? It can matter if IT recognizes that "Process is King" and that the business and IT need to work hand in hand to provide solutions that create a competitive advantage.

I spent the last two days in Weston, FL. attending various BPM and SOA related lectures. One session I enjoyed was George Paras's "Connecting the Dots: Establishing THE Enterprise Perspective". George is the editor-in-chief for the Architecture and Governance Magazine.

George reminded us that as architects we should think, act, and behave as business people. Everything we do as architects should be geared around value creation by affecting change in the name of product innovation, process transformation, technology transformation, and business transformation. In other words, we need to enable the business to be better, faster, more efficient, better informed, etc.

He then went on to point out that one of the reasons that it is so hard to create business value these days is because our technology and our business processes have become so complex. He showed us one slide that states, "Complexity INHIBITS Change....Complexity consequently INHIBITS Business Value". In my opinion, as we begin to model both our operational and technical architectures, we must make it a goal to minimize the complexity and subscribe to the KISS (Keep It Simple Stupid!) methodology.

The last golden nugget that I took away from George's presentation was about maintaining balance. George talked about how companies struggle to make progress towards establishing an enterprise architecture because the business demands so many tactical projects yielding only short term gains. The chief architect or architecture group must figure out how to get the company to start thinking strategically which yields long term gains. But be careful, you need to strike a balance between tactical and strategic. The business cannot stop while you go off for 6-12 months to create an enterprise architecture.

George's answer to the "Balancing Act" is the Managed Portfolio. The Managed Portfolio combines :

• Enterprise Strategy & Planning
• Project Portfolio Management
• Enterprise Architecture

The leaders of these three groups need to be in sync and must have shared goals geared around value creation.

During a break, I had a chance to speak to George (us Greeks need to stick together). I discussed my project and our approach to implement SOA only for the services required to support our BPM initiative, as opposed to creating a full blown SOA implementation. He agreed that we were on the right path which will make me sleep a little better tonight. I then handed him my last business card which was crumpled and stained with spaghetti sauce. Talk about great first impressions!

There were many other meaningful lectures at the show, but this was the best one to blog about.

If you are one of the many enterprise architects who have great visions of what SOA can do for your organization, but can't find anyone in the organization to provide the financial support needed to bring SOA to realization, listen to this real life story of how you can get the business to go to bat for you.

A medium sized company, who will remain unnamed, was supporting a sophisticated, proprietary supply chain system that was taking an army of developers to keep it running. The business was changing rapidly and the demands for new features and enhancements were far exceeding development's ability to meet the customers' needs. The system was several years old and needed to be integrated with several other silo legacy systems. The UI was not user friendly and was in dire need of workflow capabilities. The business wanted something to change but did not want to sign up for another long painful rewrite. So IT turned this liability into a win-win proposition for both the business and IT. Here's how.

First, IT put together a cost/benefit analysis for only the IT side of the equation. There was a huge opportunity to reduce the amount of resources required to maintain the legacy system while providing business value by improving speed to market, increasing new development capacity, and improving quality. When presenting their findings, the IT group pointed out that they thought there were even more substantial opportunities on the business side of the equation if the business would evaluate their existing, 20 year old processes.

After convincing key executives in both the business and IT that a large opportunity existed, the business funded the first phase of this initiative and brought in an outside firm to perform a business process analysis. In just 90 days, the current state and future state business processes were modeled and a few compelling opportunities were discovered. What the business discovered after interviewing numerous people throughout the organization, including several customers, was that their existing processes where not only costing more then twice of what the future state would cost, but they were potentially limiting the company's ability to create more sales.

At the end of this phase, the major IT and business stakeholders held a future state design meeting. In this meeting, the business discussed specific use cases that they needed in the future state that would allow them to reduce costs and increase revenue. For each use case, IT would describe the technology required to satisfy the use case. The use cases contained items like wizard based data entry forms, visibility into orders, B2B web interface, operational reporting, and integration with several legacy systems. IT was able to map those use cases to a BPMS/SOA solution to create a robust, user friendly, workflow system that could be deployed rapidly without rebuilding years of legacy systems. What the business heard was BPMS and SOA would answer their prayers.

The next step was to sell the CEO and his leadership team on how the company could be transformed by reevaluating its business processes and applying BPMS and SOA to help the business achieve its goals. It was the business and IT, hand in hand, selling this idea to the executives. The sell was easy and the company has moved on to the next phase with sufficient funding and executive level support.

I know this was a long story but here is the point. The IT team knew for 3 years that a BPMS and SOA solution was the answer to the problems that they faced each day. But IT had not figured out how to sell it and how to fund it. Finally, the pain became so great that IT realized that they had to figure out a way to sell these concepts to the business. First they showed that the cost/benefit just in IT was substantial. This got people's attention. Once they got people's attention, they were able to engage the business and gave the business ownership of transforming the company. Once the business had bought in, the rest was history. So what is the moral of the story? Aligning technology with real business cases increases your ability to sell technology to those who write the checks.