Enterprise Initiatives

I have been blogging for quite some time about SOA and Event Processing and have recently been getting more experience with Cloud Computing. The last few weeks I did a series of posts on agile SOA:

• Agile SOA: Empower the Business with Business Rules Engines
  
• Agile SOA: The Flexibility of Business Processes
  
• Agile SOA: Leveraging Data Services
  
• Enterprise Mashups: The Icing on your SOA
  

Today I completed a nice presentation on Slideshare called Agile Architecture which combines SOA, EDA, and Cloud Computing in a strategy to support the ever demanding needs of our dynamic business environment. I have the luxury of starting from scratch because we are in startup mode and do not have the burden of years of legacy systems and legacy cultures. The goals of our architecture are many, but here are a few key goals that speak to the topic of agile:

I have discussed the role of the SOA Evangelist and the importance of the architects. Today I will discuss the different developer roles and the characteristics that make a successful SOA developer.

In a traditional 3-tier architecture it was common that there was a presentation layer, a middle tier or application layer, and a data layer. In some cases, a single developer role was responsible for all three layers. In larger companies, there may have been specialized roles such as a UI developer, an application developer, and a database developer. In an SOA, the concept of an application is no longer relevant except from the standpoint of integrating to existing applications. With SOA, we should be building business services that are application independent. The following chart is one I have used in previous posts which shows a typical breakdown of some of the development roles required for SOA.

From SOA Slides

So let's talk about business services. A business service is the compilation of all of the work done by the collection of developers within each layer. For example, let's discuss a "Shopping Cart" business service like the one we use on Amazon.com. It is most likely made up of services and/or components that live within each of the various layers of the architecture. The presentation layer obviously contains the physical implementation which the user sees and interacts within the browser. The business process layer contains the logical flow of steps that walk the user through the checkout process from start to finish. The business rule layer contains rules about tax codes, discounts, memberships, etc. and the underlying data elements and structures are handled in the data layer. In many instances, companies have multiple data structures that serve similar functions due to mergers, acquisitions, years of legacy systems, 3rd party application purchases, and many other reasons. To abstract these different data structures and present a common data interface that hides the complexity of the physical implementation of the data, the data services layer exists (think master data management).

So to develop this shopping cart business service, all of the people working within the different layers of the architecture must collaborate to deliver the service in a way that it both meets the business requirements and the technical requirements as defined by the SOA Governance model that the company has adopted. Examples of the technical requirements might be...

• Adheres to WS-* security standards
• Data encryption policies
• Platform neutral implementation
• Meets specific performance requirements

Why do I bring all of this up? Because developers that will succeed developing in a service-oriented architecture need to have the following characteristics:

• Flexible, open to change
• Collaborative
• Able to work in a setting with peers reviewing their work
• Able to see the big picture
  
• Not married to a certain technology
• Able to withstand constructive criticism
• Innovative

Developers who are protective of their own work or of their technology of choice will likely struggle working on SOA. One of the goals of SOA is to build platform neutral software that is highly flexible, maintainable, and loosely coupled. To build software like this we must shift away from software developing and move to software engineering. In simple terms, we must move from drag and dropping to planning and modeling. We must embrace collaboration, peer reviews, and governance. If your developers don't like these requirements, they either need to change or leave. Otherwise they will be a cancer and will continously feed the forces of resistance.

Ok, off my soap box now. So let's discuss the roles. But before we do let me note that we are talking about roles and not individuals. In smaller companies, a developer may perform multiple roles. In larger companies we may actually see very specialized developers living within a single layer of the architecture. One last disclaimer: I am assuming that an architecture team exists and some form of standards and governance is in place within each layer.

UI developers
If your company can afford to specialize, this is the perfect layer for it. Developers in this layer do not necessarily have to be extremely technical. It is more important that they understand usability, UI standards, and web interface best practices. These people should be comfortable working in prototype mode where they can start with mockups and work closely with the business or business analyst to reiterate through the various visuals until an agreed upon format is approved. These developers must be able to talk to the business in business terms and understand how typical business users interact with web technology.

Business Process developers
There are two distinct roles in this business process layer. One role deals with modeling business processes, the other role deals with integrating business processes with the underlying services and systems. In some companies this role may be served by one person but more likely it will be two because of the difference in the skills required. The business process modeler may not even be an IT person. Some companies have a department that exists within the business that focuses on business process improvement and business process automation. This is the norm for companies that practice Six Sigma or Total Quality Management (TQM).

The business process integrator is a technical role which requires expert experience working with Web services, REST, JMS queues or several other technical areas of expertise. The integrator is the person who connects business processes to the backend technologies which controls the overall flow of the business services and overall composite business application, often coined orchestration.

Business Rules developer
This one is a little fuzzier. Not all architectures have a distinct business rule layer. In some instances, the business rules are controlled in the data layer. For companies who have very dynamic business rules, especially rules that are customer centric and could be updated by end users and even customers, it makes a great deal of sense to abstract the business rules. If a company has a business rule layer and implemented a tool to manage business rules, it is highly likely that you will need technicians who govern this layer much like DBAs govern the database layer. In some case this might be an additional role for a DBA but it does not have to be. Whoever it is they must understand the concepts of abstraction and be looking for ways to empower the end user to react quicker to changes in the business. For example, let's say that a loan approval process requires state specific business rules about what percentage of capital is required from a loan applicant up front before the loan application can be expected. The frequency which this value changes varies by state and must be updated as soon as possible. The best scenario is to take IT out of the process and enable certain users (or systems) with the appropriate authority to update the values as necessary. Whoever is responsible for the business rules must be able to work with the business and/or business analysts to understand the frequency of change, the rights and permissions, and the impacts of the various business rules. There is an amount of overhead associated with creating and managing rules and the person in charge must understand the trade offs.

Data Services developer
Maybe a better name for this person is an information architect. The person in charge of this function is responsible for abstracting the underlying data layer and exposing it the other layers of the architecture and even externally to other systems. For example, let's say that our shopping cart business service allows multiple forms of payment (American Express, Visa, and Pay Pal). In addition, the different products purchased (books, DVDs, clothing, etc.) are managed from different inventory systems. To hide that complexity and to allow us to add additional payment services and inventories with ease, we leverage data services to present the shopping cart service with a logical view of the data. In other words, we create a standard payment and inventory message that the shopping cart communicates with. As long as the shopping cart communicates with these standard messages, the underlying physical implementation of the different payment and inventory services are irrelevant. The shopping cart service simply communicates these messages in the standard format it knows and the translation from the standard message to the physical layout of the receiving systems or services is performed in the data services layer.

From SOA Slides

(This slide is from BEA)

Developers or architects within this layer must have exceptional technical knowledge in the area of data modeling and database design. If the company has a tool to manage this layer (which is highly recommended) then an administrator is also needed.

Database developers
This data layer is typically an area that already exists today within each enterprise. This is where the DBAs live. With SOA, the DBAs must work extremely close with the developers within the other layers of the architecture. They must understand the performance and security requirements of each layer and ensure that the underlying data model meets the needs. Since legacy integration is common in many SOA implementations, DBAs are often called upon to create views or structures or even establish new ETL processes to provide a common view of data to expose to the other layers of the architecture.

Security developers
The security experts may not actually do the development but there must be a person or group of people who understand the new security challenges that SOA introduces. SOA can introduce the following new threats:

• Expose components of legacy systems that were never designed to live outside the firewall
• Require trusts between multiple internal/external systems without additional sign-ons
• Send a single message to multiple partners each with their own distinct security requirements
• Expose credit card and other privacy related data out on the web

From SOA Slides

(This slide is borrowed from the following presentation from Mamoon Yunus of Forum Systems)

Todays n-tier security models are not enough when dealing with B2B type SOA implementations. The developers in this area must have a deep understanding of communication protocols, security best practices, web architecture, regulation (ie. HIPAA, SOX, PCI) and WS-* or other standards.

Summary
As you can see there are many distinct developer skill sets required to implement SOA properly. Trying to find a person that can master each layer is not realistic. If you have this person, it is likely that he or she is on your architecture team. The beauty of this architecture is the ability to divide and conquer the problem within each layer. A single business service can be worked on by numerous people within each layer. It requires solid governance, strong collaboration skills, and teamwork to work this way which is why it is critical that the right team of people is assembled to tackle SOA. My next post will discuss the testers and their relationship with the development team.

I am supposed to be landing in France in one hour but instead I am posting this blog from my hotel in Philadelphia due to a missed connection. This situation could have been avoided if there were better business processes in place. This unfortunate situation coupled with inflexible business processes on the customer service side of the business has cost this airline a few customers for life. I bet a company with good business processes, like South West Airlines, would have handled the situation better.

Here is my story. My flight out of Tampa on US Air was supposed to leave at 2:15 en route to Philly. Due to weather we left 75 minutes late. When we arrived in Philly, we had 15 minutes to get to our gate which happened to be on the other side of the airport. We showed up at the gate at 6:02 for a 6:00 flight and the plane was gone. My colleague and I were not the only ones left stranded. Here is an opportunity to improve business processes and leverage technology to avoid losing customers. Business Process Opportunity #1. US Air should consider having processes that detect the impacts of delays in connecting flights. They should have known that there were a few customers who were arriving from Tampa and were at risk of being late for the connecting flight to France. The flight to France is seven hours so holding the flight for an extra 15 minutes would not cause a delay in the arrival time. They could easily make up that time in the air.

The story continues. We then had to get another flight to France. We were told that there was no other solution then to wait 24 hours for the next flight to France. We asked about other airlines and the answer was nobody had any other flights. She did not look this up on the computer she just declared it. We argued and argued and it went nowhere. Business Process Opportunity #2. A business process that explored alternative flights on other airlines would be nice. If they could have put us on another flight I would be able to write this off as a bad weather issue. Since they had no acceptable solutions, I will write this off as poor customer service and inflexible business processes.

It gets better. Now we need to get a hotel. This process seems to take an eternity. We get sent to different hotels and get $15 of meal money for a 24 hour period. That's enough for one Philly cheese steak sandwich! If you want to go on a diet fly US Air! Then I ask about our luggage. She has no information on it and directs us back to the original gate that we arrived on. Business Process Opportunity #3. Part of a customer retention strategy should be to have business processes to easily accommodate displaced travelers. Better collaboration with hotel partners and information about the whereabouts of my luggage should be a click away.

Not done yet. Then we grumble and moan our way back to the luggage claim another 15+ minutes away. The good news. Our luggage is at the airport and did not go to France without us. The bad news. We can't have it! It is checked into customs and cannot leave the area. I asked for them to rethink this since I have some mediations that I need to take each morning for my kidneys. Her response, "That will be a problem". Arguing led to yet another reality that their existing processes are very rigid and they have a culture of procedures over customers. Business Process Opportunity #4. Revisit legacy business processes and make sure they still meet the needs of the business. Identify exceptions and see which ones make economic sense. How hard would it be to get me my bag. Luckily I can go a day or two without my pills before my blood pressure goes off the charts. I hope no heart patients had the same scenario.

In total we had to talk to four different people in four different locations to get all of this done. The outcome was totally unacceptable (24 hour lay over with $15 for food and no access to my luggage). Every time we asked them for alternatives we heard the same clear message, "We can't". This is not a people issue, this is a process issue. In fact, the employees were very sympathetic and professional, but their hands were tied due to ineffective business processes. I am sure with the right business processes, these same people could have helped us out. Since US Air does not seem to think that business processes can be a competitive advantage, they have lost these two customers for life and possibly anyone else who reads this.

I would love to hear from anyone in the airline industry how your company deals with delays and exceptions.

Here is another lessons learned from my world, but this time the focus is BPM, not SOA. I have mentioned many times how we sold the business on BPM and SOA. After modeling the future state processes and creating a roadmap of projects based on a combination of business priority and service reuse, we took our business case to the finance department and secured funding for a number of high ROI projects. The funds secured were for procurement of BPM and SOA tools (BPMS, ESB, Data Services, etc.) and for capital labor to work on the various projects. We also were able to justify two new positions in a new department in the business who is responsible for business processes. Sounds good so far, right?

Well, we should have funded one more thing and that is an initiative to change our culture to be process centric. The two new resources are consumed in the new projects which means the process of becoming a process-centric culture will take a very long time. It takes focus and dedication to the cause to train staff, hire experienced talent, and move the organization to a place where optimizing business processes is the norm. So what is the down side?

Although we are providing a ton of value by automating processes, connecting legacy systems, providing visibility into the workflow, shortening the order life cycle, and leveraging operational dashboards and reporting, we are not creating processes that are as efficient as they could be. In some instances, users are asking for the same functionality and processes that exist today. There is still too much focus on reports and not enough focus on data and there is still too much emphasis on exceptions and not enough standardization.

Is it a disaster? No. Could we be developing more efficient processes? Yes. I am hoping that with each iteration we will improve the processes and the business folks will become more process-centric over time. I do feel that if we could have justified an initiative to create a process-centric culture, we would achieve a higher ROI and see the benefits of it quicker.

So the lesson learned that I would like to share is don't forget to invest in change management initiatives. Getting funds for a culture change initiative is not easy, but you should strongly consider trying. Some companies already have business process centers of excellence or practice methodologies like TQM, Six Sigma, Lean etc. For those who don't, the earlier the culture becomes process centric, the better the Return on Investment.

Here are a few of my other lessons learned from past posts:

• Boiling the ocean
• Things we did right
• Things I would do differently