Enterprise Initiatives

This Blog is Moving to a new Home!
This is the last post I will do on Blogger. I am moving Enterprise Initiatives to KavisTechnology.com. The material will stay the same but the landing page will be different. If you are a subscriber, you can get the new RSS feed here.

Why is the blog moving?
I started my consulting company this past June and created a new blog on my company's domain. That made my fourth blog that I have to maintain. When I started blogging a couple of years ago I started here on Blogger with the Enterprise Initiatives blog. After a month I discovered the ITToolbox community and started blogging there as well. I was going to move this blog to the Toolbox but Techdispenser started including this blog in their community. Then I was invited to write for CIO.com on their SOA Drilldown blog which I did for three months.

Although not all of my articles go on all three - blogger, ITToolbox, and KavisTechnology - most of them do. I wind up doing a cut and paste exercise everytime I blog. So to save some time I am going to cut it down to two blogs, the ITToolbox (generates a little revenue for me) and my company blog @ KavisTechnology.com.

Blog Editor Ranking
I would like to rank the three different blogging platforms that I have used. For this blog I use Blogger. ITToolbox uses Typepad and KavisTechnology uses WordPress.

#1 - WordPress - WordPress rocks! The administration panel simply blows away any other blog editor that I have used. I just upgraded to 2.7 with ease. There are thousands of plugins and templates available to use. In addition, there is such a huge WordPress following that you can find countless articles and even blogs dedicated to tips and tricks for blogging with WordPress.

#2 - Blogger - Blogger has a very nice editor and integrates easily with many Google owned products like YouTube, Picasaweb, AdSense and others. It has some nice templates but its administration panel is limited compared to WordPress. With Blogger I was able to easily integrate rich media content into blog posts. Blogger plugins are limited to what Google allows. Of course you can always add your own code but most bloggers prefer not to.

#3 - TypePad - My evaluation of Typepad is a little unfair. I believe that ITToolbox limits some of the funcitionality so that they can get hundreds of bloggers to deliver in a consistent format. We are not allowed to choose our own templates and they only offer a few plugins. The editor is way more cumbersome then Wordpress or Blogger which is why I cut and paste my code from WordPress into the other blogs.

I highly recommend WordPress as the blogging editor of choice!

There is so much confusion (and fear) about what the term Cloud Computing means and what it really is. Now that everybody kind of "gets" SOA, the new buzzword that people can't grasp is Cloud Computing. Everyday I see the same questions being asked in LinkedIn like What is the difference between Cloud Computing and SaaS? A simple question like this will get 100 different answers. So this is my attempt to clear the fog on Cloud Computing in a simple non-technical way (sorry for boring those that already know this stuff).

Here is a picture that shows a high level view of different types of cloud computing


This picture is very basic but shows different types of companies leveraging the cloud model. You can also see that some types of computing are a subset of the cloud. Let's talk about the different types and what they represent (forgive me as I reuse some slides from a previous presentation).


First you have the Internet which we are all familiar with. When the Internet first became the buzz, a lot of the same confusion and fears that surround cloud computing today were being discussed back then. How is it secure? Who owns the infrastructure? How do I control it? Early adapters saw the opportunity to reduce costs by moving brick and mortar operations to the Internet and increase revenue by providing services and goods 24x7x365. This following picture shows some major web sites that leveraged the Internet (or should I say Cloud 1.0 to be hip).


Then came Software as a Service which was pioneered by companies like Google and SalesForce.com. With SaaS, instead of buying shrink wrapped software that you must install on your infrastructure, patch, administer, and do all of those other non-value added tasks, you simply "lease" the rights to use the services that are provided. In the case of Google and services like GMail, you get it for free.


Companies like Google, SalesForce.com, and Amazon have built out huge datacenters in multiple locations with an enormous amount of capacity. They cannot afford to have a huge spike of traffic take their sites down and risk losing millions of dollars a day. At some point, these companies realized that they could offer their excess capacity for a fee to other companies to use. These companies would be able to take advantage of world class production environments that are proven to withstand millions of concurrent users. This is what is known as Platform as a Service (PaaS) and Infrastructure as a Service (IaaS). The difference is the platform.



PaaS providers like Google's App Engine and Force.com allow you to build your own applications on top of virtual server instances but restrict you to using their development languages. For Google it is Python, for Microsoft's Azure it is .Net, for Force.com it is Appexchange. One could argue whether Facebook should be classified as PaaS. I put it there because they now provide a full development platform for running applications on Facebook. Checkout this Facebook App by Starbuck's.

Infrastructure as a Service (IaaS) removes this limitation and gives you the ability to create virtual servers and develop in whatever you choose. The most popular IaaS vendor is Amazon who is in my opinion the leader in innovation and maturity in the cloud computing space.


So when people ask me what cloud computing is, I include all of these different models into my definition. "In the Cloud" to me means leveraging infrastructure off-premise. With SaaS solutions, the underlying infrastructure is hidden from you. With PaaS, you manage the amount of virtual server instances you use but you must use the technologies required by the provider. With IaaS, you manage the resources you use and are free to leverage whatever technologies you choose to deploy on.

Here are a few other useful resources that show a fairly complete list of vendors and the niche they fill.

Peter Laird - Visual map of cloud computing

Markus Klem - Cloud Classification diagrams

John Willis - Tools to use in the Cloud

John lists a number of vendors who provide a wide variety of tools to manage, configure, monitor, and scale including open source alternatives.

Check these resources out. I hope this gives you all a clear picture of what Cloud Computing is. The benefits and the risks are blog topics within themselves. I'll save that for another day!

I have been blogging for quite some time about SOA and Event Processing and have recently been getting more experience with Cloud Computing. The last few weeks I did a series of posts on agile SOA:

• Agile SOA: Empower the Business with Business Rules Engines
  
• Agile SOA: The Flexibility of Business Processes
  
• Agile SOA: Leveraging Data Services
  
• Enterprise Mashups: The Icing on your SOA
  

Today I completed a nice presentation on Slideshare called Agile Architecture which combines SOA, EDA, and Cloud Computing in a strategy to support the ever demanding needs of our dynamic business environment. I have the luxury of starting from scratch because we are in startup mode and do not have the burden of years of legacy systems and legacy cultures. The goals of our architecture are many, but here are a few key goals that speak to the topic of agile:

Back in September I shared with all of you the presentation I gave about SOA and Organization Change Management. Today I am happy to share the podcast from that discussion that took place at the quarterly SOA Consortium meeting in Orlando. Here are the slides.

And here is the podcast that goes with it.

The panel discussion at the tail end of the presentation is fantastic. There were a lot of great questions and many lessons learned shared from experts like Todd Biske, Brenda Michelson, Fill Bowen from IBM, and others. If you have the time, listen to this podcast. Failure to recognize and counteract the resistance to change is the number one cause of failure for all enterprise initiatives, not just SOA.

If you have been following my blog recently you would have seen my previous posts that discuss the benefits of the various layers of the stack (mashups, data services, business processes). Although many IT folks talk about the benefits of SOA in terms of code reuse, the real benefit is in the ability for IT to adapt their systems at the speed that the business is changing. One of the best ways to accomplish this feat is to put tools in the hands of the business to empower them to dynamically alter the business rules as they change in real time.

In the old days we built systems with rigid rules and users were forced to adopt to the "laws of the system". Today, users are king and they expect systems to adapt to the way they need to use them. In order to satisfy today's tech savvy users we must build dynamic and customizable systems. We also need to make sure that IT does not become a bottle neck and hinder the rate of change that occurs within the business. The answer is simple, abstract the business rules into its own layer within the architecture and empower your users with the appropriate permissions to make the necessary changes in a timely manner.

Let's discuss the following order process example that I used in the post about business processes.



Now what if we had different rules on what a good credit score is? What if those rules were dynamic? Do we want to hard code these rules in the system and accept change requests from the business to every time they change or would we rather abstract them and give the business a tool to change them as needed? I'll vote for the later! Here is the same business process integrated with a business rule engine (BRE).



Here is a simple example of how the business can set up rules to apply different criteria to different customers.



In this example you can see that Gold customers get special treatment and will be allowed to proceed regardless of their credit score. Other tiers have higher standards applied to them. Now let's say that due to the credit crisis the company decides to apply stricter criteria on their customers and raise the minimum at each level by 20 points. The business can simply make these changes from the business rules engine interface which is most likely done from a browser. If the company wanted to add a Platinum user type, they simply add a record to the customer type table and the appropriate rule for Platinum customers in the BRE. Done! No change request, no priority setting, no change control board, no development, no testing, no rollout!

I also used a callout to point to a business rule lookup at the fulfillment process. Let's say that our company is an eCommerce seller for many different wholesalers. We may sell books, music, appliances, clothes, etc. but we are simply the middle man and don't own the inventory. Instead we sell, take a cut, and trigger transactions to the appropriate wholesaler for distribution. Each wholesaler has a unique set of business rules for their fulfillment process. In addition, we may add and remove wholesalers several times during the year. We have a standard business process for dealing with fulfillment but different wholesalers may have different rules due to industry specific or geographic requirements (age limitations, state laws, tax rules, etc.). Once again, an appropriate business person with the proper permissions should be able to make these changes as necessary without having to deal with IT.

Business rules have their place and purpose but they also come with some baggage. Like any other layer within the architecture, business rules need to be administered, maintained, and governed. Business rules need some level of change control, backup/recovery, and auditing like any other artifact. Most BREs either provide that functionality as well or hook up with other tools that manage artifacts (CMDBs, governance tools, etc.). But business rules are only worth the hassle if the business environment is dynamic. If the rules are static in nature, it is probably not worth the effort from an architectural stand point to create an abstract business rules layer complete with the appropriate security, auditing capabilities, etc.

The higher up the stack we go the more opportunities we create for the business to become agile. Business Rules create the ultimate flexibility and agility that most of today's businesses require. Implementing a BRE in your SOA is another great step towards IT becoming an enabler instead of a cost center!

I have spent a lot of time recently questioning the leadership of IT organizations who become a cost center due to a "keeping the lights on" mentality and have asked the question Are we Sleeping at the Wheel?. The other day I stumbled across a great article (thanks to some of my pals from Twitter land) that really hits the nail on the head. This is a must read article that brings to light what I think is the main reason why many IT leaders are missing the boat on emerging technology trends. The article is written by Steve Andriole is called Managing IT: Changing Our Minds (About Everything) and discusses how IT leaders who have been around a while have to let go of solutions of the past and totally change the way they think. Here are a few excerpts...

Here’s the deal. The world has changed – forever. First, hierarchical management structures will weaken as we continue to globally decentralize our business units. We have to change the way we think about control, standardization and the overall governance we bring to technology acquisition, deployment and support.

Here is his thoughts on Open Source which I have been championing for quite some time...

Open source is here to stay. Even the established vendors have “embraced” open standards. They have no choice. Do you?

And what about cloud computing?

We need to change how we think about cloud computing from an incremental shift in technology offerings to a whole new way of acquiring, delivering and supporting digital technology

Here is my favorite...

Debating endlessly about whether or not open software, cloud computing or SaaS have any merit is a waste of time – and most likely a diversionary tactic designed to slow – if not outright kill – the pace of change.

Please take the time to read the entire article. I think the message is an important one. If you are an IT leader who is missing the boat, you need to reevaluate your positions on the emerging technologies and solutions. If you don't you are damning your organization to more years of fire fighting and being a bottleneck to the progress of the business. Don't miss the boat!

An article written by Joe McKendrick called SOA eased the way for recent shotgun financial mergers inspired me to take this discussion one step further. To me, the biggest benefit of SOA done right is agility. I define agility in IT as the ability to adapt to change at the speed of business. Companies that invest in architecture, specifically in SOA, are in a much better position to adapt to mergers and acquisitions, connect to partners and suppliers, take on new business opportunities, and continually drive down the costs of maintenance and support.

I read many blogs and subscribe to numerous group discussions on SOA, EA, and cloud computing. There is so much discussion arguing the semantics, the ROI, and the viability of these technologies/architectures that it sometimes becomes discouraging. At some point we need to hike up our boot straps and invest in our business's future and stop fire fighting. Those companies that invest now (and do it right) will create a competitive advantage over those that do not. Here are just a few of the harsh realities that our businesses are faced with:

• Decreasing margins
• Fierce global competition
• Mergers/ Acquisitions/ Consolidations
• Recession
• Demanding, connected, and empowered consumers
• Increased regulatory constraints
• Dynamic business requirements
• Green initiatives

All of these things mentioned above are issues that we must deal with on top of our company specific issues and requirements that are far exceeding our capacity. To keep up with demand we need a flexible and agile architecture so we can make configuration changes instead of code changes, empower business users to change business rules instead of requesting expensive development projects, and leverage partner and supplier services instead of building everything from scratch, thus reinventing the wheel.

IT is only a cost center if the IT leadership allows it to be. The role of IT should be to innovate and help the business achieve its goals. IT must invest in strategies and architecture to allow this to happen. It is too easy to say we don't have the time, resources, and money. That's a cop out. As I have said before in my article The keeping the lights on mentality

The key business driver in a "keeping the lights on" shop is minimizing costs (usually over anything else). Which makes me wonder why more shops in this mode do not go down the outsource IT path. If lower cost is so important and large innovative initiatives are typically out of the question, why not radically lower the cost by outsourcing IT? I am not recommending that IT shops do this, instead I recommend that IT shops become good business partners and enablers of business success. But if I was a CEO or CFO and my IT's sole purpose was to be a cost center to keep the lights on, I would try to drastically reduce that cost. After all, keeping the lights on is not a core competency of most businesses. It is a necessary evil. Having hordes of full time staffers and paying for their benefits, stock options, and bonuses just to keep the lights on is not smart business. I know my view here will not be popular with most IT folks. I am not down playing anybody's abilities here. All I am saying is that if we don't need our staffs to be innovative, proactive, and be advocates of our business partners and instead just want to the staff to think short term, there is a much cheaper model out there.

So IT leaders have a choice. They can refuse to invest in SOA and continue to support and maintain an inflexible environment that takes a lot of time and money to change or they can invest in the future by justifying an initiative to provide the business with flexibility, agility, and empowerment. Is it easy to do? Heck no! It requires talented staff and partners and most of all top level support. Ask yourself this question. If your company was to merge with your number one competitor tomorrow, how long do you think it would take to integrate these systems? If your competitor was buying your company and your systems were archaic and hard to integrate with, what do you think the odds are that they would keep you around? That's what I thought! Pay now or pay for it forever.

Dave Linthicum wrote a post today called Open Source SOA provides some major advantages. In his post Dave stated:

When it comes to SOA, I think open source provides two major advantages:

• First, it's typically much less expensive than the tools and the technology that are proprietary.
• Second, they are typically much more simplistic and easier to understand and use.

To the second point, simplicity. The open source SOA vendors seem to take a much more rudimentary approach to SOA, and their tools seem to be much easier to understand and, in some cases, use. While some people want complex, powerful tools, the reality is that most SOAs don't need them. If you're honest with the requirements of the project, you'll see that good enough is, well, good enough.

Great point Dave. I would also add another clear advantage which I learned the hard way. On a previous enterprise wide SOA initiative, I drank the cool-aid that the vendor stack was an integrated stack and was simpler to deploy and manage over a stack of a mix of vendors. What I found out is that the mega vendors (IBM, Oracle, etc.) have bought so many pure play tools (rules engines, BPMs tools, data services and MDM tools, governance tools, etc.) that the smooth integration ends when the Power Point decks are closed. In reality, the mega vendor stacks are a hodge podge of rushed acquisition and integration efforts. The underlying architecture of each tool within the stack are completely different and there are very few people (if any) within the organization who understands the complete stack. In fact, we were dealing with two very different organizations when dealing with support and they were not in sync. Eventually the entire company was consumed by another mega vendor (you can probably guess which acquisition this was) and the whole product roadmap was turned upside down.

Now let's look at some of the well established open source stack vendors like WSO2, MuleSource, and RedHat. These vendors do not suffer from acquisition madness and chaos. If fact, they are all built on a consistent architecture and do offer smooth integration between the various layers of the stack. Do they have all of the features of the commercial products? No. Do they have enough features for most SOA initiatives. Definitely. I wrote a post on CIO.com called Tight Budgets? Try open source SOA. Here is a quick summary of the advantages I discussed (read the article for the details):

1. Try before you buy
2. Lower cost of entry
3. Cost effective support
4. Core competency
5. For the people by the people

So what open source options do I have, you might ask? The following picture shows the open source tools that I prefer for my new SOA initiative. We are using a combination of WSO2, Intalio, Drools, Liferay, and PushToTest.



This is just one example of many. You can mix and match tools from different open source communities or you could standardize on one community. Here is an example of Red Hat's jBoss SOA stack.



And MuleSource has a well known suite of tools as well.


Many organizations are still not very comfortable with open source for mission critical initiatives. I have debunked many of the open source myths in the past (here, here, and here).

If there ever was a time to embrace open source, the time is now in this harsh economy. As commercial SOA vendors continue to get gobbled up by the mega vendors, it is time to seriously consider alternatives.

As I have mentioned in many posts before, SOA built right can provide business agility and flexibility. Yesterday I discussed how to leverage data services and a few days back I discussed enterprise mashups. Today the focus is on business processes.



Separating business processes from code is an ideal way to build an architecture that enables systems to adapt more readily to change. Let's make believe that we are working for a large eCommerce site that sells many different categories of products from multiple suppliers and warehouses each with their own distinct business processes. Abstracting the business processes gives us some very important advantages:

• Creates a set of standard business processes for a consistent user experience
• Allows us to quickly bring on additional suppliers
• Allows us to change a core business process once and apply it to all suppliers
• Allows the business to make modifications to its processes with minimal or even no coding

Here is an example of an order fulfillment process.



This work flow speaks to my first point "Creates a set of standard business processes for a consistent user experience". The complexities of working with multiple suppliers, each with their own processes and systems is hidden from the end user. Regardless of what products the customer is buying and which supplier is fulfilling the order, the user experience is exactly the same. On the other side of the equation, bringing on a new supplier can be as simple as collaborating on a set of services to integrate the supplier's back end system to the core business processes of the eCommerce site. Had we not separated the business processes from the code, we would have to make modifications to our existing code base which is already running in production and would have to go through a complete testing cycle. Instead, we simply tell the business processes which set of new services to integrate with for the new supplier. All of the existing code is not impacted and does not need to be retested. Instead, we can just validate the new services and the new supplier flow.

Here is another benefit. Let's say that we just signed agreements with three new suppliers and have contracted to launch them all in 4 weeks. Without the separation of business processes from code, all three supplier integration efforts would have to be coordinated and most likely bundled into one build. This creates huge project risks because slippage in any one integration effort can delay all three. By separating the business processes from the code, each supplier integration can be implemented whenever it is ready. We can stub out a call for each supplier and assign the appropriate WSDL when the services are completed. For example, the "Fulfill Order" process could be an integration with SAP for supplier 1, PeopleSoft for supplier 2, a custom built fulfillment process written in Java for supplier 3, and a custom built fulfillment system written in .Net for supplier 4. The business and the business processes have no need to be concerned with the technologies behind the fulfillment process. They are only concerned with the inputs, the outputs, and the quality of service (QoS).

As we move up the stack it gets even better! Next post I will discuss the importance of business rules. Stay tuned.

The promise of SOA is flexibility and agility. I define agility as the ability to adapt to change at the speed of business. In today's global economy which has been fueled by collaboration and Internet technologies, businesses change at a much faster rate than ever before. So how does SOA help companies become agile? Let's look at a logical view of a typical SOA.



You can see in this diagram how each layer of the architecture has been abstracted and is mutually exclusive or "loosely coupled" from the other layers of the architecture. Why is this important? The answer is simple.....ease of change! Here are some advantages of this approach.

• Share services, components, rules, etc. across the enterprise (reuse)


• Isolate changes and reduce dependencies (speed to market)


• Minimize impact of business changes (speed to market)


• Easier to maintain (maintainability)

Let me give an example of how this approach helps companies become agile.

Use Case: New customer data from a new client


Let's say your company provides a service for customers in the retail industry. You have a website that offers online services for consumers and your white label solution is tailored to look like it is hosted by the individual retailers. The problem is that each retailer has their own customer database. In the past you would have to write a ton of code for each retailer that you signed up to use your services. With SOA, now it is a simple data mapping exercise. By abstracting data services, the other layers of the architecture use the logical representation of customer, not the physical implementation. Behind the scenes, the data services layer is translating the request for customer data from logical view to physical implementation. So when you bring on new clients, you simply use a tool in the data services layer to map the new clients customer data to the standard logical definition as defined by the architecture.



You can see from this example that all three retailers have an entirely different implementation of their customer database including different naming conventions and even different attributes. In the data services layer, you can map all of these physical implementations to one standard customer definition. You can also see how the business processes all use the logical view of customer. This allows us to add and change customer definitions on the back end without changing code on the front end. If two more retailers were to sign up tomorrow, we can map their definitions to the logical customer view and be done. No Code!!! That is agile!

I will give examples of agility within the other layers in future posts. If you would like help establishing a data services strategy, give me a shout!

The Dilemma
I have been spending a lot of time recently learning about enterprise mashups. The new startup that I am working for is building a SaaS solution that will be consumed by various types of customers and partners. These customers and partners may want to consume our data services as a RSS feed, gadget, SMS message, web page, within a portal or portlet, or a number of different ways. I do not want to spend the rest of my life developing new output mediums for our services. Instead, I would rather spend my time adding new business services to enhance our product and service offerings hence contributing to the bottom line.

Enterprise Mashups to the rescue
Enterprise mashups will allow me to offer my partners and customers the ultimate flexibility to access our products and services in ways that are convenient for them without having to wait on my IT shop to decide if (a) we think the request is important enough in our priority list, (b) if we have the time and resources to work on it, and (c) how much we will charge them. On the IT side of the house, with an enterprise mashup strategy in place we can be assured that whatever mashups our customers and partners create, they will be subject to the same security and governance as the services we have developed. The diagram below shows a logical view of how our SOA will be designed.



As you can see, we have clearly abstracted the various layers within the architecture and they all inherit our overall security policies. SOA governance is applied to this architectural approach to enforce our standards and design principles. Overall IT governance provides oversight over the entire enterprise which includes legacy systems (we don't have any legacy yet), third party software, etc.

Now let's add the enterprise mashup layer. We want to hide the complexity of our architecture from the end user and expose data services to them to consume. At the same time we want these mashups to be equally secure as the services we write and adhere to the same governing principles. Enterprise mashup products provide tools to make managing this layer easy and efficient. The diagram below shows the enterprise mashup layer inserted into the architecture as a layer on top of SOA.

From SOA Slides

Enterprise Mashups in simple terms
If the concept of Enterprise Mashups is still not clear, check out this white board discussion with JackBe's CTO John Crupi.



I spent some time talking to JackBe's VP of Engineering Deepak Alur a few week's back. Deepak discussed how enterprises have been focusing more on infrastructure and technology and not on the consumers of data. As he coined it, many shops are "developing horizontally and not addressing the needs of the users". He talked about how users were doing their own brute force mashups by cutting and pasting data from various places into Excel. This creates various issues within the enterprise due to lack of data integrity, security, and governance. It is ironic how corporations spend huge amounts of money on accounting software and ERP systems, yet they still run the business out of user created Excel spreadsheets! The concept of enterprise mashups addresses this by shifting the focus back to the user consumption of data. Here are some of the requirements for mashups that Deepak pointed out to me:

• User driven & user focused
• Both visual & non-visual
• Client & server side (although most are server)
• Plug-n-Play
• Dynamic, Adhoc, Situational
• Secure & Governed
• Sharable & Customizable
• Near zero cost to the consumer

Jackbe's enterprise mashup tool is called Presto. Presto is an Enterprise Mashup Platform that allows consumers to create "mashlets" or virtual services. IT's role is to provide the security and governance for each data service that will be exposed for consumer use.



Presto Wires is a user friendly tool to allow users to create their own mashups by joining, filtering, and merging various data services (as shown in the picture below).



In this example the user is combing multiple data points from many different organizations in an automated fashion. They could then present this data to multiple different user interfaces and devices. All without waiting on IT.

Here is a simple example of a mashup of Google Maps with a Carpool data service.


The folks who create the Carpool data service where able to leverage the Google Maps service without having to write their own mapping software and without having to know the underlying architecture behind Google Maps.

How this solves my Dilemma
Back to my dilemma. By leveraging a tool like Jackbe's Presto or WSO2's Mashup Server, I can now present various data services in a secured and governed fashion to my customers and partners without being concerned on how they want to consume it. Whether they want the mashup on their own intranet, as a desktop gadget, as an application on Facebook, or what ever they dream of, all I need to be concerned with is the SLA of my data services. This also makes my product offering more competitive than my competitors who have proprietary user interfaces that do not provide the flexibility and customization that the customers desire. As I said in the title, this is the Icing on your SOA cake. For those organizations who are disciplined enough to implement SOA and follow the best practices of design and governance, the reward can be an simple addition of an Enterprise Mashup Platform on top of your SOA stack. This is the ultimate flexibility and agility that SOA promises.

As I look out into the future of IT over the next 5 to 10 years, I see a huge shift in how IT shops will need to operate in order to help their companies survive. We are already well aware of the pressing needs for IT to provide agility and flexibility for its business partners due to the speed at which the business landscape is changing. The forces of globalization, economic pressures, and advancements in technology are creating as much change in an 18 month period than we used to experience in a decade. In order for companies to survive and thrive, they need to adapt. As Charles Darwin once said,

“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.”

I wrote a post last year called How did we become a Dilbert cartoon which discussed my theory on how IT has become so out of touch and bogged down with trivial issues. I see Dilbert every day whether it is at the places I have worked, the case studies that I research, the discussions I have with peers at conferences, or from the forums and user groups I participate in. Somewhere along the line, many IT professionals in the US have forgotten what IT's purpose is and take their IT profession for granted. These people put themselves and their favorite technologies first and their business and shareholders second. How many important initiatives have you seen stalled because certain individuals refused to change or learn something new? Look how many companies are struggling implementing transformational initiatives like SOA, ITIL, business process reengineering. All of these types of initiatives can make a huge impact to the bottom line. But how many of these have stalled because people fought these initiatives with all of their might? The basic problem is that transformational initiatives requires transformational leadership! How many leaders within IT do you know who excel in the three critical areas of transformational leadership required to deliver technological solutions: Business Acumen, People & Organizational Skills, and Technology skills?

From Misc IT

Looking down the road, I see certain technologies maturing and becoming critical to helping businesses staying competitive. These technologies are:

• Infrastructure as a Service (IaaS)
• Platform as a Service (PaaS)
• Social Networking/Software within the enterprises
• BPM, SOA, and Event Processing working in harmony
  

I am amazed at the number of pundits that exist for all of these technologies, especially cloud computing. Many people are strongly against these advancing technologies at the expense of their own careers (they just don't realize that they are making themselves the equivalent of Y2K programmers yet). Sure, cloud computing has its challenges with data security, privacy, and in some cases reliability, but it is in its infancy state. Instead of focusing on what the limitations are, we should focus on the huge strategic and financial opportunities that it creates. Let me quote Darwin again.

“Ignorance more frequently begets confidence than does knowledge: it is those who know little, and not those who know much, who so positively assert that this or that problem will never be solved by science.”

I still remember the naysayers doubting PCs and distributed computing and declaring that nothing could beat the mainframe. Do you here the negatively by some about social networking and social software? Doesn't it sound like the negativity we heard when people were trying to bring the Internet into corporations?

Why do we get in our own way of progress? Why are we living in Scott Adam's world of Dilbert? Why do system administrators fight cloud computing? Is it fear of job elimination? Loss of control? Not wanting to learn something new after 20 years? Why do IT professionals revolt against SOA? Does it require them to actually architect something rather than drag-n-drop some code in their favorite Microsoft IDE? Does it force them to collaborate with other people including business people and take them out of their comfort zone? Is it hard work? I don't know what the answer is but I do know that with out transformational leadership, the resistance can put up huge barriers and can kill initiatives that can give a company an edge over its competition.

What about outsourcing? Do you really want to get IT professionals fired up? Just the mention of the word brings anger and negativity to many. But why? Maybe we all need a lesson in economics or should simply read the book The World is Flat. While we in the US are sitting here complaining about change, countries like China, India, Taiwan, Ireland, and many others are graduating engineers by the thousands. These folks are more than willing to work on any task that they have the privilege to be given. And there lies the problem. To these people, work is a privilege, a way to have a better, more prosperous life. In the US, many people see their job as something that is owed to them. How many people in your shop are actively working on improving their skills? Just the fact that you are reading this post puts you many steps ahead of most. Many people that I have worked with in my 23 years do not invest their own time and energy required to continue to learn and adapt to the world around them, both from the technology and the economic standpoint. I am fine with the fact that many people value their personal time way more than some of us do, but if you don't invest the time you do not have the right to impede in the advancement of the organization!

Getting back to the emerging technologies that I mentioned above. To be successful implementing the transformational change to embrace these technologies, IT shops need the following foundation:

1. Strong leadership with the ability to promote and manage change
2. A well run and planned Enterprise Architecture
3. Solid working relationships and trust with the business partners
4. Discipline
5. Fiscal awareness
6. Numerous Strategic Partners
   

Strong Leadership
The leader(s) must be visionary, strategic, emotionally intelligent, and must be able to execute. I have seen leaders who have great ideas and are pretty smart, but they fall down when it comes to execution. Nine times out of ten, the failure can be contributed to people not the technology. In other words, resistance to change prevails and the company reverts back to the status quo leaving IT with the reputation of a cost center. The following presentation speaks to how to anticipate and plan for change up front to reduce the odds of failure.




Enterprise Architecture
To enable a flexible and agile enterprise, it all starts with an architecture that maps to the overall business strategy. No longer can we afford to build software in silos and continue to pay huge sums of money to keep the lights on. In order to be efficient with our ever shrinking budgets, we must have an IT strategy that is supported with an architecture geared towards maximizing our resources (both human resources and technology resources). The more standard and consistent the architecture is, the easier a company can move to the clouds, alter business processes in days instead of months, change business rules on the fly, adapt to mergers and acquisitions, and connect to partners, suppliers, and customers. Remember this, your biggest threat tomorrow might be a company that does not exist today! Why, because a startup does not have legacy to deal with and will most likely embrace these new technologies from the start and race right by you to the finish line. Companies must change or die. The following presentation speaks to the value of EA.


Relationships and Trust
In order to accomplish transformational initiatives, IT must forge great working relationships with its business partners, both internally and externally. Without the trust, funding and executive level support will be extremely hard to come buy. To earn that trust, IT must understand the business and put forth solutions that are beneficial to the business first and IT second. The following presentation shows an example of how SOA was explained to the business in tangible business terms (increase sales, customer satisfaction, etc.) instead of technology terms (reuse, ESBs, services, etc.).


Discipline
This is critical. We can no longer fly by the seat of the pants anymore. It costs the company too much in maintenance and hinders agility. That is not to say that we should all embrace heavy methodologies like CMM. There needs to be the right balance between process and agility. And for those IT professionals who will fight process to their death beds, there are millions of knowledge workers in foreign countries hungry to do your job for you. Today's resistance is tomorrow's unemployment.

Fiscal Awareness
Being fiscally aware is a key contributing factor to allowing IT professionals to see the "big picture". After all, the main purpose of most companies is to make money, or in government, to make the best use of tax payer dollars. It's all about money! So why are so many IT professionals so clueless about the financial impacts of the work they do and the decisions they make? How many times have you sat in meetings with armies of people for an hour or two and nothing gets accomplished? How does that contribute to the bottom line. When technologists argue about .Net vs. Java, or IBM vs. Dell for months on end while projects get delayed, why doesn't anybody seem to care? When IT professionals make technology decisions primarily for the sake of technology, they often make a choice that is not fiscally responsible. When making these decisions we must think about more than how we will use the products within IT. There are many factors that need to be consider. We should look at the feasibility not just from a technical standpoint, but also from an economic, operational, and political standpoint as well.

Strategic Partners
And finally, there is so much change and so much to learn, it would be suicide to think that we could deliver anything transformational in a reasonable amount of time without the help of partners. The business can't wait for IT to train an entire staff to a high level of competence on these emerging technologies. They also can't wait for us to stumble and learn from failing in production. Instead we will need strategic partners in many areas to help build the agile enterprises of tomorrow. These partners may be used for the following reasons:

1. Business Process Outsourcing (BPO) - (example: Payroll, HR, web site hosting, etc.)
2. Acquire new skills (SOA implementers, BPM experts, Cloud specialists, etc.)
3. Strategic partners (Organizational Change Management expert, EA help, etc.)
4. Technology outsourcing (IaaS, PaaS, SaaS, etc.)
5. Project based outsourcing

Outsourcing does not mean the same as offshoring and outsourcing does not always relate to development. When cost is an overriding factor offshoring is a no-brainer. Of course without EA and discipline, outsourcing may cost more than expected. There is simply too much to do and so little time to do it. Find the right partners and hold them and yourself accountable for transferring the knowledge to members of your staff.

Summary
I apologize for the length of this post (if you made it this far). I am very concerned for the future of my IT colleagues in the US and had to do a "core dump" on this post. After many years of prosperity, many IT professionals in the US have become intellectually lazy and blind to the opportunities and challenges that this new economy has created. The combination of our financial crisis coupled with the forces of globalization and technological advances will create radical changes over the next decade. Many professionals are too wrapped up in reality TV to realize that the world is catching up and is ready to pass us by. Those who understand that can embrace the new opportunities that these conditions will create. The others will be the equivalent of the Y2K programmers who did not embrace the post 2000 innovations. I'll leave you with another great quote from Darwin...

“In the long history of humankind (and animal kind, too) those who learned to collaborate and improvise most effectively have prevailed"

I was invited to speak at a forum for IT executives in Detroit this week sponsored by Information Week. The purpose of the forum as described in the agenda goes like this...

This executive breakfast, specifically designed for senior business-technology executives, will explore why the pressure is on IT to help the business transform, and how it can meet those expectations. More than ever before, companies are demanding their CIOs to be strategic thinkers in helping them innovate and operate at peak performance - especially as businesses are under pressure from the poor state of the economy and the ever-faster pace of change in a global market. In this environment, you can't miss this opportunity to gain insight into the tactics and strategy that will help you be on your best game.

I was specifically asked to talk about why transformational IT initiatives like BPM & SOA fail and what advice I would offer to prevent failures from happening. I put together the following presentation which is a combination of some of my previous presentations, Preparing for SOA and SOA & Change.

I wrote a very popular article on CIO.com a while back about the Top 10 reasons why SOA fails. I speak to each of these points in the presentation and present solutions for each. I also discuss using John Kotter's 8-step process for managing change which I highlight in the presentation. Here are my keys for preventing failures.

1. Plan for and manage organizational change


2. Key drivers should be business focused not IT focused


3. Evaluate internal skills and fill gaps. Do not try it without help!


4. Don't let the vendors drive your architecture. Do your homework.


5. Grow your governance model over time

Speaking of governance, here is an analogy I like to use...

Implementing SOA without a solid governance model is the equivalent to having an airport without a control tower. Sure, there are some very good planes and talented pilots, but without the proper planning and timely information the end results would be disastrous. So make sure you build a control tower and hire some air traffic controllers!

If you would like me to create and present a custom presentation like this one, feel free to contact me

I am a huge fan of social software, especially when it comes to networking with experts in technology. Those of you who follow me on Twitter know that I am a fanatic NY Giants fan who travels from Florida to hostile stadiums in places like Dallas and Pittsburgh to watch my beloved GMen go to battle. Last week, Jordan Haberfield, an expert at finding and placing talented SOA technicians at major corporations, asked me about a wager on the game between the Giants and his favorite team, the sell proclaimed America's Team and arch rival Dallas Cowboys. My normal bet with friends is for lunch or a few beers. Since Jordan is in New Jersey and I am in Florida, that bet did not make sense. What did make sense was betting that the loser would have to post a congrats on his blog.

Well, that was like taking candy from a baby as the Giants put a thumping on the depleted and heartless Cowpokes. To see Jordan pay his debts, go to his website called Agile Elephant. Jordan did manage to find a picture of one of the few positive plays that his Cowboys had in the game. My favorite Cowboy moment is still the Terrell Owens "My Quarterback" crying session after the Giants knocked the 'Boys out of the playoffs last year.



So add wagers to your list of things you can do with social software. It's the gift that keeps on giving and is searchable on Google!

A few days ago I had a chance to catch up with iTKO's co-founder and "Chief Scientist" John Michelsen to discuss SOA Testing. For those of you unfamiliar with iTKO, they produce one of the industry leading SOA testing products known as Lisa.

I have been writing about SOA testing a lot lately and the folks at iTKO contacted me after I wrote Six Questions to Consider before Building a SOA Testing Team on CIO.com. Since this article was fresh on my mind, I asked John some of the same questions that were posed in the article.

My first question to John was to identify some of the characteristics of successful SOA implementations he has seen from his customer base. Without hesitation, John pointed to customers that integrated testing into the development cycle. Successful SOA implementations tend to shy away from old waterfall methodologies and involve QA early in the life cycle. In many cases, testers work side by side with developers. John mentioned that "development builds things from scratch, QA does not". He pointed to this three step process which is the trademark of successful SOA testing:

1. Development builds the testing framework
2. Development builds initial test cases
3. QA enhances the test cases

Steps 2 & 3 is an iterative process and requires good collaboration between development and QA. This advice is similar to what I recommend in my article SOA Skillsets: PArt 4 - The Testers. In this article I recommend that a Test Architect be part of the EA team and design the testing framework and strategy. This person needs to be extremely technical and have a deep understanding of SOA concepts. The test leads also need to have a thorough understanding of SOA while the remainder of the testing team does not need to be as strong technically. However, they do need to be able to write some code as the developers hand over their test cases for refinement.

I asked John how technical the testers need to be. He stated that "most QA folks are still business analyst types and it should stay that way. If QA gets too technical then they lose the focus on testing". I agree which is why I advocate having the Test Architect as a member of the EA team focusing on building out the framework, evangelizing, and training the QA team.

I asked John about the importance of testing tools. He stated that the "Testing tools need to let less technical people perform technical tasks". This is a good point because your testers need to more focused on validating that the business and technical requirements are met than figuring out how to master a new tool.

This last point led us to a discussion about iTKO's Lisa products. John mentioned that they have just released version 4 which has many new features. In addition to the core product, Lisa Enterprise, the Lisa Server product has two new important modules, Continuous Validation Service (Lisa CVS) and Virtual Service Environment (Lisa VSE). Lisa CVS addresses the challenges of integration testing, deployment issues, and validation of governance compliance. Anybody with experience in SOA will tell you that the distributed and loosely coupled nature of SOA adds a whole level of complexity in the areas of builds and deployments.


Lisa CVS helps manage and automate some of those complexities and takes a more proactive approach to management and governance by allowing you to "revalidate expected behavior". Lisa VSE takes advantage of virtualization technologies so that you can create images that can simulate behavior of your heterogeneous environment. They accomplish this by installing agents on your various environments (mainframes, database servers, web servers, etc.) which allows VSE to monitor and emulate performance by creating a server side model of your production environment. This model can then be deployed in a virtual testing environment and be tweaked to simulate various scenarios such as abnormal spikes in traffic, simulating new customer loads, or whatever scenario you wish to entertain.

John was real excited when he talked about version 4.6 that is due out towards the end of the year. In this release, Lisa will add functionality to allow customers to test stateful services. This has been the missing link in SOA testing up to this point. John also mentioned that Lisa is now integrated with all major registry and repository tools. In addition, Lisa is also being used for Web 2.0 testing due to the work they have done with web services and WS-* standards.

In summary, testing is a critical component in the quest to launch and implement a successful SOA. By creating a well thought out testing strategy that includes tools to automate and simplify testing, organizations can focus more on the business side of SOA and less time focusing on the technology side of the equation.

My friend and colleague Todd Biske has recently published a must read book about SOA Governance. Most books that discuss process can require a ton of caffeine in order to make it from cover to cover. Not this book! Todd uses a unique style of combining a story of a fictional company along with his well served advice. He walks us through a multi year SOA project with a fictional insurance company called Advasco. Advasco, like many of the companies that we work for, has years of silo legacy applications that are the direct result of acquisitions, mergers, and years of developing in silos. The goal of their initiative was to "provide sales agents and marketing staff for the insurance division with a single view of the customer".

Todd takes us through the evolution which spans five years. Over this time, you can see the company make continual strides towards the overall vision but not without challenges. With each challenge comes another opportunity to mature their governance model. At the end of the book, Advasco's IT staff have made a major impact to the company's bottom line and ability to react to market changes. This is one of the few books that actually shows us what success looks like. This is why the book is so good. Many books talk about processes, structure, and controls but fail provide us with a glimpse into what the fruits of that labor looks like. Todd takes us from project inception, to successes, to set backs, to mitigation strategies, and ultimately to enterprise wide success. By taking us through this journey we can get a better understanding of the impact of the recommendations that he makes for SOA Governance. I won't go into details on what he recommends. I will let you read it yourself. But I can guarantee that you will enjoy the book and will be able to relate your real world experiences with the experiences of the fictional characters who helped move Advasco forward by leveraging SOA.

What made Advasco successful?
If you follow everything that Todd recommends in this book, you still are not guaranteed success. There were some specific events and characteristics that made this journey a success. Here is a short list:

• Initiative was driven top down
• Very little resistance to change
• Great working relationship with business and IT
• Very talented staff that was willing to learn
• Effective EA team
• Project was business focused

Final Thoughts
SOA Governance is a must read for any company preparing for SOA or for companies struggling to make their SOA initiative successful. Follow the advice and examples that Todd provides but also address the bullets I listed above. If you have the great staff that Advasco has you will find success. But in the real world, the effort to promote this type of change in the enterprise will usually require much more focus on change. At the SOA Consortium meeting last month in Orlando, Todd provided us with a great presentation about SOA Governance. I followed that up with this presentation on change.



If you focus on Organizational Change Management and heed Todd's advice on SOA governance, your odds of succeeding with SOA will be greatly enhanced. Buy the book, it is the best book on governance that I have read so far.

A little off topic today.

Each evening I help my kids with their homework. They are both very capable with computers. My daughter has been blogging about her pets since she was 8 (she is 10 now). She creates blog entries complete with multi media with no assistance from me. Other than creating the Blogger template for her, all of the work is her own. My son is the wizard of Wikipedia which he discovered on his own a few years back. The fact that my kids are computer savvy gives them a huge advantage when it comes to learning.

For example, my son has a weekly technology project where he must report on how certain technologies work. Like me, he is fascinated by how things are manufactured. So each week we pick an item that we are familiar with and he Google's it. Then we find a video on the manufacturing process like the one below about how bubble gum is made.



This is so much more effective than how I had to learn. I was forced to use either an outdated encyclopedia that my parents purchased or I had to go to the town library. Neither of these experiences offered rich media options. I often had to perform a ton of reserach to get the desired knowledge in order to write my papers. My son and I are able to knock these weekly assignments out in a half hour. At the same time he gets to see the actual manufacturing process in the video which enhances the learning experience. Then he notices similar videos and starts learning how marbles are manufactured.



It is so obvious to me that this generation can consume large amounts of pertinent information in short periods of time. My kids are very up to speed on the current election process and even know who the Governor of Florida is. I am not sure I knew who the governor of NY was when I grew up there in high school. This is the beauty of Web 2.0 and Internet technologies. The whole ranking, tagging, and social networking processes that are taking place are allowing my kids to learn about the world around them.

Here is another great example. My daughter gets assigned a science project. So I am prepared to carve out numerous hours each night to help her get this done. Instead, I see here collaborating with classmates on Google Talk and exchanging links and images. She would draft a story board on paper, scan it, and send it to her classmate. I was floored. She completed the entire project without my help using free collaboration technology on the web. And she's 10 years old! So I ask myself. Are you smarter than a 5th grader?

Tomorrow at 4pm I am presenting at the annual EDM Summit in Orlando on the topic of Business Intelligence as a Service. Check out the presentation below.

This presentation is based on a real life project that I once worked on. We had a scenario where the business had a very complicated set of business rules required to determine what inventory was available to sell. Inventory for a loyalty marketing company is very dynamic and is not a physical thing like a widget. Instead it is a data mining exercise comprised of many "What-if" scenarios.

The old way of doing things was to get data from many different sources, some from systems, some from spread sheets, and some from somebody's head, and go through an ugly and error-prone process to determine what was available to sell. The process took many days which puzzled the customer why we couldn't tell them right away if we could run the program or not. Through some analysis, we defined new and improved business processes and data services that would allow our sales people to enter numerous parameters on a new web UI and let the systems return information to the screen with potential sales opportunities.

When the designers brought their solution to the architecture team for review we saw a huge opportunity to change the approach that was being recommended for the user interface. The designers were Java guys so naturally they recommended a Java web based UI (which infuriated the .Net community). But as I looked at the multiple screens that they story boarded it became obvious that building a what-if type UI was not best served by building proprietary code. Business Intelligence tools make a living doing just that. So I put a hold on the design and recommended that we brought in our BI partner for a proof of concept.

To make a long story short, we proved with our BI partner, that we could leverage the BI tool to create a robust what-if style GUI which we could talk to all of the layers of our SOA (see slides for details). In other words, we abstracted the BI tool as our presentation layer. What that gave us was all of the bells and whistles that come with BI tools like:

1. Flash enabled output
2. Subscription services
3. Mobile capabilities
4. Alerts
5. Great scalability
6. Logging
7. and much more

All of these things I mentioned above were out of the box features of the BI tool that we did not have to code in Java. In fact, we offered to the business a much more efficient solution. Instead of submitting data and reviewing rows on the screen, we offered to automate the business rules and allow them to subscribe to categories. This means that they only needed to go to the system to tell it what to look for and the system would alert them when categories were available. Now they could go into a sales meeting knowing in advance what was available to sell. When certain categories became available they could get an alert and immediately schedule a call to the appropriate customer. A future step could be to tie the alert into the CRM system.

So the key point to this story is that you can leverage BI as an abstracted layer within your SOA which will help maximize the value of your existing BI investment.