Enterprise Initiatives

I was on Microsoft's website trying to understand the business benefits of upgrading to Vista. I found this article called Key reasons to upgrade to Windows Vista. To sum it up, here are the four reasons from Microsoft that justify this expensive upgrade:

1. Empowering users to find and use information
2. Enabling mobile workers to stay connected and productive in and out of office
3. Helping companies to make corporate systems and information more secure
4. Making it easier to deploy and manage company PCs

That's it. I would think that corporations would like more business value to justify the cost of hardware upgrades and PC purchases that will be required to run Vista. I can only think of two reasons to upgrade to Vista.

1. XP will eventually not be supported by Microsoft (currently targeted for 2014).
2. As other businesses upgrade, your company will not be able to open Office 2007 documents unless you install the Office Compatibility Pack.

Now let's discuss Microsoft's four reasons to upgrade. First, the improvements in searching for data. I use Google Desktop which gives me Google's world class search functionality across all my files and emails. The cost...Zero! It runs great on XP. No need to upgrade for this.

Second, enable mobile workforce. A lot of nice to have's here. Most companies already have tools that address the security and the collaboration. These features do not justify upgrading all of the hardware in your enterprise, especially when most of your hardware is not mobile.

Third, security. Wasn't this one of the selling points for going to XP. I feel like Bill Murray in Ground Hog's Day. Security is one of the main reasons why people are looking at alternative operating systems. Yes we need better security. Do I think that Vista is the answer? The jury is still out.

And finally, easier desktop management. These are commendable features. Many companies already have enterprise tools for managing their desktops and others are already moving to a desktop server approach that centrally manages desktop images and automatically updates client PCs when they connect to the network. I think the advancements in desktop management in Vista are great, but it does not justify the upgrade by itself.

So of the four reasons to upgrade to Vista giving to us by Microsoft, only security is compelling enough for me to even consider it. The shelf life of XP is the real business driver. We can argue all day long whether Vista is more secure then XP or any other OS for that matter, but other then a few reports from Microsoft's own Jeff Jones, I haven't seen any compelling facts to make me want to start an upgrade tomorrow.

I did stumble across a good article from Kelly Martin from SecurityFocus called The New Vista Waiting Game. He predicts that XP will be the corporate standard for years to come. Here is a quote from the article:

“ Despite all the coming advertising and sales pitches about early Vista installations, most businesses would be foolish to upgrade to Vista in the coming year. Businesses want stable, reliable environments. They want to see service packs that address problems even before they encounter them. They want secure environments as well, but to senior executives and other decision makers, this is still a function of Security Risk Management that can be mitigated in various different ways. ”

The good news is that XP will be supported for several more years. That gives corporations time to wait for Vista to become more stable and mature. It also gives corporations time to test different distributions of Linux and have an alternative to Vista once XP gets put out to pasture.

My good friend James McGovern asks, "I wonder if Mike Kavis understands why SOA shouldn't be sold" to the business and references the article Most enterprise SOA deployments fail to deliver ROI. This article makes some good points about the total cost of ownership (TCO) which includes extensive training and expensive tools like registries and repositories. The article continues with this quote:

The survey found that companies using SOA did experience an improvement in developer productivity by an average of 28 percent; however, the productivity savings do not warrant broad SOA deployment.

A little further down in the article, a very important point is made.

Despite these obstacles, Nucleus Research’s survey found that SOA is assisting companies in the areas of business process improvement and portals, followed by master data management and partner integration.

I think this sentence is where many of us EA bloggers start to disagree on whether you should sell SOA to the business or if an ROI is even achievable. In a previous article I described my view on selling SOA to the business. In my case, I was not trying to implement SOA by itself. Instead, SOA was being implemented in conjunction with BPM and Master Data Management (MDM). If I was only trying to implement SOA, I would have to agree with James's stance. But since SOA is part of a larger project, which includes business process reengineering, and the funds are coming from the business, I had to sell SOA to the business.

For James's sake, no I did not Power Point them to death or talk about web services and JMS queues. Instead I explained SOA in business terms and as a major contributor to the overall ROI. In an earlier post James states that if the business trusts IT then IT shouldn't have to sell SOA to the business. Once we convinced the business that SOA was the key to maximizing their BPMS investment, they trusted us to go figure out what tools we needed. Not once did we have to sell the concept of the ESB, MDM, repositories, training, etc.

Back to the ROI. The ROI will be achieved through huge operational efficiencies that lead to increased sales, improved customer support, better quality, and improved speed to market. One could argue that the ROI is a result of the process reengineering and not SOA. I am fine with that argument although SOA does allow us to leverage our legacy systems without causing major disruptions to our business and current projects.

So hopefully we can put an end to the "Selling SOA" discussions and move on to implementing SOA. As far as SOA and the ROI goes, SOA by itself is a cost of doing business. SOA in conjunction with BPM can pay for itself if done right.

James McGovern put out a call for more EA blogs a few weeks back. I'll take that one step further and call for more EA collaboration. There are a lot of different opinions about how to approach SOA, process, and many other topics. James has some strong opinions and calls out his peers when he disagrees with them. I love constructive criticism and I sure get it from James. The problem I have is that he doesn't post my comments and allow a discussion to take place. That is not collaboration, that is being closed minded.

I have been having many healthy debates on how or if to sell SOA to the business with fellow EA bloggers Nick Malik, Jack van Hoof, and Alastair Bathgate. I think it benefits the readers to see all of the different solutions to a problem. James just flat out says that I am wrong, end of story. So here is my challenge to James McGovern. Let's collaborate and post your readers' comments so we can see everyone's point of view. Let's have more collaboration. If we want more EAs to blog, let's provide a platform for them to share ideas and give and receive constructive feedback.

I just read two really interesting articles (Giving proprietary vendors a run for their money & Could Linux become the dominant OS?). These articles and a discussion I had yesterday about budget constraints for the next calendar year makes me think that Open Source Software (OSS) is on the verge of becoming mainstream over the next few years. I have already seen the statistics where 51% of companies are using OSS in mission critical applications. This is starting to look very similar to the days where everyone was fleeing the mainframe for client server technology. The client server craze was driven by lower cost and greater flexibility. Does that sound familiar?

Back to my budget discussion. I was having a discussion with a peer about budget constraints for the upcoming year. Our budgets typically remain flat or slightly increase each year. But each year the cost of doing business rises so we really have less to work with. We have been leveraging newer technologies, like virtualization, disk consolidation and compression, and others that have been driving costs down. Over the past few years we have been dealing with our budget constraints through technology improvements in the hardware area. Now its time to look at software.

As I look at the back end servers, I can't see how we can continue to justify spending the money on licenses and maintenance for proprietary operating systems like AIX or SCO or Windows 2003 unless the applications we are serving up mandate them. For example, we obviously need a Windows server to run Exchange, but many third party packages we buy give us the option of Windows or Linux. For those worried about support for OSS, read this article about open source service providers. With the advancements in virtualization, I should be able to create as many test and development environments I need as long as I don't have to continue paying for the OS licenses. Linux gives me that flexibility. I think a good strategy this year is to look at all of your software assets to see if there are candidates to move off of proprietary solutions to open source solutions. Once you have identified the candidates, put a plan together for replacing these systems over time.

Then I started looking further down the road. I have written many articles about my concerns with Vista and how this might be the right time to start a Linux on the desktop pilot. With the potential of Linux on the desktop being introduced to the enterprise over the next few years, coupled with applications moving towards SaaS models and rich AJAX enabled interfaces, does it still make sense to leverage .Net technologies and force the .Net framework and ActiveX controls on clients? If it makes sense to reduce licensing costs at the middle tier, Java, Ruby, or LAMP technologies sure look like better solutions.

So as I look down the road and see a continuous push to reduce costs while increasing value, I wonder how much proprietary software companies will be purchasing 5-10 years from now. Will it be like the mainframe where the only systems left standing are the ones that have no cost justification to replace? Will the norm be that new applications move to OSS? I know we are still a few years away from this but OSS is becoming more mainstream and widely acceptable in corporate IT whether we want to admit it or not.

In Part 1 of this series, I explained my reasoning behind creating an open source strategy. In Part 2, I will discuss our progress. But before I start, here are some predictions from Gartner:

• By 2010, 75 percent of mainstream IT shops will have a formal open source acquisition policy in place.
• By 2008, open source will compete with closed source in every infrastructure market.
• By 2010, mainstream IT shops will consider open source for 80 percent of their infrastructure software needs.
• By 2010, mainstream IT shops will consider open source for 25 percent of their business software needs.

Our first step was to create an inventory of the open source products that we use at my IT shop. We have a few areas within the organization that were early adopters of OSS and have a variety of products in use. When polling the staff for OSS products, I expected to find between 20-30 actively being used. I was shocked to find that we have around 100 different OSS products in our inventory (not including the ones packaged within proprietary closed software products). What an eye opener!

There is a lessoned learned here. Since the company as a whole still has not fully embraced OSS and still looks at OSS as the red headed step child, individuals have gone into stealth mode and started assembling a massive inventory of products that help them get there job done at a very low cost. What I found out is that we have a lot of duplication of products including various different versions. Some of the products are best of breed while others are questionable. If there is ever a need of a strategy, the time is now! Since we rely so heavily on OSS, we must embrace it as a strategic part of our enterprise and put the necessary governance around it. This takes us to step two of our strategy.

Today we spent a couple of hours with Sourcelabs, an open source service provider. This was another eye opening event for me. I knew open source service providers provide support for a wide range of OSS products. But here are a few things they do that I didn't know:

• Stress test and certify OSS products
• Contribute code to numerous OSS products
• Provide a one stop self service portal with information on numerous OSS products, including patches security alerts, product roadmaps, known issues, etc.
• Provide advice and guidance for product evaluations
• Assist in the creation and/or validation of your Open Source Strategy
• Fix product bugs and submit to the product's community for the next patch or release
• Provide certified Java middleware suites
• Provide open source policy and process best practices

All of this from one vendor across a whole suite of tools. This is so much more cost effective then paying 20% maintenance on every single product you buy in the world of proprietary software. We have a use case where we purchased a 20 node cluster of servers from a major vendor. We were required to purchase support for each node. The vendor mandated that we use SUSE Enterprise which more then doubled the cost per node. To make matters worse, the vendor is one to two years behind in the version of SUSE that they support on their hardware. The reality is that these servers just run and we rarely need any support for the operating system. So how cost effective is that model? For this use case, service providers are a no brainer. Not only is it more cost effective, but we can also choose whatever distribution of Linux we want because the service providers do not mandate what software we must use. Suddenly, the overall price of the cluster just dropped in half. Now for the same price (and better support) I can purchase a second cluster and drop it at our disaster recovery site!

So much for the myth that you can't get support for OSS. So to recap where we are with our strategy:

• Step 1 - create an inventory
• Step 2 - educate IT - this started today with our discussion w/Sourcelabs

I will continue this series as we move forward with our strategy. If any of the readers out there have experience with this process I would much appreciate hearing your lessons learned and recommendations.

The hot SOA topic the past few days has been how to sell SOA to the business. I have seen many authors talk theory on this topic. I am getting a little annoyed of when "experts" who don't actually have to sell SOA to their business partners tell me what works and what doesn't work. I am here to give you a real life story.

There are three camps on how to sell SOA to the business. Some experts recommend that you sell executives on the technology aspects of SOA only. The second camp recommends that you speak only about the business aspects of SOA. The third camp, which is where I go camping, recommends that you speak to both the business and technology aspects of SOA.

My recommendation, which comes from a very successful real life example, is to start with the business benefits of SOA. In my case we were pushing both BPM and SOA. We sold the business on the benefits of BPM and then explained how SOA was the key to allow the BPMS tool to talk to our legacy systems. This approach is much simpler then drawing multiple layers of an architecture on a white board and describing what an ESB is, what MDM is, and how web services or JMS queues work. In fact, we were able to get the funding for our SOA initiative without having to describe in gory detail what the different software modules were. Once the business knew that SOA was the enabler for their BPM initiative, which happened to have an eight figure ROI over five years, they didn't need to hear anymore. They only wanted to know how much!

For bonus points, we discussed how SOA (over time) would allow us to rapidly deploy applications because of reuse, increased flexibility, leveraging existing assets, and modular development. This became a win-win conversation. The business was getting a huge return on investment from the operational efficiencies gained by process reengineering, while IT was finally getting the funding they needed to build the architecture that they could never justify in the past.

I am sure there will be many more arguments on the "right" way to sell SOA to the business. The right approach depends on many factors that are unique to each project and each culture. My recommendation is to talk to people who have successfully sold SOA to the business and learn from their experiences. I shared my success story, who wants to share theirs?

My article Open Source and Microsoft Free was posted on Slashdot last week. I woke up on Saturday and looked at my traffic on Sitemeter. My daily traffic before this day was in the 100 range. That morning it read 2000+. I rubbed my eyes thinking that I was not quite awake yet and hit refresh. It shot up another 100. I was averaging several hundred hits an hour. Holy Smokes, I thought to myself! I then went and looked at my referrals and it was all Slashdot. After lunch, I looked again and I was over 6000. A quick peak at the referrals and I saw the Diggs coming it. The next few days were incredible. The article drew over 50,000 hits in a week! Then came the Del.icio.us, StumbleUpon, Craigslist, Linuxworld, Linux.org, and Japanese & Polish versions of Digg & Del.icio.us. It is been over a week now and I still get a few hundred hits a day from that post. That's the good part of being Dugg.

I probably received north of 500 comments across these sites. I am used to the comments that I receive on ITToolbox where I host my main blog. The comments are usually very collaborative in nature, even if the commenter disagrees with my point of view. I have built a very nice network of experts from the professional community on ITToolbox. Compare that to the discussions going on at Digg and some of the other social bookmarking sites. My article turned into an all out war between Microsoft and Linux fan boys. There was so much negativity, profanity, and non fact based opinions going on that I received very little value from the 500+ comments. As a matter of fact, I stopped reading it after a while. I would have had more factual conversations had I walked into a bar loaded with Yankee and Red Sox fans and argued whether Big Papi or A-Rod was a better hitter!

So being Dugg is nice from a traffic point of view, but my real goal with these posts is to share my opinions and collaborate with professionals to refine, defend, or validate my opinions. On Digg, it's all a bunch of noise.

I read this article today about outsourcing that claims that outsourcing is not just about cost anymore. The author points out these four reasons for outsourcing:

-Accessibility to right talent
-Geographic expansion
-Reinvention of their business model
-Promote innovation

After I finished choking on my cornflakes I decided I must offer a different opinion on this matter. First I want to differentiate between outsourcing and offshoring. Offshoring is obviously when you send work to other countries. Outsourcing is when you send work to companies outside of your corporation that may or may not be in the same country. Since the article was based on this link from The Times of India I know that they are really talking about offshoring.

Here is my take (from a US point of view). Offshoring is a cost savings exercise, period. There is a lot of overhead involved with offshoring due to language barriers, cultural barriers, and time zone challenges. To successfully offshore development, the paying customer must provide a maximum amount of oversite and process to overcome these barriers. Outsourcing is different. With outsourcing, I can bring a team of consultants on site for any amount of time without paying the expenses of flying them in from the other side of the planet. You still need oversite and process but not at the same level because the language and cultural barriers do not exist and the time zone differences are manageable (if there even are any).

I wrote this article a while back about agile development and consulting. In summary, if you want to be agile, don't engage with an offshore partner. Agile development requires face to face interactions and loosely defined requirements due to the iterative nature. The requirements get nailed down over time by cycling through requirements and protoyping. This model is a recipe for disaster if you are using offshore resources. For offshore to work you need to have a more waterfall type approach were the requirements are fairly static.

The article that I came across did not link back to the original research provided by PWC. If you read the PWC article you will see that over 50% of the service providers surveyed are US companies like PWC. So the findings in the research make a lot more sense to me because we are really talking about outsourcing and not totally focusing on offshoring. US companies are definitely leveraging consulting companies like IBM, Accenture, and others to foster innovation, reinvent their business models, and access the right talent. They don't use these companies to save money (they cost an arm and a leg). Saving money is all about offshore development.

The lesson learned for me is be careful what you read on the net. The PWC article is a very good analysis of what is happening in the marketplace. The problem is all of the offshoring blogs are using it to say, "See we are more then a cheap alternative", which does not represent the facts of the PWC research. For those who just read the headlines and don't take the time to read the details, beware!

According to opensource.org:

the promise of open source is better quality, higher reliability, more flexibility, lower cost, and an end to predatory vendor lock-in.

But that it is not the biggest driver for our Open Source strategy at my shop. One of our biggest drivers is budget constraints. Every year our IT budget remains flat or increases modestly. When you throw in merit increases, promotions, rising health care costs, and maintenance on software and services acquired during the year, you must get creative to stay within budget. With the exception of Linux on some of our back end servers, most of our enterprise software comes from the major vendors like IBM, Microsoft, BEA, Oracle and many other big names. But when it comes to developing software, it is hard to justify spending big dollars on the large numbers of tools we need to do our job when there are cost effective alternatives.

Our Open Source strategy that we are putting together addresses this. The purpose of our strategy is two-fold. First, we must educate our peers in the enterprise about Open Source. There are many myths that must be addressed to get everyone on board and feeling comfortable when leveraging Open Source. Tim O'Reilly listed 10 myths about Open Source in this 1999 article that still prevails today. Here are a few myths that we will address in our strategy:

Myth #1. It's all about Linux versus Windows.

Myth #2. Open Source Software Isn't Reliable or Supported.

Myth #3. Open Source projects are written by a small group of amateurs in their friend's garage.

Myth #4. The Open Source movement isn't sustainable, since people will stop developing free software once they see others making lots of money from their efforts.

Debunking Myth 1
Go to this page on Sourceforge.net and you will see the wide range of software categories that have active and established Open Source projects. If you wanted to, you could run your entire enterprise on Open Source. There is more to Open Source then Linux on the desktop.

Debunking Myth 2
For well established Open Source projects, it is not uncommon that you can get faster and better support in forums then from the expensive "Gold Support" that the major software providers charge you an arm and a leg for each year. It is not uncommon for small and medium customers to see unacceptable levels of support despite being a paying customer. The less money you spend with a vendor the less pull you have escalating support issues. There are plenty of Open Source service providers who provide support for a suite of Open Source products which is an extremely cost effective way of doing business. Traditionally, we pay vendors 18-20% of the cost of the original purchase price of each product. Support and maintenance can take up a huge chunk of an IT budget. With the service provider approach, we can pay the provider one flat fee and get support for several products at a much cheaper rate. What is even better is that the service providers core competency is support. That's all they do and their mission is to do it well. For purchased software, support is pure overhead and usually not the strength of the company.

Debunking Myth 3
Some of the most popular Open Source projects have hundreds or even thousands of developers world wide contributing to the overall code base. Scores of people respond to questions and support issues often within minutes after posting on the forums.

Debunking Myth 4
I wrote an article called Still Afraid of Open Source a while back that discussed how companies like Google and Yahoo are leveraging Open Source while the big guns like IBM, BEA, and Sun are investing big money in support of open source initiatives

The second purpose of our strategy is to identify software needs that we currently can't fulfill with our existing budgets. There are many development, testing, and software development lifecycle tools that we could benefit from but never have the funds to acquire. We need additional testing tools for our SOA initiative, a new defect tracking system, a portfolio management suite, and many others.

By putting a strategy together that identifies needs while addressing the concerns that people might have with Open Source products, we stand a better chance of fulfilling our team's needs with the support of our management.

Once we create a culture that sees value in Open Source, then we can start talking about evaluating Linux as an alternative to Windows when Microsoft drops support of XP in the future. I wrote this article called Open Source and Microsoft Free that discussed why I believe it is important to at least test Linux to understand what the issues would be in a production environment.

The worst thing that can happen with a small pilot is that you discover that Linux won't work for your organization. At least then you can sleep at night knowing you did your homework and made a strategic decision based on real information.

So what is your Open Source strategy? When you evaluate software, are Open Source products even considered? If it was your money, would you think differently or do you always buy the biggest and most expensive toys? As IT professionals, our job is to bring value to the organization. If you are not even considering Open Source alternatives for any solution, are you really looking out for the best interests of your company?

I have been blogging about my SOA & BPM implementation for the past few months. My earlier posts talked about how we sold SOA to the business, the vendor evaluation process, and our bottom up approach. We have been partnering with a SOA implementation consulting company for the past 10 weeks practicing an agile approach to delivery. In this short time frame we have installed our stack (BPMS, ESB, Data Services) and built a beta version of a B2B portal. Life is good!

So why can't I sleep at night? Because I have only a few months to ramp my organization up to take this over from our consulting partners. This may be the biggest challenge of implementing SOA that we have faced so far! We have formed a business process management organization in the business and hired our first Process Analyst. I feel good about that. We created a team of architects including a testing architect to help build out and govern the architecture. I feel good about that. We have access to network and hardware architects and moved someone into the role of administering the stack. Real happy there! But that was the easy part.

The hard part will be getting the rest of the organization on board with SOA. There are numerous challenges. First, SOA introduces numerous new roles and responsibilities for our staff. We will have to move from being SMEs (subject matter experts) in an application to becoming SMEs in a layer of the architecture (see chart below)
Second, we are moving away from our current methodology which is slightly waterfall in nature to a more agile and iterative approach. We are currently ramping up to implement our governance model.

But the biggest challenge is organizational change management. SOA concepts are drastically different then the way we currently develop applications. We are moving away from thinking about stovepipe applications and moving towards building business services. We still have some legacy apps that are written in VB6. For the folks maintaining those apps, they have not been exposed to web services yet. The world of testing changes drastically with this layered and loosely coupled approach. The DBAs now need to create a logical representation of data to hide the complexity of the joins and relationships from the services. The business analysts role is practically a different skill set now. Traditionally, our project managers have been assigned to applications and work side by side with the development manager who has a fixed team of resources. With SOA, we are moving more towards the classic PM role where the PM drafts a team from a pool of resources. I could go on and on.

So as you can see, every single person's role is changing. At the same time, we have a third party who is cranking out code faster then we ever dreamed of. We are unintentionally setting an expectation that we will be able to move this fast once we take this over. We are not getting incremental head count so we need to figure out how to transform our existing staff from application SMEs to specialists within the architecture over the next few months.

Anyone who has been responsible for managing culture changing initiatives understands the challenges that this presents. For each individual, we must answer WIIFM (what's in it for me) , how will this help the business, what's wrong with the way we do it now, and many other questions that cause resistance when left unanswered. At the same time we need to quickly educate everyone on SOA. Meanwhile, everyone has a current full time job and are assigned to existing projects.

A while back I wrote an article about the role of the enterprise architect. Many people only consider the technology side of the equation. There are days where I don't even have time to deal with technology. Some days are spent evangelizing to create more buy-in throughout the organization. Other days are spent working with business owners on a plan to reprioritize projects to free resources up to work on our SOA initiatives. And even more days are spent working with the development managers on plans to free up resources for temporary assignments (learning opportunities) on SOA projects.

So as we continue down the road with our SOA implementation, I have discovered that the most challenging part of the project thus far is adapting to change. The technology part seems easy compared to this. I will write more about this as we work through future opportunities and challenges.

Check out this article by Dana Gardner from ZDNet about Microsoft's approach to SOA. Microsoft just doesn't get it. Their approach to SOA requires an upgrade to .NET Framework 3.0. Funny, I always thought that the beauty of SOA was it allows you to deliver new applications while leveraging your legacy applications, not upgrading them! If you follow some of the links that Dana provides you'll see these quotes from the article Microsoft Does Have a SOA Strategy:

(Microsoft) so far declined to participate in certain key emerging industry standards relevant to SOA.

The more vocal critics claim Microsoft's approach to SOA not only goes against the technical grain of competitors, but may also not be in the best interests of customers. They believe the company's approach is too tied to pushing sales of its core desktop and server products, which are more expensive, complex and proprietary than alternative offerings.

Microsoft is primarily concerned with its [own] business strategy. It wants to continue to produce these fantastic profits but that runs counter to what many IT shops are focused on, which is cost-reduction, simplification, consolidation and modernization...

Such standards are too closely tied to rival technologies and platforms for Microsoft's taste, Heffner says: "That would be a hard pill to swallow. Microsoft doesn't want to do the tools that will help people use some other platforms."

Let me say that again, "Microsoft doesn't want to do the tools that will help people use some other platforms." Think about that when evaluating vendors. Unless you have a Microsoft only shop, stay far away from this mine field! Notice that they always tend to focus on the developer and not the the developer's customers.

Everyday as I sift through the articles in my Google Reader, I see countless debates about the relationships between SOA & BPM, IT driven vs. business driven, and top down vs. bottom up. There is so much debate about these topics that I sometimes wonder if anyone is getting any work done. Glance at these headlines and ask yourself how confusing this must be for somebody who is in the research stages of SOA.

* The proper relationship between SOA and BPM
* The awkward dance between BPM and SOA
* BPM Driven SOA
* BPM Without SOA: 'Like One Hand Tied Behind Your Back
* Why BPM Screws up SOA

* Business Driven SOA
* Who's in Charge of your SOA?

* Another view: avoid bottom-up SOA like the plague
* Bottom-up SOA is harmful and should be discouraged
* Should SOA be Top Down or Bottom Up

I have mentioned it in the past and I'll say it again. There is no single answer to these questions. There are many factors that can influence these decisions such as:

• Whether you have strong executive sponsorship or not
• Your IT staff's capacity to change
• Your EA maturity level
• Your staff's talent level
• Budget
• How much time you are given to deliver
• The main driver for the initiative

These are in no particular order. I am not going to tell everyone how I think you should run your SOA projects. I will give you insight into the decisions we made on my project.

In this article I tell the story of how we got the business to support a BPM and SOA initiative. IT had been pushing this for a while but could not get the funding. Once we convinced the business to reengineer their business processes we were able to come up with the justification to buy BPM for operational efficiencies and SOA as the technology to enable BPM.

We then launched into a process reengineering exercise which produced a portfolio of a dozen or so initiatives with an extremely attractive ROI. This determined the bottom up approach for us. We were funded specifically for delivering the projects identified from the process reengineering exercise. We then analyzed the projects and identified the services that would be required to support the new business processes. We did this for each project. Then we recommended a priority order which was based on two factors:

1. Business benefits - ROI, operational efficiencies, customer service, etc.
2. Architecture benefits - Service reuse and speed to market

We performed a three week "SOA roadmap" exercise that took these two factors into consideration. There were huge advantages of moving certain projects to the front of the priority list because of the number of services contained in the project that would be shared by the other projects. To figure this out we mapped out all of the services for each project and identified projects in the portfolio that would give us the biggest bang for the buck from the standpoint of service reuse.

The other constraints we had to deal with was time and money. Each of these projects in the portfolio had to be justified individually. They each had very specific funds and very aggressive timelines. The first project which included implementing the stack and delivering a beta version of a B2B portal in 10 weeks really constrained us from a governance standpoint. For this first project it was critical that we delivered quickly to deal with the perception that SOA initiatives take forever to implement. Funding for the other projects in the portfolio were also dependent on the results of the first deliverable. Due to these constraints, we did not have the luxury to establish a governance model up front. We alerted everyone of the risks of not establishing our governance model and agreed that we would be allowed to implement our governance model for the next projects.

As we sit now, we are wrapping up the first project and getting the funding to tackle several projects concurrently. We are gearing up with our implementation partner to start establishing our governance model that will help us grow our SOA with the release of each new project.

So for us, we are building SOA from the bottom up, the business and IT are working together to drive the initiatives that drive SOA adoption. The business is the owner of this multi year initiative. We are focusing on a specific area of the business that has 20 year old processes. Other business units are seeing the benefit and lining up to launch their own initiatives. The company is changing the way they think because of BPM and SOA. Twelve months from now we will be a different company because of this.

To sum it up, I can't tell you how you should address top down vs. bottom up, BPM driven SOA or SOA driven BPM, business driven or IT driven. I don't think there is a silver bullet. I believe the decision should be based on the environment you work in and the constraints that you are faced with.

I would love to hear from others who have been down this road.

Back when Jaws was still considered a scary movie, the mainframe dominated the hardware marketplace. Well, Just when you thought it was safe to get back in the water, the mainframe or at least the mainframe mentality, is coming back.

Virtualization is as hot of a topic as BPM and SOA these days. Companies are saving millions of dollars by consolidating hundreds or even thousands of individual servers onto small clusters of servers serving up virtual machines. Other drivers for this technology are reductions in energy, emissions, and floor space, improved manageability, and easier disaster recovery strategies.

The Butler group published an article called, "The King is Dead - Long Live the Mainframe". If you have the time, this article is a great read. Here is a quote from the article:

We believe the wider adoption of the mainframe beyond these markets will be influenced by developments in the Service Oriented Architecture (SOA) paradigm, and the impact that the advancements in the capabilities of x86 server virtualisation is having in the market.

One of the many reasons for the decline in mainframe usage over the years is the lack of products that are available for the mainframe platform. This is changing as Linux can now be the OS of choice on the mainframe. The article continues with this quote:

Another argument against mainframes has been the lack of commercially available software developed on the platform, which at best tends to be ported to the system at a later date, or not at all in some cases. This has created the ‘inhouse’ or customised solutions that have become associated with many mainframe implementations. However, since IBM announced support for Linux on its Z series this has become less of an issue.

But even if companies are not considering mainframes as a platform for virtualizing their enterprise, one can't help but see the resemblance of today's virtual infrastructure with the mainframe infrastructure of the days gone by.

As I continue to research the virtualization movement, I keep stumbling across articles that point to various issues and challenges with virtualization. These range from security to inadequate monitoring and managing tools. When companies like VMWare and Open Source solutions like Xen resolve these issues, won't these solutions closely resemble the mainframe? If you think about it, the virtual server concept is basically the same thing as LPARs. The architecture behind the mainframes of yesterday are starting to look very similar to the architecture behind virtualization today.

IBM is using this opportunity to revitalize its mainframe sales. Most of their sales in recent years can be attributed to the fact that companies cannot afford the cost to migrate off of the years of legacy built on top of mainframe technology. Now, IBM can leverage the new mainframes running Linux as a solution to virtualization and Green IT initiatives. And by the way, the are eating their own dog food too.

IBM saves $250 million consolidating Linux servers on to mainframes

I have been thinking about writing this post since I wrote this article about Second Life. As I was getting my daily updates from my Google Reader I came across Nick Malik's article called

Using Massive Multiplayer Online Concepts to Build a Shared Architecture

In this post Nick writes:

What I haven't seen yet (and perhaps it is the nature of the child-like games my kids play) is a MMO game where every person plays a role to build something instead of defeating something. It is easy to tear something down. Divide and Conquer. Building something up is much harder.

Welcome to Second Life. If you still are not up to speed on how Second Life is being used in the business world you must read this article called Alternate Universe. As I continue to follow the success of Second Life I often wonder why is it that Linden Labs can create such a successful architecture that runs a virtual world with over 8MM users world wide while most companies struggle to create an architecture to support the real world?

So I did a little homework and as I suspected, Linden Labs leverages open source technologies like MySQL, Apache, Squid, Mono, uBrowser (for online help) and Debian as its choice of operating systems to run its grid technology. Linden Labs provides developers with APIs that are simple REST style web services. They have their own proprietary XML format (called LLSD) that supports Perl, PHP, Python, and Ruby. They have a very robust portal and wiki which uses Mediwiki, another open source product.

The beauty of this architecture though is not the open source technologies that they have chosen but instead is the use of open standards which allows them to create a platform for others to build from. People can contribute to this platform as C++ developers, they can use Second Life's scripting language, they can leverage a host of third party tools and services, or they can simply use the tools provided by the user interface. This is a very open environment, much like many of the successful open source projects that exist today.

Now back to the topic of architecting the real world. As architects within an enterprise, we should take the time to review the benefits of the Second Life architecture and apply it to our work. Here are some of the obvious benefits:

1. Open; supports multiple development platforms
2. Easy to use; simple interface & documentation is easy to find on the wiki
3. Scalable; scale up by adding more nodes to the grid
   
4. Cost effective; use of open source reduces high cost of software licensing
5. Enables users; users are free to innovate and are not restricted by the architecture
6. Extendable; other companies can leverage platform to drive more revenue inward

Isn't this what we should be delivering in the real world? Do you think they spend a lot of time arguing over process? Do you think they spend a lot of time gathering user requirements? Or do they have a vision and know how to anticipate user needs? Have they tried to handle every possible exception in the world or did they create a robust set of standards that do not restrict usability and functionality?

There are still many people who are unaware of what is happening in the virtual world and others who are not taking this movement seriously. We just may be looking at the future of business where individuals and businesses do most of their commerce in virtual worlds. The architecture will definitely support it. Whether virtual worlds are the future or not, we should at least study the underlying architecture and the interactions that take place in their open development community.

I stumbled across this webcast that contains a 26 minute video explaining Microsoft's virtual licensing. When you need to spend a half hour just trying to figure out how you are getting shafted on licensing then it is time to look for alternatives.


Video: Licensing Microsoft Servers for Virtualization

Oh, and by the way, there are nice alternatives to VMWare also.

I started blogging in March of 2007 and have enjoyed the experience thus far. I have learned a lot about writing, received valuable feedback (both positive and negative), and have built a great network made up of very smart people. Heck, I even had a few people approach me about job opportunities out of the blue. The purpose of this post is to share a story about why I started blogging.

Back in February, I ran into an old friend of mine named Mary. Mary is a retired professor who co-authors a book on database management. She was telling me about her retirement and how she and her husband spend 6 months of the year driving their RV across the states. Every other year she writes a new addition to her book which is used in several graduate programs. I asked her how she was able to keep up with technology when she is on the road. Her answer....blogs. In her earlier editions of the book she spent a lot of time and money interviewing technology folks across the globe. Today, she relies heavily on blogs. As she travels the states, she stays up to date on emerging database technologies and case studies using her broadband mobile satellite dish. The blogs allow her to create a powerful network of top notch professionals whom she reaches out and collaborates with from the comfort of her RV.

Before I had this conversation, I read blogs but not religiously. On the way home I starting thinking about the conversation with Mary and came to the conclusion that there was a lot more value to blogs then I was aware of. Over the next few days I started researching blogs and seeking reasons why people choose to blog. Then I started studying blogs about enterprise architecture and was amazed to see so many relevant conversations happening in cyberspace. So I started subscribing to relevant content using Google Reader. In the past, I would buy and read book after book from Amazon to learn as much as I could about the topics that interested me. The down side of books is that they are based on the past where as blogs are based on the now. On occasion, I was lucky enough to attend some conferences where I would talk to as many people as I could to share lessons learned. The problem with conferences are two-fold. First, there is too much bias because of the amount of people who are associated with vendors and second, a lot of people there don't really have a clue and are just chasing the technology of the month.

After only a few weeks of reading blogs, it became real apparent to me why Mary was able to continue updating her book by the use of blogs. There are a lot of great minds out there sharing their experiences, refining other people's ideas, and giving writers feedback. Then I stumbled across a post where an architect described the reasons why he blogged (I can't remember who the author was). There were many valid reasons but one stood out for me. After a year of blogging the architect was able to look back over his posts and view his journey over time. That was intriguing to me. I was in the middle of launching a project that is transforming my company. We are implementing BPM and SOA. I thought that it would be a great idea to capture my journey. I also felt that I had a lot of good experiences that I could share with those who read technology blogs. So I started writing.

I have been writing for five months now and I have benefited from it in many ways:

1. It has taught me how to write better.
   
2. It forces me to research topics thoroughly.
3. I have built a network with some top notch people in the industry.
4. It allows folks at my work to see what my thoughts are on different topics (I lead the architect group)
5. I get feedback (good & bad) on my thoughts and opinions.
   
6. I am able to see other point of views. People tend to challenge ideas more readily in blogs then they would face to face.
7. Most importantly, I learn from other people's experiences

The last bullet is extremely important for me. I have been at the same company for 12 years. I have a perspective of the impact of decisions over a long period of time. Many bloggers out there like my friend Eric Roch, are consultants who have experiences that cross many companies. I have seen a company grow from an IT shop of 30 to a shop of 200. I have a ton of lessons learned on how to grow and evolve an IT shop. Eric gives me the perspective of how different companies have tackled similar issues (both successfully and unsuccessfully). This is one example of how blogging has helped me build a valuable network that I was lacking because I haven't changed companies in over a decade.

I get so much value out of blogging that I kick myself for not knowing about it earlier. I am now on a mission to bring Enterprise 2.0 technologies like blogs and wikis into the workplace so that our shop can share similar experiences within our own walls.

The architects who blog are an interesting group of individuals. They are passionate about their topics and many are extremely opinionated. We tend to argue over definitions, methodologies, and approaches. In reality, there is no right or wrong answers. The "right" answer often depends on a variety of factors including company culture, budgets, leadership, and the talent level of the team. With that said, I still get a lot of value from the back and forth arguing because I get to see things from various perspectives. One of my favorite articles was about how we successfully sold BPM & SOA to the business. Computerworld.com, IBM, and a few other web sites linked to it. On the other hand, James McGovern had this to say:

Sadly, I hate when other enterprise architects give out bad advice such as selling SOA to the CEO by tying it to cost/benefit when in all reality, a good enterprise architect would know that the CEO should stay focused on business strategy while his/her deputies should stay focused on business architecture. At no time should SOA be sold outside of IT. If anyone tells you otherwise, run in the opposite direction.

Like I said, what works at one organization, may not work at others. In my case, the CEO has claimed that this is the top priority in the company and has backed it with funding. Sorry for the "bad" advice. I do love James's blog though. He has a built a rare brand that combines technical expertise with an in your face delivery style. I am always happy when my posts make it by without showing up on his site!

I write about the technologies that I deal with at work like BPM, SOA, Virtualization, and Portfolio Management. I often write about controversial topics like open source and offshore development. Both of these topics tend to get people animated and generates a lot of conversation. This is also a big benefit for me. If I am trying to sell the value of open source at work, it is helpful for me to understand opposing views. I sure get those views from the comments on my blogs. The same for offshore development. Whether I like it or not, my company has made a strategic decision to offshore certain projects. It is my job to make it work. I have written various posts about this topic which is a great way to attract opposing opinions. Understanding opposing views helps me understand the type of resistance that I might face at work.

To wrap it up, blogs are changing the way the world gets information. We no longer have to be force fed information from vendors and paid analysts. Now we can collaborate with experts across the globe. Here are some of my favorites:

Andrew McAfee - Associate Professor at Harvard Business School
Michael Hugos - His blog on Computerworld
Nick Malik - Inside Architecture
Eric Roch - SOA Blog
Don Hinchcliffe - Enterprise Architecture
Todd Biske - Outside the Box
James McGovern - Enterprise Architecture: From Incite comes Insight
Dana Gardner - FASTForward Blog
Joe McKendrick - Service-Oriented Architecture

and my all time favorite...

Athena's Blogs about Cats and Dogs - My 8 yr. old daughters blog (story here)

Virtualization is another hot topic in IT today. It allows you to run multiple virtual or logical representations of physical machines, side by side on the same physical server. Many companies are putting strategies in place to consolidate servers and to reduce the cost of hardware, maintenance, power, and floor space. At my work we have a cluster of physical machines using VMWare that have allowed us to run 115 virtual servers on it. There is room for more and we will continue to move to that platform. Virtualization makes a lot of sense in the eyes of the system administrator.

So why is a software guy like me so excited about it? To me, it is much more then a way to control costs. It also allows us to be more agile, to react to capacity issues faster, deploy easier, and create faster applications. Allow me to expand on each one of these points. The following statements assume that you have a cluster of boxes with available capacity to add additional virtual machines.

Agile - How long does the procurement process take in your company? I have seen companies' server procurement cycles take from two weeks to six months. The norm in my world is about a month. A virtual machine can be created in 15-20 minutes without having to purchase anything.

React to capacity issues - On Friday at 10pm, your web site generates a ton of unexpected traffic which causes your servers to exceed their capacity and fail. Your systems administrator logs on from home and quickly adds five more virtual machines using a snapshot of an existing production virtual machine. In under an hour he has the system back up and running.

Deploy easier - Your application can be configured as a virtual appliance. This allows you to create a virtual machine that consists of the operating system and application software for all the tiers of your application and bundle it as a single file. Microsoft is leveraging this approach as a way to allow people to try out the latest version of Outlook without having to install hardware and software. Think about being able to deploy your multiple tier environment as a single file. Take that snap shot and deploy it at your DR site and you have knocked a huge chunk of time off your implementation phase.

Create faster applications - This is the topic that gave me the urge to write this post. Think about this architectural strategy for your web sites. Put all of your web servers, application servers, and database servers on virtual machines and create your virtual appliance. Since all of these virtual machines are on the same physical server, you could actually leverage the PCI bus instead of the Gigabit Ethernet. The PCI Bus is over 400 times faster then Gigabit Ethernet which should give you incredible response times for your application. What's even cooler, is that you can have your virtual web servers "sitting" outside the firewall in the DMZ while your application servers and database server is "sitting" inside the firewall. You read that right! Even though these virtual machines are physically installed on the same server, you can configure them to logically be located in different areas.

Think of all the possibilities. We all know the power of distributed systems. Distributed systems allow you to perform parallel processing across multiple nodes to achieve more work in shorter intervals. The downside of distributed systems is the cost of the physical servers and the management of software across the nodes. Enter virtualization. You can install several virtual machines at minimal cost and easily manage the software by updating one virtual machine and taking a snapshot. Then apply the snapshot to the other virtual machines.

To take that concept one step further, you can leverage this distributed model to create multiple virtual machines to act as an application server farm. If you are implementing an SOA and have an ESB in place, you can configure the ESB to load balance your services across these virtual machines to maximize your throughput.

The possibilities are endless. The point is that virtualization is more then just a great solution for consolidation and server management. Virtualization should be included as part of your application architecture strategy.

I stoled or as architects like to say, "reused" the title of this post from an interesting article in Computerworld by Steve Duplessie. The article starts out talking about how Web 2.0 will change our world and finishes with an example of a stupid business decision, hence the title, "You can't fix stupid". On the second page Steve has a point that gave me a reason to blog. Here is a quote from the article:

Free means crap, right? Wrong. How many of you keep Yahoo mail running on your BlackBerries along with Exchange because you know Exchange will be down sooner or later? When is the last time Google Mail was down? So my enterprise-caliber application running on an enterprise-caliber infrastructure with enterprise-caliber IT talent is less reliable than my free e-mail? Yep.

The "free means crap" perception is still shared by many people these days. Steve's email scenario is one of many real life examples that should help start putting this misguided perception to bed. I struggle to understand how smart people can refuse to at least investigate the possibility of leveraging Open Source software or Google Apps.

Many people are complaining about IT jobs moving offshore. One of the main reasons this is occurring is because IT is being asked to keep their annual budgets flat or even reduced year after year. This is extremely difficult to do when health insurance costs are going through the roof and the IT maintenance costs increase every year as we purchase additional products and services. The "easy" answer is cheap labor. Maybe it is time for an Open Source strategy. Look at how much money we are paying for Microsoft Office and Outlook. How much does it cost to purchase and maintain software like project management tools (project scheduling, portfolio management, requirements management, defect tracking, etc.)? Heck, their are even enterprise packages in the Open Source community for CRM, SOA, and BPM to name a few. Folks, there are alternatives out there that could save big bucks.

Does free mean crap? There are tons of quality free or inexpensive solutions in use by major corporations today. Let me ask you this. Does expensive mean good? Not necessarily.

I have written a couple of articles (here and here) recently about offshore development and how the success or failure of offshoring depends heavily on the US companies' ability to manage their offshore development partners. I may have come across as a strong supporter of offshore development, which was not my intent. My intent was to point out that if you are going to send work overseas you better have good processes and controls in place to manage it.

This takes me to my next point. I am a huge fan of agile development. One of the many things I like about agile development is that it does not require you to spend countless hours analyzing and documenting requirements and designs. Instead you cycle through many iterations of requirements, design, and prototypes with your key stakeholders and deliver functionality in small chunks.

Now lets look at the offshore model. Because of the communication barrier, the lack of business knowledge offshore, and often the limited experience level of the development team, the amount of documentation required to make this work increases immensely. Without the "war room" mentality of agile development, the projects tend to lean more towards a waterfall approach. Waterfall projects are rarely discussed in the same breath as "speed to market".

On my current BPM/SOA implementation, we have about a dozen projects that we wish to implement in the next 12 months. As we work with our SOA consulting partners (a US company with an offshore partner), we put together two different pricing models for our executive sponsor for each project. One price is for the most cost effective approach which leverages offshore development resources, and one is for the fastest to market approach which requires more on-site collaboration. So far the business has only chosen the speed to market solutions. Why is that you may ask? Well, all of these projects have a huge ROI if we replace our 20 year old business processes with new streamlined processes. Every day that we continue to do business with our legacy processes we are wasting valuable corporate dollars. For this reason, our sponsor is willing to pay a premium to get the work done quickly.

So if speed to market is your goal, agile development could be your answer. If you want to be agile and deliver functionality every 10 weeks or so, you must have great collaboration amongst your team and should not follow an extremely rigid methodology. In my opinion, this is when you should not rely on offshore development.

I came across this article in the WSJ where companies are testing job interviewing in the virtual world called Second Life. Huh? I thought this was another gaming environment where people could spend countless hours wasting their life away. So I Googled Second Life and looked for more business uses of this virtual world. There where way too many articles about virtual sex but I did find a few articles like this one about virtual classrooms in SL and this one where Dell set up Dell Island. As my research continued I stumbled across one grassroots initiative who started leveraging SL for streaming video of keynotes and allowed you to attend breakout sessions. Another article from Brandweek discusses marketers challenges as they try to use SL to promote brand awareness.

Perform this search and you can see countless businesses dabbling in SL ranging from recruiting, advertising, educating, collaborating, to countless other innovative ideas. Then I found this article that shows how the Weather Channel, IBM, and Cisco are leveraging SL to advertise and create brand awareness.

All of this is mind boggling to me. I can understand people using SL as a gaming environment. But for someone like myself who has a full time job, a family, and a life, how can you justify spending countless hours "conducting business" in a virtual world? As the head of the architecture team at my work, it is my job to learn about all emerging technologies. I am constantly studying topics like Web 2.0, Open Source, SOA, virtualization and best practices such as change management, project and portfolio management, and IT governance. I feel obligated to take a deeper dive into SL but after walking around SL aimlessly for 30 minutes I decided to let others do the research for now and I'll stick to reading articles on this topic for the near future. Besides, it is hard enough to get new initiatives going in the real world. I can't even imagine selling management on the virtual world!

I would love to hear from anyone who has some real experience using SL for business reasons.

Nick Malik has a great post on this topic. Here is how he defines the terms:

1. Top down - central group decides everything and the dev teams adopt them.
   
2. Bottom up - central group provides a directory and dev teams make whatever services they want. Dev teams go to the directory to find services they can use.
   

I am in the early stages of implementing a SOA. About five years ago we embarked on a project to create a component based architecture where we isolated the business rules as business components. Back then, we did not have a central group focusing on architecture. I was driving this initiative without the necessary authority to make it an enterprise solution. We successfully create a repository containing many reusable components. Another team bought in to the idea and started architecting their application in the same manner. In true Bottom Up fashion, they chose to create their own component library for their application specific business rules. They were able to reap some of the benefits of this approach but because we had no authority to govern this project we wound up with two separate repositories. This speaks to Nick Malik's warning that "Bottom Up SOA is harmful and should be discouraged".

Now, five years later, we do have a centralized architect group with the necessary authority to govern our SOA initiative. We will use a Top Down approach to ensure we don't relive the mistakes of our past. This works for us because we are a small shop (about 200 IT staffers). Development is done in three business units, two at corporate and one in Europe. Our approach is that the architect group controls our repository of objects and services. All teams can access the repository in read-only mode and submit candidates for consideration to be added to the library. If the artifacts meet our architectural standards then they will be added to the repository.

I believe Top Down SOA will work for us because of our size and because there is a limited amount of staff who have experience with SOA. If we were a large shop with several different groups actively working on SOA then this might be a major challenge. Bottom Up is a faster way to market but can lead to maintenance nightmares if any group thinks departmental instead of enterprise.

I would love to hear from those who have experience in either Top Down or Bottom Up SOA.