MIT CIO Symposium - 2013 Synopsis

MIT Sloan has for 10 years held a “CIO Symposium.”  This is an opportunity for thought leaders in industry and academia to compare notes and contemplate the future.  This year’s theme, “Architecting the Enterprise of the Future.”

My synopsis of the one day session? IT is DEAD.  The CIO title often referred to as the Chief Information Officer (not the Chief Investment Officer of financial services firms) really means Career Is Over.

Many things in IT have died over my career.  The mainframe was declared a dinosaur in the 70s and 80s as “distributed computing” took over.  Companies like Prime, Data General, and Digital Equipment Corporation were poised to dethrone International Business Machines (IBM) as the leader of computing.

Who remembers Prime, Data General, and Digital Equipment Corporation (DEC)?  Each of these companies was a once proud Massachusetts computer company, and IBM thrives.

I left the session with a distinct feeling of coming from a wake, where memories of the dearly departed were shared.

Sage advice like “IT needs to align with the business” has been a mantra for decades.  “IT needs to speak in business terms and not IT terms” is an evergreen comment.  I was live tweeting from the audience, and at one point was challenged by a friend:

@glkelley #mitcio this is not news. See “Undercover Boss”. IT should doublejack with a phonerep, process a backoffice tx, use the website.

— Ray Chang (@rjchang) May 22, 2013

“The Use of Power and Influence During the Process of Innovation” was a theme of an early session. 

My tweeting shows some good soundbites, and perhaps nothing new:

@glkelley Really like #HIPPO. Might start using the term more often!

— SearchCIO.com (@searchCIO) May 22, 2013

“Big data” got a bit more of a reaction:

Indeed. RT @glkelley: Big data is a Management Revolution. #MITCIO twitter.com/glkelley/statu…

— Bob Madaio (@bmadaio) May 22, 2013

And some reaction was bittersweet:

 

As an IT professional, do I think IT is dead?  No (although I did tell my son not to get into the business, and he did anyway.)  IT needs to evolve or perish.  (This isn’t new either.)

IT is seeing a repeat of the mainframe death knell because so many business lines can now “do their own thing.”  Salesforce.com has revolutionized the way sales teams work, and business lines feel empowered.  Are there issues around privacy and “islands of automation?”  You bet.  And many organizations are balancing agility with regulations and are being successful.

I’m going to submit “islands of automation” was one of the cries in the era of Prime, Data General, and Digital Equipment Corporation.  It’s déjà vu all over again. 

What drives business lines to do these projects is IT is viewed as an obstacle, not an enabler.

Georgia Papathomas, VP and CIO of J&J Pharmaceuticals, had some great insights in a session, “Driving Innovation and Managing Expectations:”

• “Everything in IT takes too long,”
• “There is only a business strategy, no IT Strategy.” It’s an IT roadmap at J&J.  This is a subtle powerful shift.
• We are using technology to evolve to a Health Care company, from a Product Company.  And that doesn’t mean it’s IT leading the charge.

The session “Strategic Agility through IT: Harnessing the Convergence of Data, Analytics and the Cloud” offered some jewels like:

• Agile methodology not just for IT; for business unit as well (somehow that doesn’t sound like business speak.”
• “It used to be work had the best technology, and not home. It’s switched. And the people managing tech need to understand,” offered Michael Relich, EVP, CIO and Strategic Planning for Guess
• Sharing data between private clouds (like fraud data) is a large opportunity for companies

The award for the best stream of soundbites in the session, “The Evolving Cloud Agenda” went to Scott Blanchette, SVP of Information and Technology Services at Vanguard Health Systems:

• Security and Privacy regulations most significant barrier to cloud
• We put the appropriate (security) wrappers around data in the cloud.
• Buying equipment is cheap today; even EMC. (Crowd knowingly laughs)
• Don’t ask your barber if you need a haircut yielded  don’t ask a hardware vendor if you should go cloud from @ValaAfshar
• Processing speed is accelerating faster than collection of health care data
• We put governance in place for people who come up with the best ideas.  “The best ideas will come from someplace that we don’t expect.”

My view of the best session, the one giving me the most reason for hope, was put on by Andrew McAfee, Principal Research Scientist at the MIT Sloan School of Management.  Sadly he was on at the very end.

He posits the steam engine propelled humankind as first machine age and we are on the precipice of 2nd machine age.  He sees the following CODE:

• Cyborg - new man/machine combinations will propel us, combining human/digital contribution
• Open - organizations will be more open. The smartest people work somewhere else. With enough eyes the bugs are small
• Data-Driven - rigorous analytical approach is driving economy. (Think a restaurant owner can identify theft)
• Evolving - how quickly innovation is happening and disruption happening.

Whether Information Technology stays in the “C” suite or not, it is clear IT can study the past, and use this knowledge as a way to leapfrog (and not simply repeat.)

Gary L Kelley’s twitter feed from the MIT CIO Symposium and other extraneous topics can be viewed at https://twitter.com/glkelley

The Importance of Staff and Shifts

In the course of our business, we see many data center/applications migrations and/or high-severity issues.  One observation we always share with our clients is to plan for staff rotation.  As you might expect, some listen and others do not. Here’s why it’s important.

Migrations often happen overnight…when the business sleeps or operates at a lower activity level.  Organizations without satisfactory disaster recovery plans often incur an outage to do a migration.  People are resilient for so many hours, and then they crash. 

What often happens in migrations is everyone wants to be at the starting line, and the adrenaline keeps them engaged.  If shifts are not “forced,” then there is often nobody left with “gas in their tank” to troubleshoot issues.  People simply have to disengage to be fresh.

We saw this at a large customer where the team had persevered, declared success, and then dragged themselves home.  There was an issue, and the on-call was unwilling to make changes as he didn’t understand the changes that had taken place (a change management issue.)  NOBODY involved was responding to calls.  As it turned out, the group’s manager lived in my town, and I got to knock on his door at 10:00AM on a Sunday morning.  His wife wasn’t happy (he had been up all night) and did indeed get him up.  While he resolved the issue, a few months later he resigned and went to work at a different company. 

In this case, the team was not structured to focus on a multiple day issue….and response was poor.

In another case, a new virus definitions in client’s antivirus system determined the operating system was bad, quarantining the operating system.  The client had a policy to delete quarantined files, so with the speed of automation thousands of operating systems were deleted.

The senior manager quickly determined this would require a sustained 24/7 response, and teams were “nominated” to cover 12 hour shifts.  We were asked to help on a sustained basis, providing process oversight and helping with crisply doing turnovers.

To the credit of the senior manager, this approach allowed a sustained response as systems we recovered from (gasp!) tape.

Large IT shops often run with multiple shifts and a technical response is more organic.  Smaller shops tend to have an operational capability 24x7, and may lack the detailed technical response.

When planning or reacting to major events, think in terms of how to rotate your staff for a sustained time.

Why Resist Process?

We’re admittedly process consultants, and have a bias towards supporting process.  That said, we see a number of instances where process is resisted.  Why?

One client asked for a gap analysis of their processes to industry standard process.  We started down the path of a binary (yes, you have the process….or no, there is a gap.)  As we got further into the engagement, we were struck by the naiveté of this approach.  While technically NONE of the processes were documented, the truth is the organization was following a process.  So a binary wouldn’t fairly evaluate status.

A colleague suggested a shift to something more of a Capability Maturity Model Integration (CMMI) approach.  In this approach, we evaluated the Maturity of the processes.  Using this approach, we were quickly able to evaluate a series of processes and their current and intended maturity. 

What we discovered was a series of processes suitable for the existing business, largely operating on the “tribal knowledge” of a series of long term employees.  Adding staff was a challenge, as this knowledge needed to percolate to anyone new to the group.  This often took time, a rare commodity in this growing organization.

That said, the only way for them to grow and leverage their processes was to document them, in simple, clear language, so others could be brought to bear.

Sometimes it seems process is equated to bureaucracy or a way of doing things in the least efficient manner.  The US Tax Code comes to mind as the working example.  Well-designed process does not need to be bureaucratic…if anything, it should be enabling.

Another client was working on the handoffs between various parts of the organization.  In (lightly) documenting the process, some “sticking points” were apparent.  We challenged the process owner to address those sticking points.  “Oh, I am concerned about rocking the boat.”

Our view is simple.  Improving process can be incremental or disruptive.  Either approach has merit and application.  If you’re not improving the business process…then you are not making a difference.

“Process” is not a four letter word.  Process is important in achieving consistent, repeatable results. 

Some organizations are successful delivering business process outsourcing (BPO).  How have they done this?  They’ve passionately tackled certain processes and optimized every step.  Certain functions lend themselves nicely to business process outsourcing (payroll as an example.)  It’s our belief non-core business functions should be considered as candidates for BPO.

Core functions should be where businesses focus their attention on optimizing process.

Legacy Systems Don’t Have to be Bad

I was reading an article, NASA’s older Mars rover notches another milestone, in USA Today and was reflecting on how we don’t do enough to praise and learn from the engineering required for systems to last well beyond their useful life.

Opportunity, the Mars rover, was scheduled to work for three months and is going on its tenth year.  In the article it points out “It has an arthritic joint in its robotic arm and it drives mostly backward due to a balky front wheel,” but also says these are “more annoyances than show-stoppers.”

We all have stories of systems operational for ten or more years.  Mine was around an HP3000 mini-computer running a trading system for an investment management firm.  It took us around ten years to retire the system because it never caused us a problem, and never became a priority.  The tipping point was the half-million dollars in maintenance we were going to be required to pay for the hardware and software.

• What makes a system last a long time?  Do we ever ask ourselves how it was architected?
• When does a system become “mature?”  What design changes enabled a reduction in support?
• Why do mature systems have fewer outages?  What is the impact of changes on stability?
• Do mature systems have the functionality needed by users or are users just compromising?  Are users capable of compromise?

What I have come to realize is we equate “old” with “bad.”  I am currently doing a consulting engagement as an interim CIO for a mid-size company.  The executive team keeps asking about the strategy around a critical system deployed in 2001.  They think they must deal with the system because it is over 10 years old.  After a few months, I have assessed there are many more systems which are vulnerable, end-of-life, and high risk.

As I examine this 10-year old system I have discovered the following practices keeping it stable and productive:

• A development, test, and production environment all of which have been properly capacitized
• Strict release practices and schedule
• Strong QA practice
• Skilled and disciplined team of developers with strong management
• A well thought-out and designed database, application, and middleware architecture

As I reflect upon my experiences with legacy systems I have come to understand well designed, architected, and supported systems can last a long time.  The more useful life a system achieves the better the return on investment and the better it is for IT.  

People Buy From People – and You’re not Ready Until You’re Ready

It’s pretty common knowledge Thanksgiving is my favorite day.  My daughter and her significant other were helping me get the bird ready.  And while I had put a filter and coffee in my old Mr. Coffee, water was still needed.

After asking Mr. Sig Other to add water and perk the coffee, it was all I could do not to laugh out loud when the poor guy poured the cold water right over the grains.  As I thought about it, he’d grown up in the age of Keurigs and coffee stores.  Old school Mr. Coffees are dinosaurs.

I could hardly blame him.  My coffee tends to come from Dunkins during the week, with a stop at a coffee shop on the weekends or in the evening (for a decaf.)  My Mr. Coffee pot was used all the time to water plants, and rarely for making coffee.

Being in a happy/celebratory mood, and with visions of fresh lattes enjoyed in front of the televised Macy’s Thanksgiving Day parade, I ran over to my local Starbucks for a Verismo machine.

I thanked the young Barista for working Thanksgiving Day, and asked for a demo of the Verismo.  The machine worked flawlessly (as it should for two hundred bucks.)  She let off a squeal of delight when I said I would take one, as it would be the first sold, it was Thanksgiving Day, and apparently she gets a little spiff for selling it.

The machine worked great all day, and everyone had a caffeinated Thanksgiving. 

I kept having issues with the used pods falling out, and discovered (when I actually read the manual the next day) the used pod storage container was missing (based on the pictures in the manual.)

No worries.  It’s Starbucks.  They’ll handle.

1. I called Verismo tech support.  They said I had to go to the store.
2. I went to the Store a few days later, and I was told I had to call Corporate.
3. Corporate was quick to give my Starbucks card an inconvenience credit, and then patched me over to Verismo tech support.
4. Verismo tech support said to go to the store with the machine.  “If they can’t help you, have them call us.”
5. After packaging the machine back up, I went back to the store, and was told there is nothing they can do.

At this point, I was getting tired.  And with my machine down I couldn’t have coffee. 

So I told the delightful and helpless Barista I was leaving the machine, calling the credit card company to dispute the charge, and telling everyone I know about the experience.

For me, that’s social media.

So I put some posts up. These posts are all linked so it didn’t take much to tell my friends to beware.   My friend Marsh Sutherland shared.

 

Within minutes, I got the attached (names removed):

Subject: Verismo

Message: Hi Gary.  This is Starbucks in Westborough.  I was unsure if it was you who had a problem with your Verismo until I just saw your status update and verified that it was you.  The policy surrounding Verismo returns is that we can’t refund a Verismo without a receipt. While my employees were simply following policy, there is a little leeway on my part, especially if I know the customer and may be able to waive the requirement of receipt. I can definitely help you with the part you need if it the part I am thinking of.  Feel free to contact me directly at my personal email address, which I provided.  I am sure we can get this resolved to your satisfaction.

The next day, I went in to the store, reclaimed my machine, left with a part from the demo machine, and shared stories with the store manager on the Verismo (her wife is pregnant, and she is looking forward to the day the little one can use the Verismo it’s that easy.)  I called the credit card company to resolve the dispute, and did another post.

All is well.

I thank you if you are still hanging in there on this long set up.  This isn’t about the power of social media.  It’s about basic corporate blocking and tackling.

While my personal machine is all set, it is clear Starbucks is having an issue.   From my perspective, Starbucks “rushed” the product to market (saying in October it would be in 65% of retail stores) without having the appropriate support structure in place.

What do I do now if a part on my machine gets accidentally broken? I can’t go into the store and hope they’ll scavenge off the demo machines.

We see this happening in IT all the time.  Deadlines loom, and the IT Service Desk is not prepared to support a new effort (be it hardware, software, process or combination.)  Yet a deadline is coming (especially year end as they are often tied to bonuses) and so something is pushed to production without a support model.  At Harvard Partners, we believe the support organization should participate in the Change and Release Management processes and be empowered to say NO if they are ill prepared to support something.

At Starbucks, this would mean a better job coordinating between Verismo support and Starbucks Store Operations (aka process!)  It might also mean delivery of a spare parts ordering website consumers can use.

Far too often the pressure to hit a deadline overrides support.  Looking back at my Verismo story, think about how much time (Verismo Support, Starbucks Corporate, Barista’s and store managers, me!), costs (toll charges, complementary drinks) and ill will was created over product management not spending a little more time to get the processes set.  You know if I had the issue, others are as well. 

I expect better from Starbucks, as I do IT support organizations!

Drinking the Kool-Aid

I have never been an “Apple Person.”  I have nothing against their products or company, but I have always (and still do) use PCs at work.  My house is in a “dead zone” and Verizon was my best option (I had to install a home repeater) to get any reception.  This meant I couldn’t get an iPhone from AT&T back in the days when Verizon couldn’t sell iPhones.  Therefore, my phones were Blackberry’s and a Motorola Droid.

When my Motorola Droid X could no longer be updated, and everything started to slow down, I knew it was time for a new phone and I decided to get an iPhone 5.

What has been interesting to me is the amount of time it takes me to personalize any device (smartphone and PC).  Getting an iPhone was an interesting test of how much time it takes to personalize and make a device fully productive.  I expected the iPhone to be easier, but what I found was it was harder than the Droid.  Here are some of my areas for personalization:

Home Screen – I miss the active icons on the Droid.  Things like a list of recent emails and my next few meetings were very valuable

Email Signature – On the Droid I can use color, links, and fonts.  The Droid allowed me to replicate my desktop email signature so no one knew what device my email was sent from.

Notifications – I like almost all notifications, with the exception of phone and text messages, turned off.  This is a nightmare on both devices.

Email Settings – I have about four email accounts on my device, and the iPhone handled setting these up much faster than the Droid.  Also, the iPhone doesn’t seem to complain about certificates and stuff like that.

Background Image – A picture of my beagle, Milo, is on all my devices.  I would rate the iPhone as being easier than the Droid for making his image my wallpaper.

Lock Screen – I hate lock screens but use them when working with corporate clients.  Turning off the lock screen on the Droid was a pain, but on the iPhone was very easy.

Adding Apps – About the same on both device and maybe a little easier on the iPhone.  It seemed, to me, that the apps on the Droid updated every few days.  Very annoying.

WiFi vs 3G/4G – On the Droid using 3G saved battery life, while on the iPhone using WiFi seems to save battery life.

Settings – On the Droid app settings live with the app.  On the iPhone app settings are under settings.  I prefer the Droid’s method as it tends to keep me focused and is a better reminder to do things like turn off notifications.

Deleting Apps – Completely annoying on both phones.  I do not understand why you can’t delete the default apps installed on the phone.  I have no need for the “Game Center” on the iPhone or “Need for Speed” on the Droid.

Overall, I do like my iPhone and am glad I replaced my Droid.  The iPhone is fast and “polished.”  When using the Droid you feel like it is a work in progress.  When using the iPhone you feel like you are using the finished product.  At the end of the day, I see this as a phone and I just want it to work.  Kudos to Apple.

IT Operations – The Unsung Heroes

This is a story of how one company and its operations staff kept the lights on in the face of post-Hurricane in New York City.

Three blogging websites are under my general control:

• Garylkelley.com – a site about technology, fatherhood and restaurant reviews
• Curriculotta.com – an “alter ego” site for the properly starched shirts over at Harvard Partners
• Markfidrychfoundation.org – a site for furthering the community work of the late ball player Mark Fidrych.

Squarespace is used to “host” these sites.  WordPress was our original choice, and served us for a while.  I was just never a fan for how WordPress “thinks.”  Personally, I prefer Squarespace.

Squarespace uses PEER1 as their co-location provider, located in New York at 75 Broad Street.

You can imagine my personal dismay when Tuesday, October 30 at 11:34AM I got the following message from Squarespace:

I have some unfortunate news to share. Our primary data center, Peer1, in Lower Manhattan lost power yesterday at about 4:30PM local time. At that time, we smoothly made the transition to generator power and took comfort over the fact that we had enough fuel to last three to four days. (Peer1 stayed online during the last 3 major natural disasters in the area, including a blackout that lasted for days.)

At 8:30PM yesterday, we received reports that the lobby in the data center’s building was beginning to take on water. By 10:30PM, as is sadly the case in most of Lower Manhattan, Peer1’s basement had experienced serious flooding. At 5AM, we learned our data center’s fuel pumps and fuel tanks were completely flooded and unable to deliver any more fuel. At 8AM, they reported that the generators would be able to run for a maximum of four more hours.

Unfortunately, this means that Squarespace will be offline soon (our estimate being at 10:45 AM today).

I then did what any IT ops person would do…and notified my users of this outage:

Of course, I then did what any user would do, and emailed Squarespace support (like they had time for me.)

Can you guys toss up a graphic of some kind so people accessing my sites won’t get a dns error?

(Also, when you’re back there is nothing to do)?

An amazingly fast 26 minutes later, I had a response:

Great question! We will have a holding page up (hosted outside of Squarespace) that will provide messaging about the downtime. Any customers trying to access sites during that time will see that message. Once we are able to bring the system back up, there will be nothing required of you in order for your website to come back online. We expect sites to be available for another 45 minutes at least and please keep an eye for updates on Twitter (@Squarespace, @Squarespacehelp) as we will be providing updates as regularly as possible from there. 

Hope this helps!

Shaun H

Of course, being rather chatty, I responded with:

I will expect something very creative….

Like an overhead view of Sandy going down a toilet bowl. (Trying to bring a smile to your face during this tough time.)

Think the Squarespace version of the Twitter flying whale.

And again, had a quick response

Hey Gary,

That’s a great and hilarious suggestion, thank you :) We will definitely keep you updated on our Twitter accounts and Blog page (for as long as we can):

https://twitter.com/squarespace

and

http://blog.squarespace.com/

Hope this helps.

Paulina V.

What then followed was something I find speaks to the spirit of a team focused on service. 

They carried fuel to the generator on the roof.  17 floors.  All by hand.  Squarespace, their co-lo provider PEER1, another company Fog Creek (an online project management firm for collaborative software development) and some hired contractors carried the fuel to the 17^th floor where it could be pumped up to the generator on the roof (18^th floor).

Ok, let’s do some math.  According to PEER1’s Meredith Eaton, a company spokeswoman, the generator’s consumption rate was about 40 gallons/hour.  That’s eight 5 gallon pails an hour.  At 7.15 pounds/gallon diesel, that’s 286 pounds an hour up 17 floors.  And they did this for a couple days…so at 48 hours this is 13,728 pounds of fuel, or nearly 7 US tons of fuel.

The following pictures are used with permission of Squarespace:

 Thirsty generator, on the roof above 17 floorsBasement level, where the fuel is supposed to be stored

Diesel fuel on street waiting for a lift

Part of the bucket bridgade

Now, many would argue Squarespace would be better off with a second data center somewhere with automated failover.  That would carry an increased cost, something this author wouldn’t be willing to pay for.  Disaster recovery desires must be analyzed in light of the costs.  It’s almost laughable to consider a Recovery Time Objective or Recovery Point Objective for these blogs.  If they are down for days, frankly it wouldn’t matter.  These blogs are not time sensitive, with the closest financial impact being on the Mark Fidrych Foundation with donation ability (I encourage you to use!)

So due to the heroic efforts of the unsung IT Operations and associated people, PEER1 stayed up, and you are able to enjoy reading this post.

My hat goes off to these people who persevered, with determination and grit, to keep the site going.  In a word, amazing.  I find IT organizations do this often to keep the ship afloat, often without complaint.

Will we continue hosting on Squarespace?  You betcha.

What stories do you have of heroic IT efforts?

One midnight shot of a total bucket brigade

Why are Consultants more Effective than Staff?

With the specific exception of consultants having skills and expertise a client needs and does not possess, we find staff can be more effective than consultants, and are often blocked from successful delivery by their own management.

Seeing this TED video makes the point about people working at work:

As a consultant I often find myself working in company cafeterias.  It allows me to “get in the flow” (or on the jazz) and get some work complete.

With the specific case of consultants having skills and expertise a client needs and does not possess, we find many staff are quite knowledgeable.

The can’t always be effective because of the M&Ms (Managers & Meetings)

As consultants we don’t have

• monthly budgets (we have project/program budgets)
• to attend company meetings
• to write/present performance appraisals
• etc.

In short, we are dedicated to a heads down function over time.

The same could happen with staff if they were able to be dedicated to a function over time.

So the question I ask managers is how can you let your staff have some dedicated time?

Meeting free Fridays is not the answer either.  These are simply used to get caught up on email.

What can YOU do to make a difference for your staff, or if you’re an individual contributor what do YOU need from your manager.

Disaster Recovery vs SHTF Planning

Maybe it is the economy, or maybe the election.  Increasingly I am seeing people talking about “prepping” for when we descend into anarchy.

From SHTFPLAN - 8 Reasons Why The Great Depression Is The Best Case Scenario

The number of websites for this is impressive…here is a sample:

• SHTF Plan - When It Hits The Fan, Don’t Say We Didn’t Warn You 
• Homestead Survival
• Cultivating Home

 There are also people analyzing where to go and publishing books on it.

Far be it for me to criticize these people.  Heck, they might be right.  That said, if you really believe the end is coming, you’d buy a piece of land and go “off the grid.”

It’s hard to fathom what off the grid would be like.  While it may be fun to flirt with the idea of living simply, the truth is the vast majority of us would struggle to do so.

Where is this rambling post going?

It’s about what we can do to make sure we are not thrown back into the dark ages. 

So many companies still believe Disaster Recovery for Systems is optional.   That’s just irresponsible.

DR plans need to be put together, and tested, so outages are minimized.  Plans need to meeting the needs of the business from a Recovery Point Objective (RPO) and a Recovery Time Objective (RTO).  This all starts with a Business Impact Analysis….understanding the impacts of outages.

Depending upon your business, the recovery plan can be very simple or very elaborate.  As a consultant, I live on my laptop and so have a backup (old) laptop.  My documents are stored in the cloud and are backed up.  Simple stuff.

If you run a trading business, your needs are for real time replication and activation.  More elaborate and certainly costly…and “cheap” compared to being out of the market during a market swing.

Do I have some MREs (Meals Ready to Eat) at home?  Yes, enough for a couple weeks as a simple storm can take me personally off the grid.  I’d be fine for a couple weeks.  So for me DR preparedness is about common sense, spending wisely, and knowing you can recover.

If the end of the world does come, let me apologize now for asking you to prepare for an event.  You would have been better off digging a hole for your bunker.

What are your thoughts?

When IT is in Crisis

We often get asked to help companies return their environment to stability.  Companies can erode customer good will and lose business with extended or recurring IT issues.

When IT organizations find themselves in crisis, digging out can be difficult. 

We’ve covered stability issues in the past, in two posts “The Insanity Must Stop” and“Change Management.”  For a company to reach out for help indicates strength.

Why strength?

Weak organizations will continue to try an solve the issue.  When “really bad,” this often takes a toll on staff.  People make mistakes when they are tired.  Engaged management sets ego aside and asks for (ultra-confidential) assistance.

A strong organization will recognize a fresh perspective is often needed to objectively understand the situation and make positive strides.

We deploy a team of experienced professionals (20+ years) with interdisciplinary perspectives, including:

• Process Analysis – to review processes for issues, particularly in the change and configuration management space (often weak change processes can cause instability.)
• Data Analysis – to gain a perspective from any ticketing system (such as BMC Remedy,ServiceNow, CA Service Desk Manager) or error logs.
• Interviewing – To collect data from the front lines. 
• Subject Matter Expert – if the client appears to be having issues in a single technical discipline, we bring a Subject Matter Expert in this discipline.

We do a time boxed assessment, limiting ourselves for a first pass.  We use our methodology of SCAN-FOCUS-ACT-CONFIRM:

• Scan – Rapid, high-level, holistic assessment identifying areas for further research. During the scan phase, conceptual models are established, risks and opportunities are identified
• Focus – Targeted analysis detailing opportunity and risk areas. Areas are scrutinized and evaluated thoroughly.
• Act – Execute action plans delivering solid results using client’s staff, vendors, and third-party consultants.
• Confirm – Assure deliverables achieved the desired business impact, reentering the process if needed.

The S-FAC methodology originated with MG Taylor’s Scan-Focus-Act model with a goal of a swift assessment, followed by a focused effort and action. Harvard Partners added the Confirm step as we believe validating results achieving the intended business goal is critical.

In a production stability assessment, we do a quick SCAN developing a list of FOCUS areas for further investigation.  We do this in a week to two weeks (of really long, completely intense days.)

At the end, we are able to produce an assessment with actionable steps to be “ACTED” on by the company, either independently or with our ongoing participation.

Can companies do this analysis on their own?  Of course, and we find companies doing it every day.  When companies need a fresh perspective to get them over the hump, external views are often vital.

Can companies spend more time on analysis?  Yes.  Lots longer.  We simply believe in staunching the bleeding quickly, and then taking more time for further analysis.

What thorny IT issues have you experienced?