Business Impact Analysis for Louisville Corporate Data Center
Business Impact Analysis (BIA) is designed to determine the value of an asset the company wishes to protect. In the case of business continuity projects such as this one, the BIA estimates the cost to the company of a disruption that results in the company not being able to do business. It is difficult to estimate costs associated with computer down time but be assured there is a cost associated with every minute a business critical system is not available. These costs are tangible and intangible, obvious and hidden.
Our BIA started with the identification of the business areas that access information from the Corporate Data Center (CDC). The identified business areas and the people interviewed for Louisville CDC are:
- Customer Service Sue Cole, Judy Dunn, Gary Hinton
- Finance Kevin Fairman, Ann Voigt , Ruth Beadman, Sue Sniff
- Marketing Katherine Ott, Erik Norlander
- Sales Ed Glova, Mike Sutton
- Manufacturing Jeff Begalle, Gary Wickett, Jim Craig
- Logistics Ken Miller, Alan Weappa, George Sypher, Jim Matney
- Service Management Chuck Hathaway, Travis Waters, Jennifer Oden
- Product Data Management Carole Fitzsimmons, Donna Wicker
- Engineering Development Bill Farnum, Charlie Farel, Jim Cely
- Contracts Ken Pliska, Michael Loquercio
- Legal Jeff Dumas
- Human Resources Patti Nelson
- Quality Assurance John Woods
- Public Relations Terry Monrad
From these interviews monetary values were associated with the business assets supported by these systems, and translated to the cost of in-operability of the system. As these costs are educated estimates it is important to focus on the magnitude of the cost and not the exact value.
BIA ConclusionsGeneral
The business assets to be protected are approximately $120 million in revenue per month that is based on US revenue of $1.4 billion annually.
The business groups identified the following systems as important for the success of StorageTek:
- SAP Finance, Accounting, Customer Information, Manufacturing, Logistics
- Email both Outlook and Unix
- Web Intranet and Internet
- Network connect
- Clarify Customer/product information
- Pinnacle customer and development problem tracking
- Hyperion Financial consolidation
- OFA Financial analysis
- LPA/DRP logistics
- OSW Reporting
- Resumix resume tracking and new hires
- Trim training
- CTMS/CSTR timecard entry
- Brio reporting tool
- Trilogy commissions
- Siebel Sales tool
- Seritrack re-manufacturing system
- Livelink specification reviews
StorageTek with its size and position in the IT marketplace, should be a leader in the industry for high availability (HA) recovery. With competition such as IBM, EMC, and SUN, if key systems are unavailable, it is the same as if the light is out and no one is in the shop. Across the street, so to speak, the light is on all the time and business will flock to those "open" businesses. Sunguard Recovery Service's CEO, Jim Simmons, a leader in the Business Continuity field for over 22 years, says "Im here to tell you that the recovery window is no longer 48 hours or 24 hours or 8 hours. Today, the recovery window is officially closed .the acceptable window for an outage has been reduced from days to minutes to, effectively, nothing". As I listened to the StorageTek business leaders reflect on the importance of their data to their everyday work, these statements by Mr. Simmons ring true. Not one business leader hesitated on the high cost of embarrassment when our systems are down and they are unable to respond to customers.
Embarrassment
StorageTek is a data storage company. Customers and partners trust that StorageTek products will reliably protect their data. Some managers interviewed said they believed customers might understand if StorageTek had a system outage for 2-3 days for a major event. However, other managers felt that StorageTek would suffer an immediate impact if our IT systems were unavailable. All agreed however, that the company faces an enormous potential for embarrassment (over a million dollars) and image erosion if long term IT system outages prevent us from doing business. Jim Cely, engineering director, said there would be a "big time impact that would start the moment StorageTek has to admit to a customer that we've lost data that prevents us from delivering or developing our products."
Timing
Most business areas had a critical timeframe. However, each area had different timeframes within the month/quarter. For example, Finance needs their system to close the books during the 3rd and 4th week after the end of the quarter. This could also change when they go to closing the books every month-end. Order fulfillment has a critical timeframe of 2 weeks before quarter-end and on holidays. The critical time frame for Accounts Payable is daily with savings of over $4,000 in discounts if checks are mailed daily. In Logistics with no supply of parts in Puerto Rico and only one week in Louisville, every day that parts cant be ordered and shipped in time is another product not built for shipping to a customer. Every day is a critical day for our business. You never know which day an event will bring down the system.
Productivity
Loss of productivity could be much higher than most people would think. Most areas said that productivity loss would be immediate if the IT systems were unavailable. For example, John Woods said that in his group the productivity goes down 10% the first hour and 70% by the 4th hour. This is probably a typical scenario where the work group could find work for a while but would need their IT systems shortly in order to continue to work. If you take 10% the first hour by 3,000 employees at a cost of $30/hour, StorageTek is losing $9,000 for the first hour the systems are down. If you take a 4-hour outage, the cost multiplies to $63,000 per hour! Of course if only one system is down, then the cost decreases by how many people are affected. Therefore, the more people affected by one application, the more it costs in productivity loss. Email, SAP, Network, and the Web affect the most people at the same time and therefore would cost the company the most in lost productivity.
Looking at the interdependencies of business units, one unit will affect at least one or more other units. The domino effect exists where if one business unit is stopped due to an IT outage, many other units may also stop due to that units inability to deliver. For example, if logistics cant order parts, manufacturing cant complete their units for shipping to customers. If sales cant enter a sales order, shipping cant send the product to the customer and accounts receivable cant send a bill to collect income for the sale. Another example is if customer service cant log service calls, then field service cant be dispatched, engineers dont have a trouble ticket to fix the problem, logistics cant send the needed parts to the customer site. The list goes on and on.
SAP/Clarify
SAP and Clarify are used throughout the company and a BIA for these systems was performed about 9 months ago. Since the original BIA for SAP and Clarify additional information was obtained from Sue Cole, Judy Dunn, Travis Waters, and Jennifer Oden. To briefly summarize the business impact of SAP and Clarify, they are the most important systems in the CDC. According to Sue Cole in the Customer Service area, SAP has a 4-hour window of minimum impact to the business and Clarify has 7 minutes. Travis Waters, Director of US Shared Services, indicated that in one day the impact loss could be as much as $4 million. This loss would be due to accounts payable not receiving discounts for mailing payments on time, no invoices mailed, and cant enter orders which would delay shipping, just to name a few. Jeff Begalle, Manager in manufacturing said that manufacturing would stop after 5 days. The sheer quantity of departments and personnel that depend on this system alone would justify the criticality of the system. For additional information on SAP/Clarify see attached presentation.
Email is not just a method of communication between two people! Email is an integrated part of todays business. Email is used by every employee and allows business to work at a faster pace. At StorageTek email is used for many key business processes. For example some of the comments from those interviewed were:
Transfer order information between ICOA and LPA
Audit trail. Need 24 hours due to time zones for quarter-end close information.
Send documents through email both internal and external
Fiserv transactions through email
Marketing sends huge graphical and video files through email
Distribute master schedule and forecasts, large distribution of reports
Extremely important in commissions. Distribute commission information on sales. Print emails for hardcopy audit trail.
Emails to customers, Siebel emails through system, send orders to order processing.
Link to IBM for microcode changes. Used lots in Marketing
Absolutely critical to send large files to field.
Some might say there are workarounds for email. Our business leaders say no. US Mail, phones, faxes, etc. are not able to meet the business need. Email is the only method capable of sending large files, graphics, documents, and videos with acceptable speed and automatically interfacing with computer applications. There are not sufficient personnel to even attempt using any other method. Company embarrassment is also a factor since everyone these days expect your email to be available and these goes hand in hand with our ability to respond to our customers. Email is a key business tool as well as a communication tool in todays world.
Web
The Internet and Intranet are not far behind email. Again the Web is used as a communication and business tool that is integrated with our business processes. Today, email far exceeds the Web but the trend is shifting more and more to the Internet. The Web today is used for field services to list part shortages and parts availability, earning release information, research, competition information, employee HR forms, ISO processes, knowledge map, shared knowledge of projects, sales contracts, engineering information, microcode distribution and customer presentations. The Web is also used today by StorageTek partners to get product information and configurations, and for customer support and software updates (PTF distribution). Tomorrow the Web will be used for E-Commerce to sell our products and services. The Web could become the most important application in the company. StorageTek is currently investing $10 million to upgrade our Web sites and content. Executives expect revenues to average $1 million per day next year. To emphasize again, "Today, the recovery window is officially closed."
Phone/Printers
Phones and printers are utilized a lot less than in the past before the PC and email but are still critical in some areas. The business units that have prioritized the phone are: Customer service/technical support areas, Legal, Public Relations and Sales. Most of these areas have workarounds for campus phone outages. Areas where printers are critical indicated that they could do business for at least one day without the printers. This too has workarounds of taking print files offsite to print.
Regulatory fines and legal requirements
There are some regulatory fines and contracts but these are not a significant impact on costs. Most service contracts are written such that the companys exposure is a percent of monthly uptime. Most Var/Vad/OEM contracts are vague and state that there could be legal action for non-compliance. Potentially this could have a huge impact but no one interviewed had a clear picture of the business cost in this area.
Bottom line
From the BIA, the following applications/systems are categorized by criticality for IT to meet the business needs.
Most critical systems:
Clarify
SAP
Pinnacle
Siebel
Network (LAN, WAN)
Critical systems:
Web
Seritrack
Hyperion depending on time of quarter
LPA/DRP
Trilogy
Phone
Key systems:
OSW
Resumix
Brio
Livelink
CTMS/CSTR
Printers
The criteria used for the application distribution into the categories were:
Cost to the company if the system were unavailable
Was there a work around what could be used
The more areas using the same systems, the more impact that system had on the entire business
When does the unavailability become a cost factor
Does the application unavailability affect customers, vendors, suppliers
Engineering Development
Engineering business areas were interviewed for this BIA even though they were outside of the scope of this project. The engineering groups do not process through the CDC. However it was important to gain some insight into how they do business because IS is responsible for their support. StorageTek engineering groups, especially Software Engineering, are heavily dependent on systems that IS maintains. All of their source code is on mainframe, Unix, or NT servers. If these systems go down, particularly at critical points in the development cycle, schedule slippage quickly becomes hurtful to the business. Management's more immediate concern is that customer services engineers would not be able to fix some customer problems with StorageTek products if they did not have access to the source code. The most critical systems for engineering are: SCM systems that contain product source code on the mainframe and servers, Mentor (electrical designs), and ProE (mechanical designs). Bill Farnum, VP of Disk Product Quality Assurance noted that most of their processing is done in the data center in Bld 2. Therefore if the CDC were unavailable they would not be affected. If bld 2 was unavailable he believed that they could continue their work on the systems in bld 4. Jim Celys group processes in Bld 4 the ECC lab but could use bld 2 if this data center was unavailable. This will need to be further investigated at a later time.