• Solutions
  • By Industry
    • Distribution
    • Security
    • Gaming
    • Government
    • Healthcare
    • Manufacturing
    • Media & Entertainment
    • Travel, Transportation & Logistics
  • By IT Need
    • Proactively Prevent Downtime
    • Protect Data for Compliance
    • Achieve Disaster Resilience
    • Reduce CAPEX & OPEX
    • Provide HA in Remote Locations
  • Small & Medium Business Solutions
  • Windows Business Application Availability
    • BlackBerry HA
    • Exchange HA
    • SharePoint HA
    • SQL HA
    • XenApp HA
    • BlackBerry DR
    • Exchange DR
    • SharePoint DR
    • SQL DR
    • Virtual Server DR
• Products
  • Overview
  • everRun MX
  • everRun MX Express
  • everRun MX Extend FT+DR
  • everRun SplitSite
  • everRun HA & FT
  • everRun Advantage
  • Product FAQs
• Services
  • everRun Services Overview
  • Consulting Services
  • Training Services
    • Authorized Training Centers
  • Assurance Program
  • Customer Portal
  • Requesting Technical Support
  • Technical Community Login
  • everRun Assist Web Support
• Partners
  • Partners Overview
  • Citrix Alliance
  • Microsoft Alliance
  • ISV Partners
  • Why Resell Marathon
  • Partner Program FAQs
  • Find A Reseller
  • Partner Login
• Resources
  • Resource Center
  • White Papers
  • Webinar Recordings
  • ROI Calculator
  • Collateral
  • Flash Demos
  • Glossary
    • Symmetric Multi-Processing
    • Continuous Application Availability
    • Application Availability
    • Disaster Recovery
    • Failover
    • Fault Tolerance
    • High Availability
    • High Availability Cluster
    • Server Redundancy
  • Customer Endorsements
  • Success Stories
  • Product Reviews
  • Analyst Opinions
  • Exporting everRun Products
• News & Events
  • Latest News
  • Press Releases
  • In The News
  • Events & Webinars
• About Us
  • Overview
  • Announcement
  • Why Marathon
  • Executive Team
  • Board of Directors
  • Awards
  • Careers
    • Current Openings
    • Benefits
  • Corporate Offices
  • Contact Us

Blog

OK. We’re not being coy, even though we have a big “Major Product Announcement” box on the front page of our web site. Really. Am I going to share? Yes, but not now. Instead, I’m going to provide a bit of a drum roll and some recent feedback from many worldwide discussions with some great analysts and thought leaders.

To start off, we’re not just making a product announcement. Rather, we’re proposing an entirely new way to think about availability: one that will actually work – and work for the masses. We’re not pulling any punches. It builds upon years of experience, 14,000+ implementations, and thousands of customers. Combine that with some stunning, technological, price-performance breakthroughs and the game begins to change. It’s a direct assault on what I’ll call the “norms of availability.”

These norms have forced many organizations either into accepting a false sense of security or tolerating downtime that could have been prevented. We’ll be attacking both this week. So what are some of these norms of availability?

• Recovery is OK because it’s getting faster. Really? Go ask American Eagle Outfitters about their 8-day outage debacle. Given that over 30% of recoveries don’t go as planned, this is dangerous, especially now because the stakes are so high. “Recovery” is still application downtime and still presents a high-risk of data loss. Remember: prevention is better than recovery.
• Virtualization is availability. Actually, virtualization is consolidation. Virtual machines (VMs) fail and have to be restarted. Virtualization, though very important, is a subset of availability. Want to gauge availability risk with pure-play virtualization? Look at the other pieces such as storage. No big deal? Most organizations will disagree on many fronts, including price and complexity. Virtualization matters (and we’re fans), but availability should be thought of first.
• DR keeps us going. DR is the nuclear option. It’s a last resort when major catastrophes occur. It should NOT be used when a disk drive fails. (More on this in a subsequent post.) DR is a necessity, but needs to be combined with local protection to make a very powerful availability combination.
• We have high availability support for our applications. That used to be a good, though it's really an inherently flawed approach. Remember, it’s still a restart; it’s still complex; and - do the math - it’s still expensive.
• Cloud computing will solve all our issues. Actually, it may ultimately be a great part of the DR component of broader availability, but it’s just not going to work for localized failures. Time to think holistically.
• Backups protect us. They sure do, but it may be a day or two late. Can your business afford that? Keep your backups going, but consider other things to amplify availability.
• I’m not worried because I never had a problem. Wow. Now I’m really scared. You may want to give us a call before you get fired. During recent services work, we’ve helped many of our customers not just identify, but find their critical assets. Do you really know where your important assets are? Do you want to be looking for them when something goes awry?

I’d like to say these are tales of fiction, but they’re not. They’re part of that dangerous norm and we hear this regularly. Fortunately, many organizations are getting better. Shortly, we’ll provide them with availability capabilities that they’ve never had access to, either because of economics, complexity, or scalability.

For the past few months, we’ve been briefing analysts and other experts on what we're delivering this week (and after). What has their response been?

• Wow. This is BIG.
• You’re changing the dynamics of availability.
• You’ve eliminated the triage IT decision make while tackling availability.
• You’ve emancipated applications from downtime.
• This is the “American Dream” of computing.

I like the last one, though I think the rest of the world will appreciate it as well. Stay tuned. We’ll share the word in the next couple of days. And it won’t end there because fault tolerance is about to go mainstream and the implications are substantial. And that’s just the start…

Rob Ciampa

Last week we hosted a webinar looking at why “High Availability Doesn’t have to be Expensive.” We reviewed the trends that are creating today’s “always-on” world where businesses, customers and employees expect 24x7 uptime for all of their applications. We also highlighted several common sources of planned and unplanned downtime, and identified some specific single points of failure to watch out for. We also discussed two real-world success stories of companies that have achieved always-on affordable fault-tolerant protection with Marathon’s everRun software. Below is a summary of the Q&A portion of the webinar.

Q: How much does everRun cost?
Our starting price is less than $10K USD, for an implementation that can support any type of Microsoft application. So you can get full fault tolerance for less than $10K. Contact us for more information.

Q: Have any of your clients implemented Batchmaster ERP with everRun?
We have a number of organizations running ERP solutions protected by everRun. As long as the application runs in a Windows-based environment, we can support it without restrictions. We have more than 3,000 customers running all types of applications and we haven’t seen any application-level constraints within an Windows environment.

Q: What is the farthest apart two machines can be physically located?
We support SplitSite where we can separate systems by up to 100 miles, depending on your interconnect capacity. We see this a lot at airports for example, where our SplitSite solution has one server in one terminal and the second one in a second terminal.

Q: How do you determine when you should use an FT solution vs. a DR solution?
Fault tolerance and disaster recovery go hand-in-hand, but they are two different things to achieve two different results. When planning your application availability model, you have to have solutions for availability, recovery and back-up for complete protection. When considering day-to-day uptime, that’s availability/fault tolerance to prevent those everyday failures that cause business disruption. But when you are talking about a catastrophic event, like a tornado, hurricane or the like, that’s when your DR solution comes into play. DR means recovery time however, so this is not a good solution for protecting against everyday failures. The other important note here is that testing of your backup/recovery solution is critical. Recent studies have shown that 30% or more of recoveries do not go as planned. You need to test those systems regularly to make sure that your recovery will go as planned. So you need to have a local availability solution for the everyday localized failures and then a DR/back-up solution in the event of a catastrophe.

Q: When using SplitSite to separate your servers, is a T1 connection big enough?
It depends on what the applications are doing and what needs to be kept in lockstep. For smaller applications, a T1 could be sufficient, but if the applications tends to be very busy, then that might not be enough. We can work with you to size that and let you know what you will need for your specific applications and requirements.

Q: Does everRun work with Small Business Server?
Yes it does. We have full support of Small Business Server.

Q: Does everRun work with SQL 2008?
Yes, we have full support for SQL 2008 as well.

Q: Can you force a failover manually (for example a corporate policy expects that an application be tested)?
Yes, you can test the systems live and force components and systems to fail manually to test them and make sure that everything keeps working as planned. We had one customer that had some applications being protected with everRun, and a second system that was not being protected with everRun. There was a disk drive failure on the unprotected systems, so what they did was actually pull the working disk drive from the everRun-protected system and use it to temporarily get the unprotected system back up and running. Even without the disk drive, the everRun protected system kept working. This is obviously not something that we recommend that you do, but it shows just how powerful the everRun solution is and how it can keep applications up and running, even when a disk drive is yanked right out of the system.

Today's tip comes to us from author Eric Beehler via our friends at Realtime Publishers.

Disaster recovery is somewhat of a buzzword in the IT industry, and IT professionals have all been exposed to their share of great disaster recovery ideas from business managers. These ideas are often based on the industry buzz and seem to only make more work for you with little gain overall. This is usually because the idea is not backed up with a real plan. The actual implementation of disaster recovery is usually a big chore to undertake correctly, but in the end, it is well worth the trouble.

It's important to be ready to recover your data and systems when a disaster strikes, but it is rarely a top priority in the grand scheme of IT projects when crisis has yet to strike close to home. Unless your company has decided to make disaster recovery a high-level objective, it's usually the front-line administrator that will be saddled with the responsibility of implementing some sort of plan to save the day -- but you will likely be short changed on training and resources to get the job done.

There are many ways to deal with a disaster, from having a set of cold standby machines to employing a fully redundant hot data center. In reality, as the administrator, your job doesn't change much based on the scenario for recovery; it has to be up and available to keep your business running. You likely have some kind of plan now, but if you haven't been through the real thing, you really don't know if your plan will hold water. For Windows administrators, there are several problems that seem to expose themselves when it's time to exercise a disaster recovery plan, or worse yet, go through the real thing. Here are some common ways to tell that you are not ready for a disaster.

Plan for an Alternative Site
You are not ready for a disaster if you don't have a place to go, which requires planning for a full on-site disaster in which your site is down or inaccessible. There are several methods to address this issue if you don't have a solution today, from having an alternative site with servers waiting to be loaded up for operation to a warm site that is always ready and waiting to take traffic. These decisions are not usually made by you but by the CIO. All you can often do is consider the solution given to you and how that will impact your ability to recover. A cold site, for example, will allow you to have hardware and connectivity available, but you will need to account for operating systems (OSs), drivers, configuration differences, and data center differences. In a warm site, you have to ensure that changes to configurations and data remain synched across the two sites.

Plan for Downtime
You also have to consider whether the site solution will support the Recovery Time Objective (RTO) required by the applications and business. Simply put, the RTO is the amount of time your users will be without the functions supported by your server, which could be a Web site, a mailbox, or the ability to log on to the domain. You should have this time defined per application or function supported by your server. This, of course, in a bigger effort for disaster recovery, may be defined for you, but don't be surprised if the business people you support have no idea that your server supports the functionality they require. You may need to interject with your personal knowledge of how your server functions in order to get this definition correct.

There are generally accepted categories for RTO that fall into tiers, as Figure 1 shows. Use these as a guideline but feel free to create standards within your own organization to meet your needs. If you have a need to recover applications with 2, 4, and 8 hours, redefine the tiers so that they make sense to your business through an analysis of the business impact of downtime. Just be sure that you can apply the standards as broadly as possible across the organization.
 

Plan Your Tolerance to Data Loss
You are not ready for a disaster if you don't know your tolerance for data loss. Let's start with the basic foundation of the backup. Whether you use simple tape backups or an advanced nearline solution, you have to consider that most solutions are put in place to account for day-to-day operational needs. First, the exercise you went through with RTO must be done for the Recovery Point Objective (RPO), which is the amount of data that can be lost. You have to understand what the business can afford to lose; this value is not necessarily tied to an RTO tier. Take, for example, a point of sale system. If the system is down for 5 hours, the business may be able to recover by entering the orders taken while the system was down, but data loss of 5 hours may mean millions of dollars in lost sales.

The gut reaction for your RPO on some of your systems may be that no data loss is acceptable. In other cases, 24 hours of data loss may be acceptable. The goal is to understand what can be tolerated, not what is desired. Everyone will desire no data loss, but put a realistic perspective to the real value of the data. If you define Tier A RPO as no data loss, then you have to put systems in place that allow for that reliability. This means copying transactions as they happen to a backup site, which is an expensive solution that should be used only on your critical business applications, depending on your budget. If you have Tier B systems as defined in Figure 2, you will need some sort of solution that will be separate from your nightly backups, as you cannot count on having your last nightly tape backup at your recovery site.

Considering the Loss of a Backup
You are not ready for disaster if you rely on your daily backup for a recovery scenario. You may have in your head that you can rely on the last tape backup in the event of a disaster. Whether such is the case depends on a key question: can you get your restore process to work offsite? Don't be so quick to answer this one. If you take advantage of offsite storage either through a vendor or your own in-house process, it is an excellent step, but offsite storage doesn't necessarily guarantee you can restore at your disaster recovery site within the specified RPO and RTO.

Tape drive compatibility, backup software, delivery time, drivers, and OSs are all considerations that you must address prior to saying your solution is ready. This is especially true for a third-party backup site that will provide you with "like" hardware. That equipment will not be your equipment, and even if it is, expect aspects of the infrastructure to be different, such as IP address schemes, firmware (which can be a nightmare when working with SANs), and simple access to the hardware.

You also have the issue of archive requirements and the fact that you likely rely on these tapes for your day-to-day restores. If you perform restores for file recovery and other issues, you likely want to keep those tapes close by. If you ship them away for maximum protection, it's going to cost a pretty penny in order to request tapes from your offsite storage vendor.

You also have to consider how those tapes make it to the recovery site. If you make full backups only once a week and you only do offsite storage once a week, you might only get a restore from 2 weeks prior. Why? Because if you are lucky enough to get your tapes offsite a day or two after the full backup and you get the shipment to your disaster recovery site 4 to 8 hours after they are requested, you can almost bet that Murphy's Law will strike and you will get a bad tape somewhere in the set. Then you have to move back in the chain, and with most full backups run weekly, you might be taking you system back 2 weeks or more if Murphy continues to strike. Now, the RPO of your plan that you expected to meet with your existing backup plan is not being met.

Even if you do recover your servers with no issues, how long will it take to recover them all? Consider the queuing on the tape drives, with multiple servers waiting for those tapes to be loaded. It could take quite a long time before you even get a chance to try a restore to your server depending on the technology present at the recovery site. What can you do? Well, time to restore will be reduced if you can restore large chunks at one time. Consider putting systems with like RPO and RTO requirements in the same backup set.

Better yet, host them on a LUN or set of LUNs on your SAN or other logical storage method in your situation so that a restore can be done all at once. You might even consider booting from the SAN, which might save you from having to restore the local disk of many servers. If you have a blade server solution, this may even be baked into your infrastructure.

Using Disk-Based Backup
Let's also consider disk-based backup. This solution has become increasingly popular because of the low cost of hard disks and the ease of backup and restore. In addition, disks often take minutes to back up and restore what used to take hours. The software supported by these systems even has versioning, much more frequent backups, and nifty utilities that make life much easier on the administrator. This is usually all handled by complex backup management software such as Microsoft System Center Data Protection Manager. When using this kind of solution, consider employing these often-integrated features to support data replication of some sort, although vendors name these types of features differently.

You can even copy your live data to your recovery site using a SAN/NAS vendor's Failure Resistant Disk Solution (FRDS). You should, however, consider the fact that this kind of solution will be much more expensive than tapes because it will require duplicate equipment with data replication happening across a wide area network (WAN).

You should refer to your RTO and RPO tiers to determine whether certain servers and data sets could stand to be away from your disk replication and rely more on a tape solution. You should also consider your disaster site and understand whether it can support this kind of solution. You should treat your server restores as a form of triage. You need to know, based upon RTO and RPO, what you are going to recover first and what can wait.

Considering Configuration
If you can't identify the full configuration of your servers, you are not ready for a disaster. Realistically, can you keep track of 300 shares on a terabyte SAN served by a load-balanced Windows cluster server? Do you know which shares go to which directories on which LUNs? You have to document configurations. This is true whether you have a basic bare metal restore plan or a full redundant data center. The luxuries of a production environment won't be at your disposal. A normal production environment allows you the opportunity to compare configurations when something goes wrong and work through a problem. A disaster affords you no such luxury.

No matter how familiar you are with your systems, you need to have everything documented that can be changed. For any applications, you should have a guide for their installation in your environment. You should have the servers documented with everything from IP addresses and patches to database connections and configuration files. If you run IIS for Web applications, you should have that configuration documented as well. Some sort of context diagram is often useful to determine how your server interacts with other systems.

Utilize configuration management systems, such as SMS, to do some of the heavy lifting for you. Create reports and keep them up to date in an alternative location, either a paper copy offsite or an electronic one. Configuration problems seem to be a killer when recovering because changes sometimes get applied without strict control. What seems like a small change can kill you in a disaster when it hasn't been documented.

Documenting the infrastructure goes beyond your own servers, but is just as important when it's time to troubleshoot. You can bring your file server back and you can bring your application servers back, but if you don't have proper DNS or connectivity, no one will be connecting to those systems you've recovered. If you have dependencies on other systems, you need to identify them. Know what names should be in DNS, what IP addresses and subnet you are on, what systems you interact with such as database servers or other back-end services such as the DMZ or Internet access. When you tell a database administrator that your application is taking SQL errors, you should know what database server, database, port, connection type, and authentication type you are using. You should also know the user name and password being used, if there is one. Does the server break down into pieces? Does it have multiple applications or functions? Document those functions separately.

You can't think of server as a single system if your customers don't see it as a single function. Remember that restoring an infrastructure is many pieces to a whole, and you should not expect any of those pieces to work correctly as you can in a production environment. In fact, when you face an issue in production, it usually has a single root cause, but a disaster recovery will usually experience several major issues at the same time. You need to know where you stand in the ecosystem of your environment to understand how to identify and help fix those issues.

Identifying Single Points of Failure
If you have a single point of failure, you are not ready for a disaster. A single point of failure can ruin your nicely laid out plans. Although not a requirement for a disaster recovery, the ‘N + 1' definition used when considering disaster recovery is many components backed up by a single component. You can still run into problems using N + 1, especially at a cold site where you have not been exercising your disaster recovery equipment to ensure its health. You might consider having additional servers of a similar capacity available above the minimum number required to recover just in case you experience a failure at your recovery site.

An optimal solution will have redundancy built-in to your recovery site the way you have it outfitted at your production site. If you have a failover cluster in one location, you would do the same in the recovery site, even though you could technically get by with a single server, assuming that server functions as expected. You should also consider the interdependencies of your infrastructure, such as network, when you think of this issue. Single switches, routers, domain controllers, and sources of power can also be points of failure.

Single point of failure doesn't stop at the system level. You might have that one guy or gal who knows everything about your environment. When you're at his desk and something goes wrong with the system or a specific application, he always has the answer. This gal is a good person, but when it comes down to it, you can't rely on a single person. When a disaster strikes, the go-to person may not be available during the recovery phase-yourself included. When everyone looks around and throws up their hands because such and such is down, what do you do? You wish you could go back in time and document that ingrained knowledge. This is also true for day-to-day operations, but especially necessary when everything is going wrong because of a disaster. The person who knows it all is not what you need, you need full documentation of the knowledge that person possesses. Your go-to should really be your documentation.

Integrating Disaster Recovery into Daily Life
If you don't integrate disaster recovery into your daily operations, you are not ready for a disaster. Organizations that plan for disaster recovery as a single project with a start and an end will fail. Don't let the hard work go to waste. When you put these plans in motion, get all that documentation done, have recovery solutions in place, and continue to update your documentation and test your systems. If you don't test you disaster recovery process regularly, how do you know it will work? If you don't update your documentation day-to-day when changes are made, your documentation is outdated and may even be detrimental to your recovery efforts. Don't let apathy or a disconnected process of change management get you in the end. Not only does integration help your readiness, it reduces the dedicated time necessary to getting disaster recovery ready. Find a way to make what you use in disaster recovery a part of daily life.

Eric Beehler has been working in the IT industry since the mid-90s and has been playing with computer technology well before that. From Help desk technician to solutions provider, he has been involved at many layers of enterprise solutions from the desktop to the network to the server and the SAN. He currently has certifications from CompTIA (A+, N+, Server+), and Microsoft (MCITP: Enterprise Support Technician and Consumer Support Technician, MCTS: Windows Vista Configuration, MCDBA SQL Server 2000, MCSE+I Windows NT 4.0, MCSE Windows 2000, and MCSE Windows 2003). He also holds a Master’s degree in Business Administration from the University of Colorado at Colorado Springs. His experience includes more than nine years with Hewlett-Packard’s Managed Services division, working with Fortune 500 companies to deliver network and server solutions and, most recently, IT experience in the insurance industry working on highly available solutions and disaster recovery. He has co-authored books, including MCITP: Microsoft Windows Vista Desktop Support Enterprise Study Guide (Sybex/Wiley Publishing), authored several white papers, and co-hosts the "CS Techcast" podcast aimed at IT professionals. He provides consulting and training through Consortio Services, LLC.

For additional information about Disaster Recovery and High Availability topics, be sure to check out Marathon's Resource Center which has an extensive library of white papers, webinars and eBooks availabile for download.

Earlier this week, we hosted a webinar on the topic of “How to Cut Risks and Costs with a Downtime Analysis & Action Plan.” We know from our experience in application availability that many companies avoid these types of assessments – they either don't know where to start or decide that they don’t have the time or experience to conduct an assessment, so they just live with the unknowns and hope that nothing bad happens. (We’ve seen the consequences of downtime at many companies and don’t recommend this method!)

Our VP of Services & Support, Beth Shea, explored this topic in detail and provided a simple framework that companies can use today to uncover their risks and put measures in place to minimize the impact of downtime. To learn more, be sure to watch the 30-minute webinar. You can also check out the Q&A session from the webinar, summarized below.

Q: When looking at the impact of downtime, it is just unplanned downtime, or should you include planned downtime as well?
You absolutely need to plan for both planned and unplanned downtime, as there’s a real cost and business impact to both. They both need to be included in your impact assessment.

Q: What about branch offices – should they be included in a downtime assessment?
According to Forrester Research, about 20% of a company’s business is tied up in branch and remote offices, and IT needs to include these offices in any assessment that they are conducting. You shouldn’t overlook these offices when putting together your downtime and business impact assessments. They have to be factored in.

Q: How often should I conduct a business impact and risk assessment?
What we’ve found with our customers is that conducting an annual assessment is sufficient, or in some cases, twice a year, depending on the type of business. You can then use these as your benchmark going forward to determine the success of the initiative and ensure that you have the key metrics to report to your management team.

Q: How do you determine when to use local high availability vs. a disaster recovery solution?
Fault tolerance, high availability, disaster recovery - all of these different terms can be confusing and they can have different meanings to different people. The way we think of this is that when you’re implementing high availability or fault tolerance this is to ensure that locally you are protected against the everyday, nuisance failures that cause downtime. If you lose a fan or a drive for example, you would automatically route to another server within the same building or local area. Disaster recovery solutions are really for recovery from catastrophes (fire, flood) or other events where you need to failover to a much more distant location. You don’t want to use this type of solution for everyday failures, as it can be very time consuming to failover and failback, and you can potentially lose some data. For local protection, you want high availability/fault tolerant solutions.

Q: What about hosted applications like salesforce.com, how do I account for those in this type of assessment?
In today’s world, so many applications are offered as Software-as-a-Service (SaaS) or sometimes called hosted applications, where they are no longer hosted at your site. However, they are still important to your business and need to be included as part of your overall assessment. Our approach is to conduct the assessment for your SaaS applications as if all they were onsite. Then use your tiered analysis and make sure that your SaaS vendor is meeting your availability requirements for that application, and that they have the necessary protections in place to protect that application to the same level that you would protect if it were in-house.

Q: Does Marathon offer any services to conduct this type of assessment?
Yes – this is a service that we provide for our customers. Most customers are very satisfied with the service, because it usually has an immediate ROI for their business. If you are interested in this type of service, please feel free to us at 978-489-1100.

Q: Does Marathon have any templates available to build a framework for this type of assessment?
Absolutely. From our 16+ years of working with customers on the assessment and prevention of downtime, we’ve put together an extensive list of questions to ask about the business risks and impact of downtime. Please feel free to contact us if you would like more information.

Q: How do you measure or put a price on the intangible impacts of downtime?
This can be tough to nail down, but what we recommend is developing some basic estimates. This isn’t meant to be an exact number, what we are really trying to achieve here is to prioritize applications, put them into the tiers that we discussed and make sure that you are putting the right amount of resources against the right applications. From a productivity perspective, one metric you could use is to look at the cost of employee salaries and how much it would cost in salary costs to have employees not be able to work for a certain amount of time. This is just one example.

Q: Does everRun handle quick switch over to back up site if the main site goes down?
Yes, within seconds.

Q: What are the requirements for the backup site?
The machines at the backup site are in the same pool as the primary site, so the backup machines must meet the requirements to be in the same pool as the primary site machines.

Q: How about regular data sync between main site and backup site?
Since the primary and backup site are running in lockstep mode, the application and the data are always in sync between the primary and backup sites.

Frank Ohlhorst, former Executive Technical Editor for eWeek and award-winning IT expert, was our expert guest speaker this week for the webinar, “Cut Your DR Costs and Get Better Data Protection.” During his presentation, Frank reviewed why he believes that now is the time to rethink traditional approaches to disaster recovery. He explained why the total cost of ownership for disaster recovery solutions is on the rise, and why changing data protection dynamics are making it more economical to focus your time and budget on the prevention of downtime and data loss, rather than recovery.

Below is the summary of the audience questions from the Q&A portion of the webinar.

Q: You talked about how HA can give you a geographic advantage. What do you mean by that?
Frank Ohlhorst: High availability systems are designed to work with multiple servers and there’s no reason why you can’t have those servers located hundreds or thousands of miles apart. You get a geographic advantage because your data centers is in multiple places and regional areas, so if a weather-related or other event occurs, let’s say a blizzard up north with a power outage, your data center down south can pick up the slack without kicking users off the system. The same can be said about a data center located in an area with hurricanes or other natural disasters. The geographic separation gives you added protection.
When high availability is paired with load balancing, it helps to locate the data resources closer to where the users are requesting them. Let’s say you have users in Utah, it’s better performance-wise to have them talk to the data center in Nevada rather than Virginia. It helps on that level also. HA solutions also have the tools for monitoring what is going on with your users and network, to help you plan out how you should assign users to specific data centers for the most efficiency.

Q: I understand how high availability can handle unplanned downtime, but what about planned downtime? Can it help there as well?
Frank Ohlhorst: Yes, the idea there is being as you have multiple active systems to meet the user’s needs, you can take one of those systems down for maintenance and have the users serviced by the active machines while you make the updates and improvements. Then when you are done, just resynchronize with the other systems, move the users over to those systems and update the rest of the servers.
Another great benefit of this is for testing upgrades and changes. So take one system offline and test your upgrades to see if they work properly before you return that system to production.

Q: If I have an HA solution in place, is back-up still necessary?
Frank Ohlhorst: 99% of the time the answer to that question is yes. It depends on what your corporate needs are. There are certain situations where HA might not deal with your catastrophe. Those are usually software-damaging events, like a virus infection, that winds up getting replicated across the system. Of course, that should really be part of your security planning to prevent events like that from even happening. With today’s security technologies, it’s pretty easy to prevent that. But if you did ever have one of those events, you do need something to roll-back to, and that’s where the back-up comes in to play. Ideally though, you should be preventing that type of event, because you also have the potential to lose active data if that happens. When it comes to compliance or auditing, you have to restore data relevant to that time period to meet the needs of e-discovery, compliance, accounting audits and other similar requirements. So you can’t just say, “I have HA in place, so I don’t need to back-up.”

Q: What about data de-duplication technologies, don’t they help solve this problem of managing large volumes of data?
Frank Ohlhorst: They reduce the data footprint for sure, but what we’re talking about here is availability of the data. They can certainly reduce the size of your data footprint, you can use de-dup to speed up backups. At the end of the day though, if the system or application is not accessible to the user, then it’s not available and you haven’t met your objectives. It’s a simple matter of business logic that data de-duplication can improve performance and reduce the size of the footprint, but it doesn’t solve the problem of providing access to users during catastrophic events.

Q: Do you see continuous availability and high availability as the same, and if so, how do you differentiate between the two and the costs?
Frank Ohlhorst: There was a time when those technologies were very, very different. That was way back when we relied on expensive hardware-based solutions or appliances that provided continuous availability. High availability at that time was thought of as a method to switch from one server to another using a manual process in the case of an emergency.

High Availability technology has evolved significantly since then. Now, the two are really one in the same from a planning and software point of view. Today’s HA solutions eliminate that step of manual switchover. What you see with the vendors today is automatic HA technology that really delivers continuous availability. And the cost gap today is pretty much zero, since the technology for continuous availability and high availability has evolved to be almost one in the same.

Q: With an SRDF/S-type solution, how can we get around the fact that being geographically more separated to mitigate regional disruptions can mean slower primary system response times due to the need to remain synchronous?
Frank Ohlhorst: Let’s look at this first from the ideology of what we’re trying to do which is business continuity. So, if you encounter a situation when you lose connectivity to a system and it’s still available at another location, then you’ve met the goal there of providing continuity. And you’re in much better shape than you would be at that point if you had a disaster recovery solution instead of a business continuity solution.

The question you have to ask yourself at that point in time is: Is reduced performance better than no performance at all? For most businesses, the answer is yes. For others, if the performance lag is significant enough it can impact business. In those cases, you’ll have to work out a way to develop geographically dispersed sites can that can provide enough performance to the user sets that need access to the data. You also need to make sure that your connectivity has enough bandwidth to support your BC/HA solutions, which means the ability to replicate the data in real time across the wire. You might have to invest in larger pipes for better connectivity to support that. But again, that depends on your particular business and your needs. There is no one correct answer to this question, but the good news is that there are several solutions today that can help you solve this problem and meet the levels of availability that you need for your business.

LAN Magazine, a leading IT publication in the Netherlands, recently published an in-depth review of everRun 2G titled, “High Availability for the Masses.” You can read the review (in English) here. Bram Dons, the reviewer and author of the article, had this to say about everRun 2G:

"Although recovery certainly plays a role in high availability, it is much more about preventing downtime and data loss. Marathon Technologies shows this with the new everRun 2G."

“In all evaluation tests we have conducted to date, Marathon’s everRun 2G is the only software product that manages to provide complete continuity in all circumstances. Failure of system components, power interruptions—the product does not fail."

To read additional reviews of everRun from leading IT publications like eWeek, Network Computing UK, and IX Magazine, check out our product reviews page.

The application availability experts here at Marathon were asked to put together "10 Common Mistakes Made by Disaster Recovery Teams" for a featured slideshow for ITBusinessEdge.com. These 10 common mistakes are summarized below:

1. Confusing HA and DR
A lot of companies confuse high availability (HA) and disaster recovery (DR), or implement a DR solution when they really need HA. Put simply, HA is about preventing the everyday failures that cause downtime (network card failure, storage corruption), while DR solutions are designed to help you recover from true disasters (floods, hurricanes), not minor problems.

2. No specific disaster recovery plan
Implementing disaster recovery software or speaking broadly about “what-ifs” is not enough. The IT team must be well versed in a set plan which has been tested and proven effective. IT staff, as well as upper-level management, should be trained in the DR protocols in the case of any business disruption. In the event of a disaster, team members should already be familiar with the plan and not rely on in-the-moment decision making.

3. Untested disaster recovery plan
While testing the plan may not mean that it will go off without a hitch, it is an important step in preparing the company for a disaster. After testing, improvements should be made and the plan should be scrutinized for any possible holes.

4. Only involving the IT team in the planning process
Disasters affect the entire business, not just your IT infrastructure. Representatives from all company departments should be involved in the planning process and should know their role in the event of a disaster. In addition, it is imperative to train company executives and decision makers in how to carry-out the plan. They should be aware of all protocols, and be involved in testing exercises.

5. Adding too much complexity
Many technologies actually introduce complexity into the IT environment. For example, clustering technologies may require administrators to painstakingly maintain each server in the cluster to support successful failover. IT organizations instead should find and embrace those technologies that reduce complexity for operational staff—thereby eliminating potential sources of human error.

6. Purchasing inexpensive, low-quality hardware
While it is tough to justify shelling out the extra dough for a top-of-the-line server, it is well worth it on the day that your processor fails. Many IT staffs are working with constrained budgets and therefore have to buy lower priced equipment. This equipment is more likely to see failures, increasing the likeliness of future problems.

7. Using common components in the physical network hardware
For example, dual-ported network cards share common hardware logic, and a single card failure can disable both ports. For full redundancy, you need either two separate adapters or a built-in network port combined with a separate network adapter.

8. Utilizing on-site data replication
Many factors can cause site-wide failures, including an air conditioning failure or leaking roof, a power failure, or a major hurricane. Site disruptions can last anywhere from a few hours to days or even weeks. There are two methods for replicating data across sites. One method is to tightly couple redundant servers across high speed/low latency links, to provide zero data-loss and zero downtime. The other method is to loosely couple redundant servers over medium speed/higher latency/greater distance lines. This provides a disaster recovery capability where a remote server can be restarted with a copy of the application database missing only the last few updates. In the latter case, asynchronous data replication maintains a backup copy of the database.

9. Implementing a plan that worked for someone else
DR/HA is not one-size-fits-all. Every business has different objectives for different applications. It’s ok to look to others for guidance, but stay focused on your specific goals.

10. Not understanding business requirements
What exactly is it that you need to accomplish? Implementing wrong or incomplete solutions can waste time and money. Know what clients and users need and adjust the DR plan based on the service levels that need to be met.
 

Earlier this week, we had a great webinar featuring guest speaker Greg Cullen, Sr. Director of Technology at Marathon. Greg provided tips and advice that government agencies can use to ensure optimal availability for their critical applications at minimal cost. He also reviewed  three different government agencies that protected their data and applications from downtime with Marathon’s everRun software.

These included the Brookline Police Department, which kept their 911 system available – even during a hardware failure on Christmas day when no IT staff were on duty; the City of Santa Rosa utilities department, that protects its water treatment facility against outages from earthquakes, power outages and other natural disasters; and a county court system in California that protects its virtualized Exchange server and other paperwork processing applications from downtime.

The transcript of the Q&A session with Greg from the live webinar is below.
 

Q: Can you deploy the two everRun servers in different locations or do they have to be in the same location?
You absolutely can. You can deploy the two servers in the same room, or different rooms in the same building and even in two separate buildings. We call that configuration SplitSite. That’s one way to get disaster tolerance in the event that you have a site-wide outage, rather than having them in the same room or building.

Q: Is there a limit to how far apart the servers can be?
In general, they can be separated by as much as 100 miles, although it really depends on your bandwidth and latency on the connection between the two sites.

Q: What are the hardware requirements for everRun?
You’ll need Intel-based servers that have a moderate amount of memory, or as much as the application requires. And you’ll need enough networks to do the production side of it as well as maintaining the redundancy between the severs. Generally speaking, having four network adaptors in each server and somewhere in the order of 100GB of disk drive is sufficient.

Q: How does the everRun software compare with a clustering solution?
One of the biggest difference between everRun and clustering is that everRun is a single image for the application. Instead of installing and managing two instances of your application like you do with clustering, everRun is just a single image to install and manage. Changes happen on both servers simultaneously through that single image.

Also, everRun software does not require cluster-aware applications. everRun is application agnostic, and can support almost any Windows application. And one more thing, with most clustering solutions, you also need to have a shared storage container that both servers are connected to. everRun can support that model as well, but doesn’t require it like clusters do. In fact, to remove single points of failure, it’s much better to have local storage connected to each of the servers and everRun will manage that storage as a mirrored device.

Q: I’m confused by your use of DR. Can you define what you mean by disaster recovery?
We’ve found that everyone has a different definition of what they mean by disaster recovery. At a very high-level, we see disaster recovery as the need to protect your data. By comparison, we see high availability as the need to protect your application, data and network connectivity. DR means you’re trying to copy your changed data off site to protect it in the event of a true disaster. After the “disaster” is over, you then need to bring that data back to the primary site, or configure an alternate server to use the data in the DR site.

Q: Does everRun work with Siemens building security systems?
Yes, we have been working for several years with building automation and security companies including Johnson Controls, Tyco, Andover Controls, Siemens and many others. As long as the building system runs in Windows Server 2003 or 2008, we can provide availability for it with no custom scripts or custom coding. We have many deployments of everRun protecting these building security systems around the world.

Q: Does everRun work with e911 systems?
Yes – absolutely. Generally speaking, everRun is application agnostic and can work with almost any Windows application. We have many solutions out there where these emergency 911 centers are protected by everRun so that if there is some type of disaster, these systems continue to run.

Q: Is everRun available on a GSA schedule?
Yes, through our channel partners. Contact your Marathon account representative or call 978.489.1100 for specific partner information.

Q: How does everRun differ from data replication solutions?
A lot of times when people look at availability, they simply try to replicate the data. There’s a big issue with that though. That’s only one part of what you need to recover in the event of a failure. everRun not only replicates the data, but also keeps a redundant set of your application environment and network connectivity, and everything else that is required for the application to not see any failures at all, or to recover very quickly in the advent of certain types of failures.
 

Next week Craig Resnick, research director and automation expert at ARC Advisory Group will be the guest speaker for our webinar "Best Practices for Preventing Downtime in Automation Systems."  We recently sat down with Craig to discuss some of the recent trends in the manufacturing and automation industries.

Q: What are some of the newer trends that you are seeing in the automation space?

Craig Resnick: A primary trend that we see at ARC is the convergence of automation and IT systems. Nearly every manufacturing company uses a variety of plant automation and enterprise IT systems to manage its operations. Plant floor systems, such as distributed control systems (DCS), programmable automation and logic controllers (PACs/PLCs), and a wide range of plant floor applications provide a wealth of real-time information regarding productivity, efficiency, equipment health, capability, and quality. Business systems, in turn, provide information on raw material costs, product orders and inventories, manufacturing resources, production schedules, etc. This wide range of information often remains isolated in systems such as manufacturing execution systems (MES), laboratory systems, maintenance systems, scheduling systems, enterprise resource planning (ERP) systems, supply chain management (SCM) systems, and customer relationship management (CRM) systems. Decisions based on data from any one of these system will always be less than optimal because, without the corresponding information from the other systems, the information will be incomplete.

To close this gap between automation and IT systems, and to address the trend of the plant floor becoming more IT-centric, ARC has defined a new space, defined as Collaborative Production Systems. These new systems consist of platforms in which the controls layer domains of process, logic, motion, building automation, and power control systems converge with the information layer domains of production management and MES systems. These converged systems enable, for example, the required data and information to be directly tied into applications such as corporate reporting and manufacturing compliance. Collaborative Production Systems will become the industrial blade server that provides full monitoring and control of the enterprise, from the office to the plant floor, sharing that information with the supply chain to, for example, procure materials and resources and purchase or sell power at the optimal times and prices from the smart grid, while providing full financial metrics and KPIs to ERP systems to maximize profitability.

Q: Now that corporate reporting and systems are heavily tied into the “factory floor”, how is that changing the need for system availability and data protection?

Craig Resnick: The need for system availability and data protection continues to expand, driven by a combination of issues ranging from global competition to regulatory requirements. Process safety and critical control are primarily focused on system availability and process uptime. As a specific example, take the Pharmaceutical industry, where data and batch information can never be lost or interrupted. System availability and data protection needs are also forcing E-records regulations to evolve across the globe. In the US, this includes 21 CFR Part 11, as well as the FDA’s Good Manufacturing Practice (GMP) and Process Analytical Technology (PAT) initiatives. In Europe, this includes Annex 11 of the EU GMPs, electronic Signatures Directive 1999/93/EC, and Data Protection Directive 95/46/EC. The European Data Protection Directive requires even more protection on data than the current FDA regulations and extends this requirement to clinical trials patients, as all clinical trials data requires maximum protection to remain compliant with regulations.

Unscheduled downtime is expensive. It often impacts production’s ability to meet its schedule and may cause missed customer commitments. Unplanned downtime, which also includes unexpected stoppages resulting from equipment failure, operator error, or nuisance trips, is the nemesis of all manufacturers. Statistics on the impact of unplanned downtime on plant operations show that it accounts for 2 to 5 percent of production lost in, for example, the petrochemical industry. Unscheduled downtime is also costly in terms of equipment damage, environmental harm, and worker safety. The cost of downtime is reflected in a primary key performance indicator (KPI) used by manufacturers known as Dynamic Overall Equipment Effectiveness (OEE), which helps determine the real-time impact of the performance of any individual process or piece of equipment on the overall efficiency of the plant. Unscheduled downtime is a primary factor that significantly lowers Dynamic OEE, which translates to the manufacturer decreasing both its efficiency and profitability.

Q: What are some of the basic steps that companies can implement to ensure the availability of their systems?

Craig Resnick: The first step that companies can implement to ensure the availability of their systems is to maximize their operator’s effectiveness in the control room, which is essential to minimize the risks of accidents, eliminate unscheduled downtime, and maximize production quality. The global process industry loses $20 billion, or five percent of annual production, due to unscheduled downtime and poor quality. ARC estimates that almost 80 percent of these losses are preventable and 40 percent of those preventable losses are primarily the result of human or operator error. Maximizing operator effectiveness requires automating as many functions as technology will allow, as well as reducing complexity wherever possible. For example there are still many plants where operators monitor the processes and collect data manually or semi-automatically using chart recorders. This process is both tedious and error prone, and does not provide appropriate process insight or instill a sense of ownership among the control room operators.

The Abnormal Situation Management Consortium (ASM) points out that most incidences occur from multiple modes of failure. Preventable human error is a contributing factor to these losses, but is hardly the only cause. Preventing abnormal situations requires a multilayered multi-discipline approach focused on maximizing production throughput, efficiency and quality while minimizing lost production time and preventing damage to assets and endangerment to personnel. This approach requires deploying collaborative production systems designed and implemented to be able to deliver high levels of availability and fault-tolerance expected from any other mission critical industrial system. This typically requires effective data backup mechanisms, redundant controllers for critical applications, plus industrial grade software. Manufacturers are also deploying more fault tolerant server technology to ensure continuous availability of these mission critical applications; the continuous flow of vital products to the market; and the avoidance of the potentially negative financial, social, or environmental impact that operating without high availability fault-tolerant systems might bring.

To learn more about preventing downtime in your automation applications, be sure to attend next week's webinar where Craig will provide expert info on steps for reducing the human error that leads to downtime, how to protect your hardware, storage and networks for complete availability coverage, and how to protect against a complete site failure. You can register here.
 

Marathon’s Sr. Director of Products, Michael Bilancieri, recently answered some questions for Paul Mah of ITBusinessEdge.com regarding disaster recovery planning for small & medium businesses. A few of Michael’s answers are highlighted below. For the complete Q&A with Paul Mah, see the article here.

Mah: Any tips to help SMBs with constrained budgets get management’s approval to implement a DR program?
Bilancieri: This may be the most important part of the process. Without support from the senior management team, any DR plan will be hard to get off the ground. The key takeaway here is to translate the technical language into business terms.

Since DR is not primarily about the technology (it is about the business value), it is important to clearly express what downtime means in terms of revenue loss. By creating a chart, organized by each application, it is easy to clearly articulate how much revenue is lost across each application for a certain amount of time.

Mah: What are the best criteria for determining an optimal disaster recovery plan?
Bilancieri: First, you have to identify what it is you need to accomplish. This includes defining the recovery time objectives (RTO), which is the amount of time applications can be unavailable and recovery point objectives (RPO), which is the amount of data that can be lost when a recovery is required.

Keep in mind that these values will likely vary for each of your different applications. Implementing incorrect or incomplete solutions will result in wasted time and resources. Check with your users and clients to determine their requirements and any service level agreements that must be met.

Mah: Once you determine exactly what your needs, how do you select a plan?
Bilancieri: DO YOUR HOMEWORK! Seriously, there are so many different products that claim to be “DR” solutions, all approaching the problem from different angles, it can be very confusing to determine what actually does the job you are looking for it to perform. As you research different products to implement as part of your DR plan, be sure to ask specifically what their product does (copies just the data, takes data snapshots, captures complete images of the full system, etc.) and don’t be afraid to ask probing questions.

Many vendors make the same claims using the same terms but actually deliver very different results. If you are going to test these solutions in-house, which is recommended, try to do the test under similar conditions as your production environment, with similar system and application loads. Oftentimes, something works well in a test environment [where there is] no real processing happening, [but] fails to function adequately once deployed in the live production environment.

Mah: What would a DR plan look like for a company that may face natural disasters such as hurricanes and flooding?
Bilancieri: Since hurricanes and floods can cause severe damage that can result in long-term outages, it would be wise to implement a solution that protects your systems between locations that could not be affected by the same disaster. Ensure that the backup, or DR, site is planned for a location that can be readily accessible by your users and clients should the primary location be destroyed or otherwise inaccessible.

Marathon has a customer based in Georgia, The Sullivan Group, which implemented a disaster recovery plan just for this reason. The team decided to virtualize its data center with Citrix XenServer and implement Marathon's everRun VM solution to provide redundant virtual machines and synchronized mirroring of the entire system including network, applications and data. The Sullivan Group has a small IT staff but needs to be continuously available for their clients, so they needed a solution that was fully automated and offered simply implementation.

Their first step was to identify what their customers’ needs were - and they decided that they needed continuous protection. Second, the team determined exactly what they could afford, and the ROI they would see from implementing DR software. They already knew that they would constantly face the threat of storms, and that they needed their data to be backed up in a remote location. Finally, they determined exactly what solution their IT staff could support and decided exactly which business applications needed to be fully available.
 

Nick Turnbull, Marathon’s VP of international sales explored the importance of ensuring that hospitals and healthcare providers suitably protect themselves for the possible failure of their IT systems to minimise disruption in a recent article published in HES magazine. An excerpt from the article is below. For more information on this topic, you can also download our recent white paper "Finding a Cure for Downtime: 7 Steps to Reducing Downtime in Healthcare Information Systems."

The management of administrative, financial and clinical aspects of a hospital rely on continuous uptime. Every single minute of downtime can jeopardise compliance, revenue and most importantly the health and wellbeing of patients. Downtime is a risk no hospital or healthcare provider can afford to take.

Hospital equipment and systems are designed for a variety of tasks, some keep track of the administrative issues of a hospital, or look after clinical information systems that concentrate on patient-related and clinical-state related data such as the electronic patient record (EPR) and the monitoring of life support machines, MRI scanners, etc. The ensured access to all IT systems is paramount for the smooth running of a hospital and to ensure quality patient care.

EPRs and the need for centralised patient information have seen regular attention in the media recently. EPR systems have the potential to bring huge benefits to patients and they are being implemented in health systems across the developed world. Storing and sharing health information electronically can help to speed up clinical communication, reduce the number of errors, and assist doctors in diagnosis and treatment. Equally, this kind of electronic data can also have vast potential to improve the quality of healthcare audit and research. However, increasing access to data through ERP systems also brings new risks to the privacy and security of health records, as well as practical aspects that need to be catered for in order to reap the full benefits of such a system without any disadvantages.

The importance of being able to access EPRs becomes apparent when looking at the accident and emergency setting or for example the cancer unit of a specific hospital. It is here that access to a patient’s medical history can become a matter of life and death.

It is clear that nowadays IT sits at the heart of modern hospitals, so it is key to ensure IT systems are available to healthcare professionals 24 hours 7 days a week. This means that hospital management and their IT support needs to be sure that the technology they deploy can monitor the entire system around the clock. So, for the last few years, hospitals and healthcare providers have turned to the latest and greatest technologies to support their systems, minimising the risk of disruptions to operations and ensuring availability.

The right solution needs to be 100% effective, which is only possibly if it monitors and receives data continually 24 hours, 7 days a week, with absolutely no hiccups. If this is not the case all the time, there could be lives at stake. If any unexpected downtime occurs, the ability to access records, the continued running of life support monitors and MRI scanners is at risk. Clearly, hospitals and healthcare providers need to ensure that their systems are adequately protected against any unexpected IT failures.

While undoubtedly many hospitals and healthcare providers are using various different systems, they should not forget that these systems need to be protected. If the servers behind the systems experience downtime, it could cause havoc with patient-facing devices and the EPR system. IT system availability is no longer an ideal – it is a necessity. That is why hospitals and healthcare providers need to be sure that when adopting the newest, ‘safest’ technologies, they also ensure that they come with rock solid availability. Continuous uptime for the healthcare industry is absolutely essential to ensure the success of the business as well as the safety of patients.
 

Keeping your applications “always-on” for users is no easy task, and can be particularly tricky for branch or remote locations where you probably have little or no IT staff to support your efforts. Forrester Research senior analyst Stephanie Balaouras has been studying this trend and has pulled together the top 5 best practices for supporting application availability at remote and branch locations. She presented these during a webinar last month and we've also summarized them below.

TIP #1 – Don't Overlook Remote Location Availability

While this may seem like an obvious point, it’s actually very common for IT departments to overlook their branch and remote locations when it comes to application availability. You can’t neglect these offices for both high availability (HA) and disaster recovery (DR) plans—you need a holistic approach to protect all of your business applications, no matter where they are located. This also means that you need to factor in these systems when planning your IT budget as well.

According to recent Forrester Research data, IT systems at remote and branch office locations account for more than 20% of your total infrastructure. They are critical to your business process and operations. Today, a lot of these locations don’t have HA or DR, and in some cases, they don’t even have basic back-up. Make sure that these offices and locations aren’t forgotten as part of your HA and DR plans.

TIP #2 – Classify Systems by Criticallity

When developing your strategy for operational HA and DR, best practices include performing a business impact analysis. This doesn’t have to be a lengthy process—you just need to map the dependent systems for each business process, and then create a rough estimate the cost of downtime for each. Once you have that information, you can determine availability rates as well as recovery objectives. As part of that process you should also identify the most probable types of downtime. When you put that all together, you can classify systems by criticality, such as mission critical, business critical, business supporting, etc., and you can then determine the availability rates needed for each of those systems.

TIP #3 – Develop Tiers of Service for Availability

Once you understand your range of recovery objectives, it helps to have an IT availability and service continuity catalog. This catalog defines a range of service tiers. Forrester typically sees four levels: mission critical, business critical, business important and business supporting. Each of these tiers has associated recovery objectives, technology pre-requisites and the costs to deliver that service. This catalog helps to simplify your strategy, by allowing you to assign appropriate tier classifications to new systems quickly and easily.

Another benefit of using this method is that it also helps you to limit the number of point products you are using for HA and DR. The more point products you are using, the more you complicate the sequencing and complexity of preventing a failure or recovering from a failure. Keep it simple. Every time you deploy a new application or system, assign a tier from your catalog, put the appropriate protection in place, and then communicate that to the business.

TIP #4 – Measure Availability from the End-User Perspective

Well-written objectives measure both planned and unplanned downtime and also take into account the timing of downtime. For example, you don’t take your systems down for planned maintenance during peak sales periods or at 1pm on a weekday when your traffic is at its highest level. You select times when users will be least affected. Availability isn’t about the individual IT system, infrastructure or component. Technology uptime is important to track but is not a true measure of availability. True availability has to be measured from the end user perspective. If the application or service is not available for use, even if the individual components are functioning, then that means the service is down. When making decisions about HA and DR strategies, you have to look at availability from a people perspective, not a technology perspective.

TIP #5 – Make Availability Part of Every IT Decision

Availability is no longer an optional practice. It’s essential. It’s something you owe to your employees, your customers, your partners and your investors. Application resiliency has to be part of the planning process right from the start—HA and DR should not be an after-thought. Even in remote and branch locations, these applications are critical to the success of the business, so availability of the systems should be included during the planning phases of the project, rather than an add-on after the project is completed.

Last Thursday’s webinar “Application Availability for Remote & Branch Locations” with Forrester analyst Stephanie Balaouras was packed with useful tips and best practices for protecting remote and branch offices from application service disruption. Stephanie has conducted extensive research in this area and shared her Top 5 Best Practices during the webinar. A recording of the webinar is now available in case you missed the live event.

The summary of the webinar Q&A with Stephanie and Michael Bilancieri, Sr. Director of Products for Marathon, is below.

Q: I like the idea of integrating HA and DR plans. How often should those plans be updated?
A: Stephanie Balaouras, Forrester: The ideal scenario is to update your high availability and disaster recovery plans continuously as part of your change management and configuration management. That’s the ideal scenario. They should be integrated into day-to-day operations and your plans should be updated as a part of that. If that’s not feasible, then at least quarterly updates should be made to the plans. One of the hardest parts of DR is that if you don’t keep the plans updated and you’re not testing regularly you’ll have major configuration drift between your sites. When you have a failure or disaster and have to invoke your DR plan is not the time you want to find out just how far your configurations have drifted and that you can’t recover. One solution for this is the combination of virtualization and replication, which can reduce complexity because in most cases you’re actually replicating the configuration changes as they happen.

Q: On your disaster recovery continuum slide (slide #14), can I think of that as a disaster recovery maturity model?
A: Stephanie Balaouras, Forrester: Not really. When I evaluate a company for disaster recovery maturity, I look at two dimensions – process and technology.

On the process side, I look at things such as: Have you run a business impact analysis? What about a risk assessment? Are preventative measures in place? Do you have documented plans, and are they up to date? How often do you test them?

On the technology side, I look at things like the RTO and RPO that you have defined: Are they matched up with the appropriate technology solution? If RTO is less than 2 hours and RPO is zero then I would expect that you are replicating data and doing rapid system restart with virtualization. If I find that you are using tape in that situation, then that’s a problem. I think when it comes to maturity you have to look at process and technology together. Not only should you match up with the right technology, but you might actually leverage more than one technology depending on your needs.

Q: Traditionally, HA & DR at remote locations has not been a priority. Do you see that attitude changing with clients that you talk to?
A: Stephanie Balaouras, Forrester: I do see things changing. I run an annual survey with the Disaster Recovery Journal. One of the questions we ask is: How critical is it to upgrade disaster recovery at your sites? The answer is always either “high” to “extremely critical”. It doesn’t always get addressed the way we want it to, but the recognition is there.

I see three main drivers for this trend. First, availability and disaster recovery are now considered a fiduciary responsibility. It’s no longer an optional practice. It’s essential. It’s something you owe to your employees, your customers, your partners and your investors. Second is the cost of downtime. Companies are much savvier at calculating this cost and aware of the problems they can avoid by not having downtime. When you understand those costs, you can make the right technology investment choices. The final driver I see is the changing business environment. A lot of companies are operating globally on a near 24x7 basis. Like an online retailer for example. We’re operating close to 24x7 and there is no tolerance for downtime anymore. All three of these – fiduciary responsibility, cost of downtime and a 24x7 business environment are moving the needle quite a bit.

Q: In my environment, our IT staff says they have no way to measure if an application is up or not. They can tell us if a server is up, or if a database is up, but not the application. What solutions have you seen that can tackle that issue?
A: Stephanie Balaouras, Forrester: There’s a couple of ways to address that. There are third party application monitoring tools from the large system vendors. They are great for basic monitoring and telling you if your application is up or down, but they don’t tell you about degradation of performance. The other option is that different HA solutions will be able to detect whether the applications is up or down.

Michael Bilancieri, Sr. Director of Products for Marathon, answered your questions about everRun software.

Q: Does everRun have any kind of alerting capabilities for system problems?
A: Yes, everRun has alerts. You can send notifications back to any location. It will tell you that something has failed – it’s not a downed system because everRun kept it going through redundancy, but it alerts you that it needs attention.

Q: Does everRun require that the two servers to be identical?
A: The servers don’t have to be exactly the same; however, the CPUs should be identical as a best practice. For what we call our Level 2 protection (for component level protection of the network and disk), you can use different RAM and spindle speeds on storage. Level 3 protected workloads require the servers to be alike. You can view a complete list of supported processors on our website.

Q: How much of CPU and IO payload will we have by running the everRun software?
A: It varies depending on the applications and systems and where the load may be. The general range is from 5-10%. We have specific application performance documentation for Exchange 2007 and XenApp that you can download from our website.

Q: I understand from your presentation that everRun doesn’t require a SAN, but does it work with SAN?
A: everRun can support a SAN in multiple ways. everRun can support a SAN where you have a single copy of the data. And both servers will connect to the single copy of the data. everRun also supports a SAN where one of the servers is connected to that SAN and the other server has its own storage and we can mirror between that. A lot of our customers are using that option to provide data protection and fault tolerance at the data level. We can use different types of storage on either side.
A great benefit of everRun is that is has an agnostic approach to storage. Pretty much any type of storage will work. iSCSI, fiber, direct attached, etc.

Q: Does Marathon have a strategy for SAP environments?
A: Applications are transparent to everRun. We protect many types of SQL, Oracle and SAP applications. There are some best practices around that and we can offer you assistance with those. everRun is invisible to the application, so there are no configuration and design issues. You design your application the way you need to for your business and then everRun protects it without needing changes.

Q: What versions of Windows Server does everRun support?
A: everRun supports Windows Server 2003 SP2 Standard and Enterprise Editions, 32-bit and 64-bit, as well as Windows Server 2008 Standard and Enterprise Editions, 64-bit.

Q: The requirement for redundant systems is obvious, one local and one remote, but I am concerned with the return of the repaired server back to the primary server role. Has that issue been also automated in your application?

A: Replacing one of the servers in an everRun configuration is quite simple as well. It is required that the everRun software be installed and the server be physically connected to the remaining everRun system. Once connected and configured to see each other as a pair, there is a ‘re-pairing’ process that is initiated via command which starts the process of creating the redundant OS environment on the new system and mirroring all of the storage to the new system. Once the mirroring is complete, the system is once again fully protected.

It’s always interesting at this time of year to take a look back at what was top of mind for our newsletter readers. It’s also a great opportunity for you to discover a key topic that you might have missed the first time around. Here are our top 5 most downloaded articles and white papers of 2009:

1. Configuring High Availability for Windows Server 2008 Environments
2. Optimizing Exchange High Availability - A New Approach
3. Increasing Reliability and Availability in a Virtualized SQL Server Environment
4. Reduce Downtime by 70% - Without Spending a Dime
5. iX Magazine product review: vSphere 4 FT vs. Citrix XenServer with everRun VM
 

If you would like to stay current with latest trends, developments and tools in the world of high availability and disaster reocvery, be sure to sign up for our monthly newsletter by sending an email to mstec@marathontechnologies.com or click on the Resource Center and look for the sign-up box in the right column.

We had some great questions during last week's webinar High Availability Doesn't Have to be Expensive. A recap of the Q&A is below, including the questions that we weren't able to get to because of time constraints. Be sure to check out our library of on-demand webinars, for this webinar, as well as other topics including SQL availability, Windows Server availability, everRun product demos and more.

Q: How is everRun different from replication solutions?
To understand how everRun is different from replication solutions, you need to take a look at the key differences between disaster recovery and high availability. Availability is about preventing outages instead of just recovering from them; about maintaining the user state with minimal interruptions. With disaster recovery (DR) and replication methods, if there is a failure, you lose connectivity for a period of time and then you have to recover your data and system state. Conversely, availability is about reducing and preventing downtime and keeping users online, even through a failure.

everRun is used for availability, both locally and for short-distance geographic separation as well. We have a replication and recovery solution as well that can be used for disaster recovery for long distances. You should determine what your objectives are: do I have to keep my applications up and running or do I just need to recover it if something fails? What’s the recovery time objective for each application? It’s up to your individual applications and what level of protection you need for each. Oftentimes, availability is a priority as downtime is not desirable, with DR also a requirement on top of that to ensure recovery in the event of a major outage.

Q: What kind of bandwidth requirement is needed for a two-site solution?
As a general rule of thumb, an OC3 connection is required per application workload being protected. Latency is really more critical than bandwidth and this will vary based on the applications and environment.

Q: How does everRun software compare with EMC’s RepliStor and AutoStart applications?
everRun is different from these products because it provides high availability in an automated way with fault tolerant capabilities to prevent user interruptions when hardware fails. This goes back to prevention rather than recovery.

RepliStor is a DR/replication product. While it does provide a failover/restart capability, as do most DR solutions, it is really best used for failover in the event of a major disaster. There’s usually a substantial amount of downtime and a manual failover process to get the systems back online at the secondary site and to failback once the primary site is back online. For DR, you probably want to be able to specify when your systems fail over. But, you will lose some data because this is an asynchronous solution. For minor outages, you really don’t want this. For example, let’s say the power goes out in your primary location for an hour. It can take even longer than that with DR systems to failover to the secondary DR site. You would have been better off just waiting an hour for the power to come back on and restarting the primary systems. RepliStor is more suited for major disaster scenarios, rather than just minor local or regional failures.

Auto-Start is more of a clustering type of product designed for availability and application restarts. It’s not designed to prevent downtime due to failures, but rather to recover from them.

Q: Can everRun be used for planned downtime?
Planned downtime for patches, upgrades, etc. can sometimes cost as much or more to your company as unplanned downtime. The answer to this question will depend on the type of updates. Some OS upgrades do require that there be a restart for the changes to take effect. For some types of planned maintenance, everRun can eliminate the need for downtime. For the others, one of the main things everRun can do is to reduce the risk of updating a system and not having it come back online. For example, you’ve just overwritten your production system and it worked in a test environment, but now it won’t come up in production. We can reduce that risk greatly, by getting it back online quickly without the need to rebuild the server.

Q: What is the difference between everRun and vMotion and VMware HA?
These are two different products, so we'll start with VMware HA. The HA product is a failover/restart capability. If you lose a host, the system will try to restart the virtual machines on another host on the pool. There’s no real guarantee here though. It’s going to try to find resources when a failure happens, but they might not be there. There are some checks in place to warn when over using resources will impact the recovery plan but there is nothing to prevent this. When there’s a critical RTO though, it’s better to have something that is more assured like what everRun provides. everRun uses mirrored systems, so you always know that you have resources available in the event of a failure. everRun also protects the data – we don’t require a SAN. everRun can mirror data between to two systems or two buildings and it doesn’t have to be the same type of storage on both sides. It can be SAN on one side and NAS on the other. everRun can move the data between locations and keep it tied to the applications to keep your business running, even when there is a failure.

As far as vMotion, that is primarily used for planned downtime. Motion capabilities in general allow virtual machines to be moved or “motioned” while they are running from one host to another host. everRun can provide that capability as well. We call it online migration. If you want to take host offline for planned downtime for upgrades for example, we can do that. Motioning is really for planned downtime. If something fails unexpectedly, vMotion can’t help you there. everRun provides capabilities for both planned and unplanned downtime.

Q: What versions of Windows does everRun support?
everRun supports Windows Server 2003 SP2 Standard and Enterprise Editions, 32-bit and 64-bit, as well as Windows Server 2008 Standard and Enterprise Editions, 64-bit.

Q: In considering using everRun across two sites, is everRun doing real-time synchronization between the sites?
Yes it is. It’s writing the data on both systems in a synchronous manner, so that data is always complete—system data and applications are secure and exactly mirrored on the secondary server. It protects the entire environment—the operating system, registry, every setting, etc. is completely cloned. You can turn on that system on the other site and not have to rebuild the server. We maintain exact mirror copies of both servers. That goes back to our message about prevention and computing through failures, rather than downtime and recovery.

Q: Does everRun work with SQL 2008 and SharePoint 2010?
everRun sits below Windows. It’s not in the operating environment. We protect the entire environment, so anything in that environment is automatically protected, whether it’s SQL or Exchange or anything else, even custom applications. There is no customization needed for everRun to protect any Windows applications.

Q: What are the storage requirements for everRun?
everRun offers two storage configuration options: mirrored storage and a shared storage model. When using mirrored storage, everRun will synchronously mirror all storage between paired hosts; this includes the OS, the application, data, etc. This does not require similar storage vendors or types. One host can have SAN-attached storage while the other has local SCSI storage.

In a shared-storage configuration, the everRun paired hosts must be connected to the same storage device with access to the shared LUN’s. In this configuration, everRun does not mirror the data or protect against failures within the storage subsystem. Because of this, it is critical that you ensure proper configuration of the storage devices to protect against failures.

Q: Should everRun be set up on a seperate server?
everRun is typically deployed on two new servers, however an existing server can be utilized, requiring only one additional server.

Q: How is everRun different from the NeverFail product?
Neverfail is an asynchronous DR solution with failover/restart capabilities.
 

We had a fantastic presentation last week from IT expert and author Niel Nickolaisen. Niel shared his proven methods for reducing downtime and improving the alignment of IT resources to better support business goals. If you weren’t able to attend the live event, you can watch the recorded version here.

If you prefer a white paper format, Niel’s strategies and best practices have also been summarized in a brand-new 8-page white paper, “Reduce Downtime by 70% - Without Spending a Dime” which you can download here.

The Q&A session from the live webinar with Niel Nickolaisen and Michael Bilancieri of Marathon has been summarized below:

Q: Can you give some tips on how I can educate my branch offices about my business continuity plan?
Niel Nickolaisen, CIO: At Headwaters, Inc., we have 120 remote sites. We approached this from an SLA perspective. We translated how the SLAs affected the operations at our branch locations. Then we communicated it and got them to buy into the SLAs and the things we were doing and suggested that they followed our lead.

Q: How often should you update your disaster recovery plans?
Niel Nickolaisen, CIO: In our case at Headwaters, Inc., we have Sarbanes-Oxley regulatory requirements. We do an annual formal risk assessment both for our business and for IT. When we’re done with that assessment we update our disaster recovery plans, which are based on the risks. Our disaster plan is designed to mitigate or recover from the risks that we’ve identified.

Q: How does everRun work?
At a high-level, everRun takes your entire Windows environment and protects it as a whole. Most protect from within the OS but we protect from underneath the OS. We clone to a second system for redundancy in a synchronous fashion. A good way to understand how everRun works is to watch our product demos videos and flash demos available on our website.

Q: How does everRun fit into a virtual environment?
everRun allows the ability to create multiple workloads on a single server. Our technology is based on virtualization technology – we’re virtualizing two instances to appear as one. You can create multiple workloads and put them on the same server and protect them. It’s based on Citrix XenServer.

Q: Will this work in conjunction with SAN offhost backups using Vertias Netbackup and FlashSnap option?
We are agnostic to the storage. If you’re using back-up right from the SAN, that’s fine. You can also use a mirrored option, where we can mirror the entire system in a synchronous fashion. That allows you to have SAN on one side and NAS on the other, or direct-attached, or both. It’s your choice, which gives you greater flexibility. You can separate the servers as well between buildings. The other option is a single copy of storage, not mirrored and both systems can connect to that storage, but the SAN device will then have to protect the data.

Q: How can Marathon contribute to companies considering a move to SAP?
everRun can provide availability and fault tolerant protection to that SAP environment. If you’re considering a move to SAP, I would assume you have had some discussions about how to protect that—the SLA, the data, availability and disaster recovery. everRun can protect and provide disaster tolerance disaster recovery, and high availability for that application, as well as data protection. We don’t cause any changes to the application.

Q: Should Marathon be brought in as a consultant before SAP is contracted?
Sometimes it’s a good idea to have a joint discussion with vendors. A lot of times when you look at availability and redundancy or data replication, it’s doing things to the applications and data and can cause interaction issues. Sometimes the application has to be configured in a certain way, so you want to know up front how your high availability solution could affect the data and application. We can certainly do a call with any other software vendors to have that conversation up front.

Q: What version of Windows does everRun support?
everRun supports Windows Server 2003 32-bit and 64-bit and Windows Server 2008 64-bit.

Q: What kind of performance impact does the synchronous lock-step have on the system?
That varies by application, users, data, I/O, and other factors. In general, it can range from 10-20% on your application – we’ve seen less than that and more than that, depending on the system.

Q: Do you recommend WAN optimization to be used?
Our requirements are around bandwidth between the two systems if you want to separate the systems. WAN optimization tools don’t always help. It’s really a latency requirement to maintain good performance.

If you'd like to learn more about Niel's best practices for aligning business and IT resources, be sure to check out his new book, Stand Back and Deliver: Accelerating Business Agility.

Congratulations to Thomas Burgdorf of Mii Management Group, the winner of a $100 American Express gift card from our recent everRun 2G demo webinar. If you weren’t able to attend the live event, a recording the everRun 2G demo webinar is now available for on-demand viewing here.

Be sure to join us for our next webinar on Oct. 8th, featuring IT process expert and author Niel Nickolaisen. We're really excited to have Niel as our guest speaker for this webinar. In addition to his 25+ years of IT experience, Niel is the CIO and Director of Strategic Planning at Headwaters, Inc. and also writes regular columns for the CIO Leadership Network and TechTarget's Search CIO. Niel is going to share his proven methods for reducing downtime, including:

* Conducting a gap analysis of your current IT processes
* Identifying weaknesses that can lead to downtime
* Simplifying IT processes so that your entire staff can understand and follow them

We're expecting a large group for this webinar, so be sure to register today to reserve your spot.

As companies become 24x7 “always on” operating environments, they are becoming more and more sensitive to application and system downtime. We recently conducted two surveys to take a look at this trend, specifically for Windows Server applications and environments.

Given the 24x7 nature of business today, we weren’t surprised to find that about half of the IT professionals who responded to the survey reported that 50% or more of their Windows Server applications now require “always on” availability. Whether it’s for email and collaboration tools, manufacturing systems, customer operations, financial transactions, or healthcare records, businesses today are becoming more and more dependent on their Windows-based applications.

More than 20% of survey participants reported that 76 -100% of their Windows Server applications require “always on” availability. An additional 28% of respondents stated that 51 - 75% of their Windows Server applications require “always on” availability or continuous uptime.

The surveys also found that that the number of Windows Server applications that require high availability has increased significantly in the past two years. 76% of the respondents reported that downtime to Microsoft SQL Server and Exchange Server caused the most disruption and were the most important applications that required high availability protection. The surveys also revealed that approximately 60% of participants have either already upgraded to Windows Server 2008 or plan to within the next year.

Thanks again to all who participated in these surveys. For more information and results, see our press release.
 

In recent months, Marathon has put together a series of toolkits with materials on reducing downtime and data loss, including toolkits for Citrix XenApp and Microsoft Exchange 2007.

Our latest toolkit is now available, this time for Microsoft SQL Server. Protecting SQL Server from downtime has become even more critical in recent years, as businesses run more of their critical systems, including electronic commerce, online banking, just-in-time manufacturing and streaming media (just to name a few) on SQL.

This toolkit includes materials on SQL Server high availability in both physical and virtual environments.

White paper: 5 Secrets to SQL Server Availability This paper reviews five proven secrets to affordable SQL high availability that will help IT managers implement a SQL Server environment with little or no downtime - and zero data loss.

White paper: The SSWUG.org Increasing Reliability and Availability in a Virtualized SQL Server Environment white paper, authored by Microsoft SQL Server MVP Stephen Wynkoop, provides IT professionals with best practices and considerations for designing and implementing a virtualized SQL environment including:

• Potential pitfalls to avoid when virtualizing SQL Server
• How to increase reliability and availability of a virtualized SQL Server environment
• A SQL Server virtualization case study (Sullivan Group)

On-Demand Webinar: SQL Availability: Protecting your Database and Applications featuring Microsoft SQL Server MVP Stephen Wynkoop, helps IT administrators understand SQL back-up and restore options. Wynkoop also presents his Concentric Rings of Recovery plan, which covers the four levels of preparedness for local, alternate, off-site and remote locations.

Also, be sure to check out some addtional SQL Server resources, including SQL user groups, SQL Server job boards, SQL MVP blogs and Twitter feeds, and other SQL-related info.
 

Congratulations to Richard Potter of Boeing, the winner of a $50 American Express gift card for participating in our September survey on Windows application high availability. To participate in future Marathon surveys, sign up for Marathon's monthly newsletter: http://www.marathontechnologies.com/news.html (see the right-hand column for sign-up.)