Tip

Data migration checklist: What to do before, after and during a data migration

What you will learn in this tip: As your company grows, there's a good chance you'll eventually outgrow your current storage environment. While the original move from direct-attached storage (DAS)

Requires Free Membership to View

to an array might have been not too difficult, SAN storage data migration projects are much more labor-intensive and complicated. But with proper planning and our data migration checklist, you can ensure your project doesn't take more time and money than planned.

There's a common impression that data migrations are simply moving data from the old storage area network (SAN) storage array to the new. That impression is incorrect. Moving the data is just a very small part of data migration and generally the easiest part. And according to Bloor Research, 84% of SAN storage data migration projects are delivered over-time and/or over-budget. In other words, less than 16% of those projects are delivered on time and on budget.

A SAN storage data migration is a complicated series of manually intensive tasks. In fact, there are 43 discreet tasks; a mere six are semi-automated and 37 are labor-intensive (see "Data migration checklist" below). A task usually includes many steps or processes to complete. Take the task example of application, physical or virtual server data discovery and collection. All of the steps and processes must be repeated for each application, physical server and virtual server. But it is server remediation (making sure it will boot up, is up to date, has the correct drivers, micro codes, storage adapters, works with the new SAN storage, has the correct pathing, etc.) that takes huge swaths of time and effort, as it must be provided for each and every physical and virtual server. That’s a lot of steps for just one of the tasks.

SAN storage data migration checklist

DATA MIGRATION CHECKLIST
1 Identify hosts for migration
2 Collect migration host data (host audit)
3 Collect array data (array audit)
4 Collate zoning and masking info
5 Correlate all data
6 Migration and destination remediation analysis
7 Host remediation
8 Document source and target array software versions and compatibility
9 Configure source and target array to facilitate migration
10 Create relationship between source and target array
11 Collect all hosts' storage configurations: installed OS, cluster status, DR partner
12 Collect each array's configuration
  RAID, FA port configuration, local/remote replication status
13 Collect SAN fabric configuration: directors, switches, gateways
  Oversubscription details per port—fan-in/out ratios
14 Plan storage layout at target arrays
15 Create migration configuration and pairing scripts for source and target array
16 Create FA port mapping and target array LUN assignment configuration scripts
17 Create target array LUN masking scripts
18 Create device alias scripts
19 Prepare host to target arrays SAN zoning
20 Group migration hosts in cooperation w/bus units
21 Update all operational backup and recovery scripts
22 Perform all hosts audit prior to migration
23 Create-run R1 to R2 relationship pairing scripts
24 Create-run target array mapping and masking scripts
25 Apply all hosts zoning configurations
26 Confirm hosts connections at target array
27 Commence data migration to new array
28 Snap, replicate or backup all old host configurations
29 Quiesce (stop) apps, then shut down hosts
30 Create-run scripts that split new Array 1 and Array 2 relationship once data has been synched
31 Bring hosts back up to check and validate data
32 Restart apps
33 Get bus units to sign off data migration complete
34 Check for any activity on old arrays
35 Create-run scripts to delete Array1 and Array2 relationship
36 Create-run scripts to unmap devices from FA’s on source array
37 Create-run scripts to dissolve meta configurations on source arrays
38 Create-run scripts to unmask source devices from migrated hosts
39 Reboot hosts, confirm new target devices visibility
40 Clean up zones and remove all unused zones
41 Final login table check on old arrays
42 Update device groups if used
43 Reports and updates documenting entire process
Source: SANpulse

Most data migration projects are managed manually usually Microsoft Excel spreadsheets. This approach requires multi-departmental processes, procedures and, most importantly, discipline. This is because the data migration spreadsheet must be continuously updated by multiple administrators. 

Data migration technology timesavers

There are four technology workarounds to this manually intensive process and, while each has its trade-offs, all of them can significantly reduce data migration workloads and errors.

1.      Write unique scripts for some of the functions;
2.      Utilize external SAN storage virtualization appliances or systems;
3.      Leverage VMware vSphere Storage vMotion;
4.      Implement data migration packaged scripts and services with products like SANpulse.

Scripts are the most common workaround. The good thing about scripting is they are customized for the specific environment, storage, servers, applications, vendors and infrastructure. The bad thing about scripting is that they are rarely documented, tested for quality assurance, patched, or updated as the environment changes; they have to be rewritten for every SAN storage data migration project.

There has been much hope and hype with external SAN storage virtualization appliances and systems to simplify SAN storage data migration. It does this by allowing storage systems behind the virtualization appliance/system to be migrated without touching any of the servers or infrastructure in front of the virtualization appliance/system. The trade-off is that it adds considerable upfront and ongoing costs that increase the total cost of ownership (TCO). There is additional complexity in managing both the appliances/systems and backend storage arrays. Plus, all of the appliances/systems have scalability limitations equal to or a bit more than the storage systems they front end. Some require even greater data migration complexity (both the virtualization appliance/system and the backend storage systems must be migrated) when they go through a technology refresh. 

VMware’s Storage vMotion is becoming another popular workaround. It migrates the data from each of the physical SAN storage array LUNs assigned to that VMware server to LUNs on the new SAN storage array. The virtual LUNs assigned to the individual virtual machines (VMs) do not change, masking the actual physical change in the storage arrays. The downside is that this only works for the VMs running in a VMware infrastructure. It also doesn’t provide any SAN pathing remediation or VMware host remediation. Also, it doesn't move data from LUNs that are not assigned to that particular VMware host, and there is no data migration testing or validation.

Another workaround is the software, scripts and services packaged by SANpulse as SANlogics. SANlogics is designed to automate 27 SAN storage data migration processes and semi-automate another two. It can eliminate many human errors from the process; however, it only works on EMC Corp. and Hitachi Data Systems (HDS) storage systems today, and requires a data mover from EMC, HDS, or Symantec Corp.

SAN storage data migration will continue to be a daunting effort. This is why many small- to medium-sized businesses (SMBs) often turn to their SAN storage vendor data migration professional services. Yes, it’s quite expensive. But it gives you a way to hold your storage vendor accountable. Yet even this workaround has issues. It does not automate any of the processes, but instead, it just outsources the responsibility.

Whatever you choose, planning, discipline, focus, and attention to detail are the difference between success and failure when it comes to data migrations.

About the author: Marc Staimer is the founder, senior analyst, and CDS of Dragon Slayer Consulting in Beaverton, Ore. The consulting practice of 13 years has focused in the areas of strategic planning, product development and market development. With more than 31 years of marketing, sales and business experience in infrastructure, storage, server, software, and virtualization, he’s considered one of the industry’s leading experts. Marc can be reached at marcstaimer@mac.com

This article was originally published on SearchSMBStorage.com.

This was first published in April 2011

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.