Sunday, March 19, 2017

Data Protection in Public Cloud (Networker Backups in AWS)


When a cloud strategy is developed, it is very important to include data protection. As more and more enterprise applications move to cloud, only the resilient cloud infrastructure is not enough as corruptions, accidental deletion etc. can still happen and backup must be included in the cloud strategy.
Most cloud providers provide native snapshot functionality however snapshots are not the same as backup. Snapshots are not application consistent and do not support granular recovery.

The Dell EMC Networker with CloudBoost solution provides optimized data protection for enterprise applications regardless of whether they are running on-premises or in public cloud. Here are the various use cases.


Here I am going to share more about “Backup in the Cloud”, where Enterprises have workloads running in the public cloud virtual machines (here AWS EC2) and would like to use public cloud object storage (here AWS S3) to store the backups.

Solution Components

Networker is a unified data protection software solution for the enterprises that centralizes, automates and accelerates backup and recovery across multi-platform environments, both physical and virtual.
CloudBoost is a data protection-optimized cloud enabler for use on-premises and in the cloud. It employs source-side encryption, compression, deduplication and WAN optimization for highly efficient backup in the cloud.
AWS S3 is a highly scalable, reliable, fast, inexpensive object storage provided by AWS.

While testing this, I have used public cloud infrastructure provided by Amazon Web Service (AWS). Our production servers/VMs are running as EC2 instances in a private subnet in AWS VPC and have Networker client installed. These being the Linux clients, the backup data will directly flow to object storage (AWS S3) and only meta-data will go to Networker Server and CloudBoost Appliance.

In this demo, Networker server and CloudBoost appliance are also installed in private subnet. Public subnet hosts a jump host for external users to connect and a NAT gateway.
All the production VMs (EC2 instances) required to be backed up, which are running in private as well as public subnet. With this, let us start configuration tasks and test backup and restore of EC2 instances to S3 Bucket.

High level Steps:
  1. Install and Configure Networker Server and clients on EC2 instances
  2. Deploy the CloudBoost appliance from EC2 AMI
  3. Register and Configure CloudBoost appliance in the EMC Cloud Portal
  4. Configure Networker to work with CloudBoost appliance
  5. Perform Test Backups
  6. Perform Test Restore



1.       Install and Configure Networker Server and clients on EC2 instances
It is assumed that Networker Server and Clients are already installed and configured as per standard procedures on EC2 instances. Make sure that EC2 instance has sufficient role assigned to write to S3.

2.       Deploy CloudBoost appliance from EC2 AMI
Use the CloudBoost AMI to launch an EC2 instance of size m4.xlarge in Private subnet of VPC. Also make sure that require ports are opened in the Security Group and this instance can connect to DellEMC Cloud Portal on port 443 and EC2 instance has sufficient role assigned to write to S3.

Once deployed, ssh to CloudBoost and verify the status
Change Hostname and other details if required

admin@mag-fs> fqdn cloudboostappliance.ap-southeast-1.compute.internal
 * Processing request
 * Changing hostname
 * Restarting DHCP
 * Restarting configuration controller
 * Restarting configuration agent
 * Restarting statistics collection service.
FQDN changed successfully
admin@mag-fs> status
Host Configuration:
  Hostname:        cloudboostappliance
  Domain:          ap-southeast-1.compute.internal
  FQDN:            cloudboostappliance.ap-southeast-1.compute.internal
Version Information:
  Version:         3.6.5
  Revision:        72dc0c4a
Network Interfaces:
             name            mode         address         netmask
             ----            ----         -------         -------
             eth0            dhcp       10.0.2.32   255.255.255.0

Network Routes:
           prefix         netmask         gateway
           ------         -------         -------
          default         0.0.0.0 ip-10-0-2-1.ap-
         10.0.2.0   255.255.255.0               *

DNS Configuration
  DNS Servers:     10.0.0.2
Appliance status: Not yet registered

admin@mag-fs>


3.       Register and Configure CloudBoost appliance in the EMC Cloud Portal
Login to DellEMC Cloud Portal and create a Cloud Profile pointing to the AWS account by entering proper Region, AWS Access Key ID and Secret. This shall create the required Bucket in the given AWS Region to store the backup data.


Register the CloudBoost appliance using the claim code generated via CLI

admin@mag-fs> register
Please use claim code XXXXXXX to register this device with the cloud controller.
Waiting for console to claim the code .............
Appliance successfully registered.
admin@mag-fs>

Enter the claim code in the Cloud Portal and verify the status

admin@mag-fs> status
Host Configuration:
  Hostname:        cloudboostappliance
  Domain:          ap-southeast-1.compute.internal
  FQDN:            cloudboostappliance.ap-southeast-1.compute.internal
Version Information:
  Version:         3.6.5
  Revision:        72dc0c4a
Network Interfaces:
             name            mode         address         netmask
             ----            ----         -------         -------
             eth0            dhcp       10.0.2.32   255.255.255.0

Network Routes:
           prefix         netmask         gateway
           ------         -------         -------
          default         0.0.0.0 ip-10-0-2-1.ap-
         10.0.2.0   255.255.255.0               *

DNS Configuration
  DNS Servers:     10.0.0.2
Appliance status: Registered, Not yet configured

admin@mag-fs>

Once registered, it will be visible on Cloud Portal

Configure the appliance by providing necessary information as shown below



Once completed, verify the status

admin@mag-fs> status
Host Configuration:
  Hostname:        cloudboostappliance
  Domain:          ap-southeast-1.compute.internal
  FQDN:            cloudboostappliance.ap-southeast-1.compute.internal
Version Information:
  Version:         3.6.5
  Revision:        72dc0c4a
Network Interfaces:
             name            mode         address         netmask
             ----            ----         -------         -------
             eth0            dhcp       10.0.2.32   255.255.255.0

Network Routes:
           prefix         netmask         gateway
           ------         -------         -------
          default         0.0.0.0 ip-10-0-2-1.ap-
         10.0.2.0   255.255.255.0               *

DNS Configuration
  DNS Servers:     10.0.0.2
Appliance status: Initial configuration completed


4.       Configure Networker to work with CloudBoost appliance
Set the remote mount password on the CloudBoost to be used for backup

admin@mag-fs> remote-mount-password enable YOURpassword
100% done. Status: Password changed successfully
Changing remote mount password: success
admin@mag-fs>

Use Device Configuration Wizard in Networker NMC to configure CloudBoost Device as below

You can verify that couples of S3 Buckets are created with their names starting with “mag”

5.       Perform Test Backups
Configure the Networker Policy and Workflow to backup to new Pool created above which uses CloudBoost Device

Start the Workflow to run the backup

Query the savesets and verify the Media Type


As these backups are using Source Deduplication, you can see the space savings and dedup ratio on Cloud Portal





6.       Perform Test Restore
Let us perform a test restore from the backup just taken

Here is the successful restore


As we saw, NetWorker and CloudBoost solution enables data protection for enterprise applications running in the public cloud, on-premises in the data center or both.

Hope this is helpful!

11 comments:

  1. In cloud computing data backup is very important. This blog share useful information on AWS backup and restore. Thanks for sharing.

    ReplyDelete
  2. ITS Technology Solution Private Limited (itSimple) is a value based IT backup focused organisation.
    itSimple is offering consistently improving world-class products, services & knowledge in backup (Laptop/Desktop, & Servers) along with file Archival for media.
    See More Here at :
    Cloud Backup services
    Data Backup services

    ReplyDelete
  3. Hi Bhavan,
    Nice blog you have written..However i have a query as i will be implementing this solution soon.
    1)Can Networker be configured with Cloudboost without data domain.
    2)Can we write policy to move aged data to glacier from S3 and if we initiate restore of a VM from Networker whose data has moved to glacier,will that restore work.

    Awaiting your response..

    ReplyDelete
  4. very informative blog and useful article thank you for sharing with us , keep posting learn more about aws with cloud computing AWS Certification Training

    ReplyDelete
  5. Thanks for providing wonderful information with us. Thank you so much.

    AWS Technical Essentials Training

    ReplyDelete
  6. Those guidelines additionally worked to become a good way to recognize that other people online have identical fervor like mine to grasp a great deal more around this condition. and I could assume you are an expert on this subject. Same as your blog i found another one Amazon Master Class .Actually I was looking for the same information on internet for Amazon Master Class and came across your blog. I am impressed by the information that you have on this blog. Thanks a million and please keep up the gratifying work.

    ReplyDelete
  7. Thank you for sharing your thoughts and knowledge about Data Protection in Public Cloud. This is really helpful and informative, as this gave me more insight to create more ideas and solutions for my plan.
    We are offering 1-month free trial of backup on cloud and assuring the lowest price guarantee. Please Contact us: +91-9971329945
    Please visit us our website:
    web hosting
    cloud backup
    best linux web hosting services
    best windows hosting
    android cloud backup solutions

    ReplyDelete