Wednesday, July 13, 2022

PowerProtect Data Manager - Quick DR Recovery

 

PowerProtect quick DR recovery Demo:




Posted by at No comments:
Labels: , , , , , , , ,

Data Protection for Google Kubernetes Engine


Running cloud native apps on GKE and want to protect data, most importantly stateful apps? 

Look at Dell Technologies PowerProtect solution, a simple, efficient and cost-effective way to achieve this.



GKE Data Protection Demo using PowerProtect:




Posted by at No comments:
Labels: , , , , , , , , , ,

Saturday, October 13, 2018

Backup-as-a-Service for OpenStack Cloud

Cloud adoption is a key strategy for almost every organization today. These organizations are either aiming for or already have some form of IT Transformation and Modernization. Everyone is looking to leverage the advantages like scalability, elasticity, cost reduction, on-demand self service, flexibility etc. The "Cloud" can offer wide range of services to businesses. Backup as a Service (BaaS) is a strategy for backup and recovery that involves consuming these services to be managed by a Cloud Service Provider.

If you are a Cloud provider of an OpenStack Cloud, you can offer Backup as a Service to your cloud tenants using DellEMC’s OpenStack Data Protection Extension. 
If you are an OpenStack Cloud tenant or a consumer, you can ask for the Backup as a Service.

DellEMC’s OpenStack Data Protection Extension leverages Avamar and Data Domain. I am going to share my experience in a demo environment.
I will divide this in few parts as below
  1. Glimpse of Integration
    1. High level architecture
    2. Components of OpenStack DPE
  2. Cloud Provider Experience
    1. Add protection provider
    2. Register tenants / projects
    3. Create backup policies
  3. Cloud Tenant Experience
    1. Register VM instances
    2. Protect VM instances in a scheduled backup policy
    3. On-demand backup
    4. Monitor backup status
    5. List backups
    6. Restore the VM instance
    7. File Level Recovery (FLR)

So let’s start…

1) Glimpse of Integration

In my demo environment, I have created OpenStack Cloud using RDO Ocata release. More information about it can be found on https://www.rdoproject.org/install/packstack/.

I am using Avamar version 7.5.1 and Data Domain 6.0 as my protection storage.
All connections are using flat network and Cinder volume backend is LVM which uses local volumes.
I am not using CBT in my demo setup. However for production environments, CBT feature provides incremental backups and upto 8 concurrent backups can run via a single Proxy instance.

High level Architecture

Here is a typical Architecture diagram of a multi node Openstack setup integrated with Avamar and Data Domain

Components of OpenStack DPE

These are the components providing Backup as a Service
Component
Description
Avamar Server
Backup Server - Also referred to as the Protection Provider
Data Domain
Provides scalable storage for backups, with features including source data deduplication
OpenStack DPE API
Management API for OpenStack DPE, provided as a qcow2 image.
Includes the following:
- API reference documentation via the Swagger UI.
- OpenStack DPE file-level restore (FLR) module, which provides FLR capability and a web interface
OpenStack DPE UI
OpenStack Horizon management plug-in for graphical management of OpenStack DPE
OpenStack DPE Proxy
An Avamar proxy or worker, provided as a qcow2 image and registered to the Avamar server
OpenStack DPE Service
Integrates with OpenStack controller nodes for Keystone authentication and logging. Provided as an RPM package for RHEL distributions and as a DEB package for Ubuntu
DBT Driver and Agent
Provides the ability to complete incremental backups of changed data since the last full backup

Once integrated, “DELLEMC Data Protection” dashboard will appear in the Horizon UI.

2) Cloud Provider Experience

Add protection provider

As a Cloud provider, you would add the Avamar as a protection provider. Which means all the backups will be performed using this provider

Register Tenants / Projects

You would also choose which Cloud Tenant should be able to perform the backup. You may choose to register only selected Tenants based on the service agreement

Create Backup Policies

You can create Backup Policies for the Tenant. This will include Backup Schedule, Datasets to backup and Retention policy. Tenant will choose the required backup policy for their backups. You can also specify the backup quota for the Tenant.
This would create corresponding group policy in the Avamar system

3) Cloud Tenant Experience

Register VM instances

As a Cloud Tenant’s Project Administrator, you can choose which VM instances are required to be protected and register them for Data Protection

Protect VM instances in a scheduled backup policy

You can add the VM instance in the Backup Group Policy (provided by Cloud Provider) for scheduled backups

On-demand backup

You can also choose to run an on-demand backup of the VM instance

Monitor backup status

Status of the backup can be monitored via same Dashboard

This will create a clone of the Cinder volume, mount it on the DPE Proxy and transfer the backup data to Avamar + DataDomain

List backups

You can list all the backups of the VM instance

Restore the VM instance

You can choose to restore the entire VM from the backup. You can either overwrite the existing VM or restore it as a new VM instance
Check the restoration status
Once restored, it will be available under "Instances"

File Level Recovery (FLR)

Another beautiful feature of this OpenStack Data Protection Extension is File Level Recovery from the VM image backup performed.
Here is how you perform FLR. As a Cloud Tenant’s Project Administrator, you can list the backups associated with a VM instance and create an FLR session
Once the FLR ID and Token are created, it can be passed to end-user to browse their files via a web browser
Browse the backup, click on the name of the volume
Navigate to the path and click on the file to restore
Save the file that you want to restore
Once done, user can logout from the FLR UI
And Project Administrator can delete the FLR session

 Hope this article helps you with OpenStack Data Protection.
Do share your feedback by commenting here or send it to bhavenjp@gmail.com. Good Day!

 .
Posted by at 11 comments:
Labels: , , , , , , ,

Thursday, June 8, 2017

Networker Backup automation using REST API

As the Enterprises embrace the Cloud Computing (be it public, private or hybrid), all of it's components (VM, Storage, Network etc) must be automated. And Backup or Data Protection is no exception. As someone said.... Automate or Perish.

The NetWorker REST API is an interface that provides programmatic access to the NetWorker data protection service. By using this REST API, one can build applications to automate NetWorker operations. This  helps Networker to be integrated with most Cloud Management Platforms to provide"Backup as a Service".

Starting from version 9.0.1, Networker released support for REST API and in version 9.1 it is upgraded to version two (v2) - with backward compatibility support for v1.

Here I have tested some basic operations like
  1. Configure a new client
  2. Perform backup of a client
  3. Configure a new VM client for VMWare level backup
  4. Perform image level backup of a VM
  5. Perform image level restoration of a VM

So let's get the hands dirty!

Installation:
As the NetWorker REST API service is deployed in the same Apache tomcat container as the NetWorker authentication service, no separate installation is required.
In my setup, the Networker Server hostname is "NWServer" so the REST API is exposed at URL https://NWServer:9090/nwrestapi/.

1. Configure a new client
1-1. List the Networker clients
  1. #curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients" -k -1  
  3.   
  4. {  
  5.     "clients": [  
  6.         {  
  7.             "aliases": [  
  8.                 "Client01",  
  9.                 "Client01.vlab.local"  
  10.             ],  
  11.             "applicationInformation": [],  
  12.             "blockBasedBackup": false,  
  13.             "checkpointEnabled": false,  
  14.             "clientId": "25f4943c-00000004-58aeae7b-58c8ee86-00025000-ff74b956",  
  15.             "hostname": "Client01.vlab.local",  
  16.             "indexBackupContent": false,  
  17.             "links": [  
  18.                 {  
  19.                     "href": "https://NWServer:9090/nwrestapi/v2/global/clients/47.0.232.4.
  20. 0.0.0.0.137.10.173.88.192.168.1.241",  
  21.                     "rel": "item"  
  22.                 }  
  23.             ],  
  24.             "nasDevice": false,  
  25.             "ndmp": false,  
  26.             "ndmpMultiStreamsEnabled": false,  
  27.             "ndmpVendorInformation": [],  
  28.             "parallelSaveStreamsPerSaveSet": false,  
  29.             "parallelism": 4,  
  30.             "protectionGroups": [],  
  31.             "remoteAccessUsers": [],  
  32.             "resourceId": {  
  33.                 "id": "47.0.232.4.0.0.0.0.137.10.173.88.192.168.1.241",  
  34.                 "sequence": 1  
  35.             },  
  36.             "saveSets": [  
  37.                 "All"  
  38.             ],  
  39.             "scheduledBackup": true,  
  40.             "tags": []  
  41.         },  
  42.         {  
  43.             "aliases": [  
  44.                 "NWServer",  
  45.                 "nwserver.vlab.local"  
  46.             ],  
  47.             "applicationInformation": [],  
  48.             "blockBasedBackup": false,  
  49.             "checkpointEnabled": false,  
  50.             "clientId": "b0556c84-00000004-58ad08e3-58ad08e2-00015000-ff74b956",  
  51.             "hostname": "nwserver.vlab.local",  
  52.             "indexBackupContent": false,  
  53.             "links": [  
  54.                 {  
  55.                     "href": "https://NWServer:9090/nwrestapi/v2/global/clients/82.0.114.31
  56. .0.0.0.0.160.8.173.88.192.168.1.241",  
  57.                     "rel": "item"  
  58.                 }  
  59.             ],  
  60.             "nasDevice": false,  
  61.             "ndmp": false,  
  62.             "ndmpMultiStreamsEnabled": false,  
  63.             "ndmpVendorInformation": [],  
  64.             "parallelSaveStreamsPerSaveSet": false,  
  65.             "parallelism": 12,  
  66.             "protectionGroups": [],  
  67.             "remoteAccessUsers": [],  
  68.             "resourceId": {  
  69.                 "id": "82.0.114.31.0.0.0.0.160.8.173.88.192.168.1.241",  
  70.                 "sequence": 3  
  71.             },  
  72.             "saveSets": [  
  73.                 "All"  
  74.             ],  
  75.             "scheduledBackup": true,  
  76.             "tags": []  
  77.         }  
  78.     ],  
  79.     "count": 2  
  80. }  

The same thing can be queried using any REST API client



1-2. Create a new client
Let us create a client name "01-SCM.vlab.local" and configure it to backup "/tmp/" saveset

  POST /nwrestapi/v2/global/clients

List  clients using REST API, however just filter the two fields (hostname and savesets) to be displayed
  1. #curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients?fl=hostname,savesets"
  3.  -k -1   
  4. {  
  5.     "clients": [  
  6.         {  
  7.             "hostname": "01-SCM.vlab.local",  
  8.             "saveSets": [  
  9.                 "/tmp"  
  10.             ]  
  11.         },  
  12.         {  
  13.             "hostname": "Client01.vlab.local",  
  14.             "saveSets": [  
  15.                 "All"  
  16.             ]  
  17.         },  
  18.         {  
  19.             "hostname": "nwserver.vlab.local",  
  20.             "saveSets": [  
  21.                 "All"  
  22.             ]  
  23.         }  
  24.     ],  
  25.     "count": 3  
  26. }  


1-3. Add the Client to an existing Protection Group
It is possible to create protection group, policy, workflow, action etc via REST API. However these are one time effort in most cases. Here it is assumed that Networker protection policies are setup and the newly created client needs to be added in the group to schedule the regular backups.

Let us make a PUT call to the protection group "Bronze-Filesystem" and add the "resourceId" of the client as a "workItem". The resourceId can be obtained from above mentioned GET API for a client.

PUT /nwrestapi/v2/global/protectiongroups/Bronze-Filesystem




2. Perform Backup of a Client
2-1. Perform On-demand Backup
Now let us run an on-demand backup for workflow "Filesystem" in Policy "Bronze".

POST /nwrestapi/v2/global/protectionpolicies/Bronze/workflows/Filesystem/op/backup




2-2. List the Backups
We can generate a list of all the backups performed for the client. This output shows that there are two backups found for this client.

  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients/45.0.92.4.0.0.0.0.
  3. 213.71.54.89.192.168.1.101/backups?fl=name,saveTime,retentionTime,instances" -k -1  
  4. {  
  5.     "backups": [  
  6.         {  
  7.             "instances": [  
  8.                 {  
  9.                     "clone": false,  
  10.                     "id": "1496737359",  
  11.                     "status": "Browsable",  
  12.                     "volumeIds": [  
  13.                         "2353985"  
  14.                     ]  
  15.                 }  
  16.             ],  
  17.             "name": "/tmp",  
  18.             "retentionTime": "2017-07-06T23:59:58+08:00",  
  19.             "saveTime": "2017-06-06T16:22:38+08:00"  
  20.         },  
  21.         {  
  22.             "instances": [  
  23.                 {  
  24.                     "clone": false,  
  25.                     "id": "1496736942",  
  26.                     "status": "Browsable",  
  27.                     "volumeIds": [  
  28.                         "2353985"  
  29.                     ]  
  30.                 }  
  31.             ],  
  32.             "name": "/tmp",  
  33.             "retentionTime": "2017-07-06T23:55:26+08:00",  
  34.             "saveTime": "2017-06-06T16:11:09+08:00"  
  35.         }  
  36.     ],  
  37.     "count": 2  
  38. }  
3. Configure a new VM client for VMWare level Backup
For most VMWare based cloud platforms, this is a mandatory step and can be provided as self service. Users can select their own VMs and add them to backup.

First let us get the UUID of the VM that needs to be protected

  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms?fl=hostname,uuid" -k -1   
  2. {  
  3.     "count": 2,  
  4.     "vms": [  
  5.         {  
  6.             "hostname": "vproxy_1",  
  7.             "links": [  
  8.                 {  
  9.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/50390eb3-02e9-19d7-3aa8-f5652bb40dbe",  
  10.                     "rel": "item"  
  11.                 }  
  12.             ],  
  13.             "uuid": "50390eb3-02e9-19d7-3aa8-f5652bb40dbe"  
  14.         },  
  15.         {  
  16.             "hostname": "prodserver",  
  17.             "links": [  
  18.                 {  
  19.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c",  
  20.                     "rel": "item"  
  21.                 }  
  22.             ],  
  23.             "uuid": "5039d1fc-00a0-0758-cfe8-16438fb2ea7c"  
  24.         },  
  25.     ]  
  26. }  

Here our VM "Prodserver" needs to be backedup and it's UUID is "1fc-00a0-0758-cfe8-16438fb2ea7c".
Let's add it to the VMWare protection group "Gold-Vmware"


POST /nwrestapi/v2/global/protectiongroups/Gold-Vmware/op/updatevmwareworkitems




The VM will be included in the backup from next schedule.

4. Perform image level Backup of a VM
With this, Cloud Management Platform can provide the on-demand backup feature for users.

POST /nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/op/backup



5. Perform image level Restoration of a VM
Here is how to restore a VM to revert to it's old state as of last successful backup.

Get the required the IDs of the backup to restore
  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.
  3. local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-00000006-
  4. fe36ac21-5936ac21-00045000-7a80b956/instances" -k -1   
  5. {  
  6.     "backupInstances": [  
  7.         {  
  8.             "clone": false,  
  9.             "id": "1496755233",  
  10.             "links": [  
  11.                 {  
  12.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/
  13. vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-
  14. 00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755233",  
  15.                     "rel": "item"  
  16.                 }  
  17.             ],  
  18.             "status": "Recoverable",  
  19.             "volumeIds": [  
  20.                 "2353985"  
  21.             ]  
  22.         },  
  23.         {  
  24.             "clone": true,  
  25.             "id": "1496755806",  
  26.             "links": [  
  27.                 {  
  28.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/
  29. vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-
  30. 00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755806",  
  31.                     "rel": "item"  
  32.                 }  
  33.             ],  
  34.             "status": "Recoverable",  
  35.             "volumeIds": [  
  36.                 "4263771792"  
  37.             ]  
  38.         }  
  39.     ],  
  40.     "count": 2  
  41. }  

then trigger the restoration

POST /nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755233/op/recover





Hope this was useful and hope to see more and more Data Protection and Automation for Cloud.

.
Posted by at 11 comments:
Labels: , , , , ,
Subscribe to: Posts (Atom)