Backup-as-a-Service for OpenStack Cloud

Cloud adoption is a key strategy for almost every organization today. These organizations are either aiming for or already have some form of IT Transformation and Modernization. Everyone is looking to leverage the advantages like scalability, elasticity, cost reduction, on-demand self service, flexibility etc. The "Cloud" can offer wide range of services to businesses. Backup as a Service (BaaS) is a strategy for backup and recovery that involves consuming these services to be managed by a Cloud Service Provider.

If you are a Cloud provider of an OpenStack Cloud, you can offer Backup as a Service to your cloud tenants using DellEMC’s OpenStack Data Protection Extension. 

If you are an OpenStack Cloud tenant or a consumer, you can ask for the Backup as a Service.

DellEMC’s OpenStack Data Protection Extension leverages Avamar and Data Domain. I am going to share my experience in a demo environment.

I will divide this in few parts as below

1. Glimpse of Integration
   1. High level architecture
   2. Components of OpenStack DPE
2. Cloud Provider Experience
   1. Add protection provider
   2. Register tenants / projects
   3. Create backup policies
3. Cloud Tenant Experience
   1. Register VM instances
   2. Protect VM instances in a scheduled backup policy
   3. On-demand backup
   4. Monitor backup status
   5. List backups
   6. Restore the VM instance
   7. File Level Recovery (FLR)

So let’s start…

1) Glimpse of Integration

In my demo environment, I have created OpenStack Cloud using RDO Ocata release. More information about it can be found on https://www.rdoproject.org/install/packstack/.

I am using Avamar version 7.5.1 and Data Domain 6.0 as my protection storage.

All connections are using flat network and Cinder volume backend is LVM which uses local volumes.

I am not using CBT in my demo setup. However for production environments, CBT feature provides incremental backups and upto 8 concurrent backups can run via a single Proxy instance.

High level Architecture

Here is a typical Architecture diagram of a multi node Openstack setup integrated with Avamar and Data Domain

Components of OpenStack DPE

These are the components providing Backup as a Service

Component	Description
Avamar Server	Backup Server - Also referred to as the Protection Provider
Data Domain	Provides scalable storage for backups, with features including source data deduplication
OpenStack DPE API	Management API for OpenStack DPE, provided as a qcow2 image. Includes the following: - API reference documentation via the Swagger UI. - OpenStack DPE file-level restore (FLR) module, which provides FLR capability and a web interface
OpenStack DPE UI	OpenStack Horizon management plug-in for graphical management of OpenStack DPE
OpenStack DPE Proxy	An Avamar proxy or worker, provided as a qcow2 image and registered to the Avamar server
OpenStack DPE Service	Integrates with OpenStack controller nodes for Keystone authentication and logging. Provided as an RPM package for RHEL distributions and as a DEB package for Ubuntu
DBT Driver and Agent	Provides the ability to complete incremental backups of changed data since the last full backup

Once integrated, “DELLEMC Data Protection” dashboard will appear in the Horizon UI.

2) Cloud Provider Experience

Add protection provider

As a Cloud provider, you would add the Avamar as a protection provider. Which means all the backups will be performed using this provider

Register Tenants / Projects

You would also choose which Cloud Tenant should be able to perform the backup. You may choose to register only selected Tenants based on the service agreement

Create Backup Policies

You can create Backup Policies for the Tenant. This will include Backup Schedule, Datasets to backup and Retention policy. Tenant will choose the required backup policy for their backups. You can also specify the backup quota for the Tenant.

This would create corresponding group policy in the Avamar system

3) Cloud Tenant Experience

Register VM instances

As a Cloud Tenant’s Project Administrator, you can choose which VM instances are required to be protected and register them for Data Protection

Protect VM instances in a scheduled backup policy

You can add the VM instance in the Backup Group Policy (provided by Cloud Provider) for scheduled backups

On-demand backup

You can also choose to run an on-demand backup of the VM instance

Monitor backup status

Status of the backup can be monitored via same Dashboard

This will create a clone of the Cinder volume, mount it on the DPE Proxy and transfer the backup data to Avamar + DataDomain

List backups

You can list all the backups of the VM instance

Restore the VM instance

You can choose to restore the entire VM from the backup. You can either overwrite the existing VM or restore it as a new VM instance

Check the restoration status

Once restored, it will be available under "Instances"

File Level Recovery (FLR)

Another beautiful feature of this OpenStack Data Protection Extension is File Level Recovery from the VM image backup performed.
Here is how you perform FLR. As a Cloud Tenant’s Project Administrator, you can list the backups associated with a VM instance and create an FLR session

Once the FLR ID and Token are created, it can be passed to end-user to browse their files via a web browser

Browse the backup, click on the name of the volume

Navigate to the path and click on the file to restore

Save the file that you want to restore

Once done, user can logout from the FLR UI
And Project Administrator can delete the FLR session

Hope this article helps you with OpenStack Data Protection.
Do share your feedback by commenting here or send it to bhavenjp@gmail.com. Good Day!

.

ScaleIO integration with OpenStack

I have recently tested ScaleIO 2.0 functionality with Openstack Cloud (Mitaka release) and would like to share it here.

Rather than describing detailed steps to install ScaleIO and OpenStack, as there are many references/documents available, I am describing the configuration and storage operations in this post.

Benefits: One can provision ScaleIO block volumes to OpenStack-managed VMs with Openstack UI or Cinder commands without learning native ScaleIO interface.

ScaleIO (Software defined, scale-out storage)

Downloaded the ScaleIO 2.0 software for Linux from ScaleIO Software Defined Block Storage | EMC. Detailed instructions on setting up ScaleIO Cluster is included in the documentation provided with download package. Here is my setup

• 3x Node Cluster consisting one Master MDM, one Slave MDM and one TieBreaker
• All three contain SDS and 1x 100GB disk from each SDS participate in 'default' storage pool under 'default' protection domain
• ScaleIO Gateway Server and GUI are running on a separate host in my environment.

Have also verified that the REST gateway is working fine which is included in the Gateway Server for ScaleIO.

With this we have a working setup of ScaleIO Cluster.

OpenStack (Open-source software platform for cloud computing, mostly deployed as an IaaS)

I have setup an OpenStack IaaS cloud. Used single node (allinone) RDO Mitaka running on CentOS 7 Linux server.

Details of the same can be found at https://www.rdoproject.org.

Created a Project 'Tenant01' and User 'User01' as a Tenant User. An instance 'Nova01' is created wihtout any Cinder volumes.

SDC is installed on the OpenStack Node (well.. all in one Node in my case) while ScaleIO drivers are already built-in from Liberty release.

Integration

Here is the typical integration diagram (from Liberty release though)

Edit the Cinder configuration file located /etc/cinder/cinder.conf to enable ScaleIO backend storage instead of default LVM

Restart Cinder Volume service.

Verify the volume logs from /var/log/cinder/volume.log file that drivers are successfully initialized.

Now let us test some of the operations related to Cinder volumes

• Create a Volume

Create an empty volume to be attached to existing Nova instance

You can see the volume logs showing ScaleIo volume is created

      And verify the same from ScaleIO CLI

• Attach the Volume to an Instance

Let us attach the volume created above to the existing instance 'Nova01'

Once the volume is attached, it can be detected inside the VM as any other disk.

• Extend a Volume

Current limitation on OpenStack Cinder is that volume must be detached before it can be extended. Here are the steps via CLI

Once attached back after extension, new size can be detected inside the VM

• Create, Mount/Verify (or restore required files) and Delete a snapshot

Let us create a partition, filesystem and mount the volume in the VM and write some test data to it

Create a snapshot of the volume

The snapshot created can be verified from ScaleIo CLI too

Let us delete one of the files that was created earlier

Now we have a situation where production data is missing or accidentally deleted. Let us restore the volume from the snapshot that we took

Attach the volume to the VM instance, mount it and restore the required data

 

• Detach and Delete a Volume

This is self explanatory

• Copy a Bootable Image to a Volume

Cinder volume can also be used as Bootable device by storing an Image on it. Let us create a new Volume from the Image source

This will copy the bootable image on the newly created volume

Which can be used to launch an instance as shown below

These are some of the basic tasks required by cloud users for daily operations.

Hope this is useful.