Thursday, June 8, 2017

Networker Backup automation using REST API

As the Enterprises embrace the Cloud Computing (be it public, private or hybrid), all of it's components (VM, Storage, Network etc) must be automated. And Backup or Data Protection is no exception. As someone said.... Automate or Perish.

The NetWorker REST API is an interface that provides programmatic access to the NetWorker data protection service. By using this REST API, one can build applications to automate NetWorker operations. This  helps Networker to be integrated with most Cloud Management Platforms to provide"Backup as a Service".

Starting from version 9.0.1, Networker released support for REST API and in version 9.1 it is upgraded to version two (v2) - with backward compatibility support for v1.

Here I have tested some basic operations like
  1. Configure a new client
  2. Perform backup of a client
  3. Configure a new VM client for VMWare level backup
  4. Perform image level backup of a VM
  5. Perform image level restoration of a VM

So let's get the hands dirty!

Installation:
As the NetWorker REST API service is deployed in the same Apache tomcat container as the NetWorker authentication service, no separate installation is required.
In my setup, the Networker Server hostname is "NWServer" so the REST API is exposed at URL https://NWServer:9090/nwrestapi/.

1. Configure a new client
1-1. List the Networker clients
  1. #curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients" -k -1  
  3.   
  4. {  
  5.     "clients": [  
  6.         {  
  7.             "aliases": [  
  8.                 "Client01",  
  9.                 "Client01.vlab.local"  
  10.             ],  
  11.             "applicationInformation": [],  
  12.             "blockBasedBackup": false,  
  13.             "checkpointEnabled": false,  
  14.             "clientId": "25f4943c-00000004-58aeae7b-58c8ee86-00025000-ff74b956",  
  15.             "hostname": "Client01.vlab.local",  
  16.             "indexBackupContent": false,  
  17.             "links": [  
  18.                 {  
  19.                     "href": "https://NWServer:9090/nwrestapi/v2/global/clients/47.0.232.4.
  20. 0.0.0.0.137.10.173.88.192.168.1.241",  
  21.                     "rel": "item"  
  22.                 }  
  23.             ],  
  24.             "nasDevice": false,  
  25.             "ndmp": false,  
  26.             "ndmpMultiStreamsEnabled": false,  
  27.             "ndmpVendorInformation": [],  
  28.             "parallelSaveStreamsPerSaveSet": false,  
  29.             "parallelism": 4,  
  30.             "protectionGroups": [],  
  31.             "remoteAccessUsers": [],  
  32.             "resourceId": {  
  33.                 "id": "47.0.232.4.0.0.0.0.137.10.173.88.192.168.1.241",  
  34.                 "sequence": 1  
  35.             },  
  36.             "saveSets": [  
  37.                 "All"  
  38.             ],  
  39.             "scheduledBackup": true,  
  40.             "tags": []  
  41.         },  
  42.         {  
  43.             "aliases": [  
  44.                 "NWServer",  
  45.                 "nwserver.vlab.local"  
  46.             ],  
  47.             "applicationInformation": [],  
  48.             "blockBasedBackup": false,  
  49.             "checkpointEnabled": false,  
  50.             "clientId": "b0556c84-00000004-58ad08e3-58ad08e2-00015000-ff74b956",  
  51.             "hostname": "nwserver.vlab.local",  
  52.             "indexBackupContent": false,  
  53.             "links": [  
  54.                 {  
  55.                     "href": "https://NWServer:9090/nwrestapi/v2/global/clients/82.0.114.31
  56. .0.0.0.0.160.8.173.88.192.168.1.241",  
  57.                     "rel": "item"  
  58.                 }  
  59.             ],  
  60.             "nasDevice": false,  
  61.             "ndmp": false,  
  62.             "ndmpMultiStreamsEnabled": false,  
  63.             "ndmpVendorInformation": [],  
  64.             "parallelSaveStreamsPerSaveSet": false,  
  65.             "parallelism": 12,  
  66.             "protectionGroups": [],  
  67.             "remoteAccessUsers": [],  
  68.             "resourceId": {  
  69.                 "id": "82.0.114.31.0.0.0.0.160.8.173.88.192.168.1.241",  
  70.                 "sequence": 3  
  71.             },  
  72.             "saveSets": [  
  73.                 "All"  
  74.             ],  
  75.             "scheduledBackup": true,  
  76.             "tags": []  
  77.         }  
  78.     ],  
  79.     "count": 2  
  80. }  

The same thing can be queried using any REST API client



1-2. Create a new client
Let us create a client name "01-SCM.vlab.local" and configure it to backup "/tmp/" saveset

  POST /nwrestapi/v2/global/clients

List  clients using REST API, however just filter the two fields (hostname and savesets) to be displayed
  1. #curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients?fl=hostname,savesets"
  3.  -k -1   
  4. {  
  5.     "clients": [  
  6.         {  
  7.             "hostname": "01-SCM.vlab.local",  
  8.             "saveSets": [  
  9.                 "/tmp"  
  10.             ]  
  11.         },  
  12.         {  
  13.             "hostname": "Client01.vlab.local",  
  14.             "saveSets": [  
  15.                 "All"  
  16.             ]  
  17.         },  
  18.         {  
  19.             "hostname": "nwserver.vlab.local",  
  20.             "saveSets": [  
  21.                 "All"  
  22.             ]  
  23.         }  
  24.     ],  
  25.     "count": 3  
  26. }  


1-3. Add the Client to an existing Protection Group
It is possible to create protection group, policy, workflow, action etc via REST API. However these are one time effort in most cases. Here it is assumed that Networker protection policies are setup and the newly created client needs to be added in the group to schedule the regular backups.

Let us make a PUT call to the protection group "Bronze-Filesystem" and add the "resourceId" of the client as a "workItem". The resourceId can be obtained from above mentioned GET API for a client.

PUT /nwrestapi/v2/global/protectiongroups/Bronze-Filesystem




2. Perform Backup of a Client
2-1. Perform On-demand Backup
Now let us run an on-demand backup for workflow "Filesystem" in Policy "Bronze".

POST /nwrestapi/v2/global/protectionpolicies/Bronze/workflows/Filesystem/op/backup




2-2. List the Backups
We can generate a list of all the backups performed for the client. This output shows that there are two backups found for this client.

  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/clients/45.0.92.4.0.0.0.0.
  3. 213.71.54.89.192.168.1.101/backups?fl=name,saveTime,retentionTime,instances" -k -1  
  4. {  
  5.     "backups": [  
  6.         {  
  7.             "instances": [  
  8.                 {  
  9.                     "clone": false,  
  10.                     "id": "1496737359",  
  11.                     "status": "Browsable",  
  12.                     "volumeIds": [  
  13.                         "2353985"  
  14.                     ]  
  15.                 }  
  16.             ],  
  17.             "name": "/tmp",  
  18.             "retentionTime": "2017-07-06T23:59:58+08:00",  
  19.             "saveTime": "2017-06-06T16:22:38+08:00"  
  20.         },  
  21.         {  
  22.             "instances": [  
  23.                 {  
  24.                     "clone": false,  
  25.                     "id": "1496736942",  
  26.                     "status": "Browsable",  
  27.                     "volumeIds": [  
  28.                         "2353985"  
  29.                     ]  
  30.                 }  
  31.             ],  
  32.             "name": "/tmp",  
  33.             "retentionTime": "2017-07-06T23:55:26+08:00",  
  34.             "saveTime": "2017-06-06T16:11:09+08:00"  
  35.         }  
  36.     ],  
  37.     "count": 2  
  38. }  
3. Configure a new VM client for VMWare level Backup
For most VMWare based cloud platforms, this is a mandatory step and can be provided as self service. Users can select their own VMs and add them to backup.

First let us get the UUID of the VM that needs to be protected

  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms?fl=hostname,uuid" -k -1   
  2. {  
  3.     "count": 2,  
  4.     "vms": [  
  5.         {  
  6.             "hostname": "vproxy_1",  
  7.             "links": [  
  8.                 {  
  9.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/50390eb3-02e9-19d7-3aa8-f5652bb40dbe",  
  10.                     "rel": "item"  
  11.                 }  
  12.             ],  
  13.             "uuid": "50390eb3-02e9-19d7-3aa8-f5652bb40dbe"  
  14.         },  
  15.         {  
  16.             "hostname": "prodserver",  
  17.             "links": [  
  18.                 {  
  19.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c",  
  20.                     "rel": "item"  
  21.                 }  
  22.             ],  
  23.             "uuid": "5039d1fc-00a0-0758-cfe8-16438fb2ea7c"  
  24.         },  
  25.     ]  
  26. }  

Here our VM "Prodserver" needs to be backedup and it's UUID is "1fc-00a0-0758-cfe8-16438fb2ea7c".
Let's add it to the VMWare protection group "Gold-Vmware"


POST /nwrestapi/v2/global/protectiongroups/Gold-Vmware/op/updatevmwareworkitems




The VM will be included in the backup from next schedule.

4. Perform image level Backup of a VM
With this, Cloud Management Platform can provide the on-demand backup feature for users.

POST /nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/vms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/op/backup



5. Perform image level Restoration of a VM
Here is how to restore a VM to revert to it's old state as of last successful backup.

Get the required the IDs of the backup to restore
  1. # curl -X GET -H "Content-Type: application/json" -H "Authorization: Basic XXXXXXXX" -H 
  2. "X-PrettyPrint:1" "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/vc01.vlab.
  3. local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-00000006-
  4. fe36ac21-5936ac21-00045000-7a80b956/instances" -k -1   
  5. {  
  6.     "backupInstances": [  
  7.         {  
  8.             "clone": false,  
  9.             "id": "1496755233",  
  10.             "links": [  
  11.                 {  
  12.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/
  13. vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-
  14. 00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755233",  
  15.                     "rel": "item"  
  16.                 }  
  17.             ],  
  18.             "status": "Recoverable",  
  19.             "volumeIds": [  
  20.                 "2353985"  
  21.             ]  
  22.         },  
  23.         {  
  24.             "clone": true,  
  25.             "id": "1496755806",  
  26.             "links": [  
  27.                 {  
  28.                     "href": "https://NWServer:9090/nwrestapi/v2/global/vmware/vcenters/
  29. vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-
  30. 00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755806",  
  31.                     "rel": "item"  
  32.                 }  
  33.             ],  
  34.             "status": "Recoverable",  
  35.             "volumeIds": [  
  36.                 "4263771792"  
  37.             ]  
  38.         }  
  39.     ],  
  40.     "count": 2  
  41. }  

then trigger the restoration

POST /nwrestapi/v2/global/vmware/vcenters/vc01.vlab.local/protectedvms/5039d1fc-00a0-0758-cfe8-16438fb2ea7c/backups/80524b93-00000006-fe36ac21-5936ac21-00045000-7a80b956/instances/1496755233/op/recover





Hope this was useful and hope to see more and more Data Protection and Automation for Cloud.

.
Posted by at
Labels: , , , , ,

11 comments:

  1. RJOctober 31, 2017 at 4:47 AM

    Informative blog.. Do you have any for adding DB Clients using REST API ? like for DB2 or Oracle ?

    ReplyDelete
    Replies
    1. Bhaven ParmarDecember 27, 2017 at 6:20 PM

      Hi Raj,
      You should be able to do that however lockbox part could be tricky. Haven't tried it yet. Will share if I manage to get a DB system in my lab.

      Delete
  2. UnknownOctober 10, 2018 at 9:55 PM

    Hello Bhaven this is indeed useful and nice . Do you have any other update on your test that you did using Rest API which could save time and it will be useful if you could share it . Best Regards - Karthik

    ReplyDelete
  3. RampyDJFebruary 21, 2019 at 11:43 AM

    Hello Bhaven,

    This was really informative, just had a query, when i try to extend the retention of a Backup of a particular client, by using PUT , it gives an error as :
    {
    "message": "HTTP 405 Method Not Allowed",
    "status": {
    "code": 405,
    "codeClass": "Client Error",
    "reasonPhrase": "Method Not Allowed"

    ReplyDelete
    Replies
    1. Bhaven ParmarFebruary 25, 2019 at 3:25 PM

      Hi Rampy,

      I do not think there is a method to update the backup retention.

      Delete
  4. rams4uJune 5, 2019 at 4:17 AM

    Hi Bhaven, I am using rest api client POSTMAN.


    https://NWServerIP:9090/nwrestapi/v3 #networker version 9.2
    Headers: content-type and Accept with value application/json
    Authorization: Basic Auth with username and password used to login NWserver

    Receiving 404 Error

    ReplyDelete
  5. AnonymousNovember 1, 2019 at 2:01 PM

    Nice Post and informative data. Thank you so much for sharing this good post, it was so nice to read and useful to improve my knowledge as updated one, keep blogging.
    Open Stack Training in Electronic City

    ReplyDelete
  6. Kristi RasconDecember 13, 2019 at 6:42 PM

    very nice post...informative blog
    Postman configuration – note to self, always make sure you turn off SSL verification in PostMan when you are using self-signed certifi....Read More

    ReplyDelete
  7. UnknownMay 3, 2021 at 10:55 PM

    Its really nice post and informative too . I am going to use these for sure thank you for sharing it . I am stuck in one report i was wondering if can help me . I have to report for last day backup status for all VM ( Snapshot backups) which shows VM name , status , start date , end date , and scanned size . If you can help me i will be really thank full to you

    ReplyDelete
  8. TechwellJanuary 12, 2022 at 11:37 PM

    vCloud Tech Provides secure IT services & Solutions and is considered by many organizations to be the best IT Software Reseller..

    ReplyDelete
  9. NelsinhoDecember 4, 2024 at 3:55 AM

    Hi, do you know how can I get the information of the column Status ? Succeeded/Failed ?

    ReplyDelete

Subscribe to: Post Comments (Atom)