Automating cloud backups

Upon receiving JSON certificates from S1SEVEN, there are many options as to what you can do with them. One option that is often used for backup is to store certificates in a cloud storage solution such as Microsoft Azure.

This article will show how you can easily automate the backing up of certificates to Azure.

To simplify the process, we have created a flow that you can simply import and use to get started.

Installing dependencies

Before we import our pre-made flow, however, we must first install a dependency that our flow depends on. We use a pre-made Node that allows us to connect to Azure without having to code up a custom solution.

With node-red, installing modules is easy. Simply click on the top-right menu button and select Manage palette from the dropdown menu.

Choose Manage palette from the menu

We need to install the following Node:

Just click on the Install tab, type in node-red-contrib-azure-blob-storage-aleph and click install.

Installing node-red-contrib-azure-blob-storage-aleph

Once node-red-contrib-azure-blob-storage-aleph is installed, we have to install one other dependency, node-red-contrib-fs-ops.

Once node-red-contrib-azure-blob-storage-aleph and node-red-contrib-fs-ops have been installed, your Nodes tab should look like this:

All dependencies are installed

Now that all the dependencies have been installed, we are ready to import the pre-made flow.

Importing pre-made flows

Our pre-made flow can be easily imported into Node-Red and modified if necessary. To get started, click on the hamburger menu (the 3 horizontal lines) at the top right of the Node-Red toolbar (beside the Deploy button) and click import:

The import option in the menu

Then choose Clipboard, and copy-paste the code from under the following screenshot into the box, then click the redImport button:

Importing a flow

[
   {
       "id": "f2c63d7439af7969",
       "type": "tab",
       "label": "Folder to Azure",
       "disabled": false,
       "info": "",
       "env": []
   },
   {
       "id": "d7ecb73d85fc6823",
       "type": "debug",
       "z": "f2c63d7439af7969",
       "name": "",
       "active": true,
       "tosidebar": true,
       "console": false,
       "tostatus": false,
       "complete": "true",
       "targetType": "full",
       "statusVal": "",
       "statusType": "auto",
       "x": 610,
       "y": 240,
       "wires": []
   },
   {
       "id": "346ba95d6f7bb900",
       "type": "split",
       "z": "f2c63d7439af7969",
       "name": "Create 1 message per filename",
       "splt": "\\n",
       "spltType": "str",
       "arraySplt": 1,
       "arraySpltType": "len",
       "stream": false,
       "addname": "filename",
       "x": 970,
       "y": 140,
       "wires": [
           [
               "00286f1f67b3e6d4"
           ]
       ]
   },
   {
       "id": "f448a74028a9011e",
       "type": "switch",
       "z": "f2c63d7439af7969",
       "name": "Check msg.payload contains an array",
       "property": "payload",
       "propertyType": "msg",
       "rules": [
           {
               "t": "istype",
               "v": "array",
               "vt": "array"
           }
       ],
       "checkall": "true",
       "repair": false,
       "outputs": 1,
       "x": 650,
       "y": 140,
       "wires": [
           [
               "346ba95d6f7bb900"
           ]
       ]
   },
   {
       "id": "895bc0ef5add06e0",
       "type": "fs-ops-dir",
       "z": "f2c63d7439af7969",
       "name": "Read directory",
       "path": "path",
       "pathType": "msg",
       "filter": "*.json",
       "filterType": "str",
       "dir": "payload",
       "dirType": "msg",
       "x": 380,
       "y": 140,
       "wires": [
           [
               "f448a74028a9011e"
           ]
       ]
   },
   {
       "id": "9a5eddbef34e8142",
       "type": "inject",
       "z": "f2c63d7439af7969",
       "name": "Folder path input",
       "props": [
           {
               "p": "path",
               "v": "/Users/eamon/work/node-red-work/certificates",
               "vt": "str"
           }
       ],
       "repeat": "",
       "crontab": "",
       "once": false,
       "onceDelay": 0.1,
       "topic": "",
       "x": 180,
       "y": 140,
       "wires": [
           [
               "895bc0ef5add06e0"
           ]
       ]
   },
   {
       "id": "88782f492c67f5d8",
       "type": "comment",
       "z": "f2c63d7439af7969",
       "name": "Sends a separate message for each filename in the msg.files array",
       "info": "",
       "x": 780,
       "y": 80,
       "wires": []
   },
   {
       "id": "00286f1f67b3e6d4",
       "type": "function",
       "z": "f2c63d7439af7969",
       "name": "Construct filepath",
       "func": "msg.payload = 'certificates/' + msg.payload;\nmsg.blobname = msg.payload;\nmsg.filename = msg.payload;\nreturn msg;",
       "outputs": 1,
       "noerr": 0,
       "initialize": "",
       "finalize": "",
       "libs": [],
       "x": 170,
       "y": 240,
       "wires": [
           [
               "6e2078b113354381",
               "d7ecb73d85fc6823"
           ]
       ]
   },
   {
       "id": "efa5904d03596df7",
       "type": "comment",
       "z": "f2c63d7439af7969",
       "name": "Refresh Azure to see files",
       "info": "",
       "x": 870,
       "y": 240,
       "wires": []
   },
   {
       "id": "6e2078b113354381",
       "type": "Aleph Save Blob",
       "z": "f2c63d7439af7969",
       "name": "Azure Save Blob Storage",
       "x": 410,
       "y": 240,
       "wires": [
           [
               "d7ecb73d85fc6823"
           ]
       ]
   }
]

Once the flow has been imported, your workspace should look something like this:

Folder to Azure flow

This flow is quite simple. It starts with an inject node that sets msg.path to a file path. Here you need to get the absolute path to the folder on your machine that contains the JSON certificates you want to save to Azure and paste it in here.

For example, the path to my folder (on mac) is /Users/eamon/certificates/graph_certs/. Make sure that your path ends with a / if on Mac or Linux, or with a \ if on Windows. Otherwise, the flow will not be able to correctly read the files.

Read directory takes msg.path, gets a list of all the files in that directory, filters the list to files ending in .json, and stores the list as an array in msg.payload.

An array is a data structure used in programming. It is like a box that can be used to store data. In this case, it stores the filenames of the JSON files in the directory.

Read directory configuration

The next node then ensures that msg.payload contains an array. Normally node-red flows send just one message per flow, but the next node, Create 1 message per filename, creates a new message for each filename contained in the array, ensuring that the rest of the flow will run once for each filename.

Construct filepath contains some simple code:

msg.payload = 'certificates/' + msg.payload;
msg.blobname = msg.payload;
msg.filename = msg.payload;
return msg;

It adds ‘certificates/’ to the filename, and sets msg.blobname and msg.filename to the same filename. An example filename would be “certificates/valid_certificate.json”.

Finally, the message object arrives at Azure Save Blob Storage. Here we need to set the Storage Account Name, Storage Account Key, and Storage Container Name to allow the node to connect to Azure. You will find these in your Azure account settings.

The Azure node will then take care of saving each certificate that is passed to it to your Azure storage container. It will be saved to the path passed in by msg.blobname. If the blobname contains a /, it will automatically create a folder. For example, “certificates/valid_certificate.json” will be saved in a folder called certificates.

Expanding this flow

Now that you know how this flow works, you can modify and improve it. For example, instead of taking a folder of certificates as input, you could set up an mqtt node to listen for incoming certificates from S1SEVEN and store them automatically in Azure. You could even rename the certificates based on data contained within the certificate.

‍