Saturday, December 2, 2017

Encrypt & Backup your Data With Rclone

Rclone logo


This tutorial is compatible with any storage service such as Amazon, Google Drive, buckets, SSHs or FTPs. However, I'll be using Backblaze.com buckets. As of the date of this post,Backblaze offers 200GB for $1/month, which is the cheapest on the net, followed by OVH cloud storage and Hubic (Also owned by OVH).

This tutorial will cover the following:
1- Creating remote connection
2- Creating remote encrypted connection
3- Upload Data
4- Download Data
5- Different Commands & Flags


1- Creating remote connection

Configure rclone and follow the steps
rclone config

Enter n for new connection
n

I'll call my new connection "remote"

I'll store my backups on backblaze, make sure to create a bucket first from backblaze's admin panel, I'll call my bucket etra-test

Account ID & key number can be found in backblaze, buckets section

You can leave "Endpoint" empty

This is my output:
rclone config

Current remotes:



Name                 Type

====                 ====




e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q> n

name> remote

Type of storage to configure.

Choose a number from below, or type in your own value

 1 / Amazon Drive

   \ "amazon cloud drive"

 2 / Amazon S3 (also Dreamhost, Ceph, Minio)

   \ "s3"

 3 / Backblaze B2

   \ "b2"

 4 / Box

   \ "box"

 5 / Dropbox

   \ "dropbox"

 6 / Encrypt/Decrypt a remote

   \ "crypt"

 7 / FTP Connection

   \ "ftp"

 8 / Google Cloud Storage (this is not Google Drive)

   \ "google cloud storage"

 9 / Google Drive

   \ "drive"

10 / Hubic

   \ "hubic"

11 / Local Disk

   \ "local"

12 / Microsoft Azure Blob Storage

   \ "azureblob"

13 / Microsoft OneDrive

   \ "onedrive"

14 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)

   \ "swift"

15 / QingClound Object Storage

   \ "qingstor"

16 / SSH/SFTP Connection

   \ "sftp"

17 / Yandex Disk

   \ "yandex"

18 / http Connection

   \ "http"

Storage> 3

Account ID

account> myaccountnumber

Application Key

key> mykeynumber

Endpoint for the service - leave blank normally.

endpoint> 

Remote config

--------------------

[myremote]

account = myaccountnumber

key = mykeynumber

endpoint = 

--------------------

y) Yes this is OK

e) Edit this remote

d) Delete this remote

y/e/d> y

Current remotes:



Name                 Type

====                 ====

remote             b2



e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q>




2- Creating remote encrypted connection

Create new remote in rclone by entering n for new connection
n


I'll call my new connection "encremote" which stands for encrypted remote

Choose #6 "Encrypt/Decrypt a remote"

I'll be backing up my data in the previously created bucket called etra-test, so the remote to encrypt will be "remote:etra-test".

Create a hard to guess password and a hard to guess salt, make sure to not lose these, otherwise you won't be able to decrypt your data in the future.

Click q to quit
q


This is my output:
e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q> n

name> encremote      

Type of storage to configure.

Choose a number from below, or type in your own value

 1 / Amazon Drive

   \ "amazon cloud drive"

 2 / Amazon S3 (also Dreamhost, Ceph, Minio)

   \ "s3"

 3 / Backblaze B2

   \ "b2"

 4 / Box

   \ "box"

 5 / Dropbox

   \ "dropbox"

 6 / Encrypt/Decrypt a remote

   \ "crypt"

 7 / FTP Connection

   \ "ftp"

 8 / Google Cloud Storage (this is not Google Drive)

   \ "google cloud storage"

 9 / Google Drive

   \ "drive"

10 / Hubic

   \ "hubic"

11 / Local Disk

   \ "local"

12 / Microsoft Azure Blob Storage

   \ "azureblob"

13 / Microsoft OneDrive

   \ "onedrive"

14 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)

   \ "swift"

15 / QingClound Object Storage

   \ "qingstor"

16 / SSH/SFTP Connection

   \ "sftp"

17 / Yandex Disk

   \ "yandex"

18 / http Connection

   \ "http"

Storage> 6

Remote to encrypt/decrypt.

Normally should contain a ':' and a path, eg "remote:path/to/dir",

"remote:bucket" or maybe "remote:" (not recommended).

remote> remote:etra-test

How to encrypt the filenames.

Choose a number from below, or type in your own value

 1 / Don't encrypt the file names.  Adds a ".bin" extension only.

   \ "off"

 2 / Encrypt the filenames see the docs for the details.

   \ "standard"

 3 / Very simple filename obfuscation.

   \ "obfuscate"

filename_encryption> 2

Password or pass phrase for encryption.

y) Yes type in my own password

g) Generate random password

y/g> y

Enter the password:

password:

Confirm the password:

password:

Password or pass phrase for salt. Optional but recommended.

Should be different to the previous password.

y) Yes type in my own password

g) Generate random password

n) No leave this optional password blank

y/g/n> y

Enter the password:

password:

Confirm the password:

password:

Remote config

--------------------

[encremote]

remote = remote:etra-test

filename_encryption = standard

password = *** ENCRYPTED ***

password2 = *** ENCRYPTED ***

--------------------

y) Yes this is OK

e) Edit this remote

d) Delete this remote

y/e/d> y

Current remotes:



Name                 Type

====                 ====

encremote          crypt

remote                b2



e) Edit existing remote

n) New remote

d) Delete remote

r) Rename remote

c) Copy remote

s) Set configuration password

q) Quit config

e/n/d/r/c/s/q> q



3- Upload Data

I'll be uploading the content of the directory /home/pc/pictures (Similar to C:\Users\pc\Pictures in Windows) to the encrypted remote "encremote". "encremote" will automatically encrypt the data using the password and salt that was entered and will send them to "remote", "remote" will then upload them automatically to the Backblaze bucket.

rclone copy /home/pc/pictures encremote:


This will copy everything from /home/pc/pictures to the Backblaze bucket, files that already exist will be skipped.

You can also use "sync", though I personally wouldn't recommend it. "sync" will synchronize all the files in the source directory (/home/pc/pictures) to the remote bucket, if the remote directory has files that the source don't have, they will be deleted.

rclone sync /home/pc/pictures encremote:


4- Download Data

I'll be downloading the files from the remote "encremote" to /home/pc/test. If you want to download them to another PC, make sure to follow Step 1 and Step 2 with the same password and salt.

rclone copy encremote: /home/pc/test

This will copy everything from encremote: to /home/pc/test and skip the files that already exist

5- Different Commands & Flags

You can always check the content of a remote or an encrypted remote.
The following will show the content of the encrypted remote:
rclone ls encremote:

The following will show the encrypted files and directories inside the bucket
rclone ls remote:

When copying files, you can use several paramters, most importantly
--bwlimit intValue
This will limit your bandwidth upload speed (in KB/s)

You can use -v to print details about the copy/sync. You can use -v several times for more details, example
-v
or
-v -v -v

You can also limit the number of files to copy, by default rclone copy 4 files at the same time.
--transfers intValue

Example:
rclone copy --bwlimit 55 -v -v -v --transfers 2 /home/pc/pictures encremote:




Questions or comments? Post below