LLI Wiki — My Digital Knowledge Base

Cloud backup with rclone, robocopy and rsync

Sync pCloud, Google Drive or Storj to a local disk with rclone, then copy to a NAS with robocopy or rsync.

Back up cloud storage (pCloud, Google Drive, Google Photos) to a local disk with rclone, then copy it to a NAS with robocopy or rsync.

Configure an rclone remote

rclone.exe config

For pCloud, create an application on the provider's developer portal to get a client ID and a client secret (<PASSWORD>), then declare the redirect URI http://localhost:53682/ in the application settings. Keep passwords and secrets in a password manager.

For Storj storage, follow the Storj documentation on rclone with the native integration.

Sync the cloud to a disk

sync makes the destination identical to the source (deletions included); copy never deletes. -P (--progress) shows progress.

rclone sync pcloud_back:/ "F:\pcloud_rclone" -P
rclone copy pcloud_back:/ "F:\pcloud_rclone" -P

With an exclusion file and tuned parallelism:

rclone sync pcloud_back:/ "H:\pcloud_rclone" --exclude-from excludePcloud.txt --progress --checkers 10 --transfers 5

Content of excludePcloud.txt:

.Trash*/**
.davfs.*

For rate-limited remotes (Google Drive, Google Photos), low parallelism is used:

rclone sync gdrive_user1:/ "H:\gdrive_user1_rclone" --exclude-from excludePcloud.txt --progress --checkers 2 --transfers 1
rclone sync gphotos_user1:/ "H:\gphotos_user1_rclone" --exclude-from excludePcloud.txt --progress --checkers 2 --transfers 1

Compare two local folders without changing anything:

rclone sync H:\pcloud_rclone D:\backup\pcloud_rclone --dry-run

Sync a local folder to Storj

Limit parallelism and bandwidth (here 2250 KiB/s on the schedule, OFF for the rest):

rclone sync /nas/backup/download/ storj_various:/rclonevarious/test --progress
rclone sync /nas/backup/download/ storj_various:/rclonevarious/test --progress --checkers=2 --transfers=2 --bwlimit 2250:OFF

Copy the disk to a NAS with robocopy

robocopy H:\pcloud_rclone \\192.0.2.10\backups\pcloud_back /E /DST /FFT /XO /XJ
Option Meaning
/E copy subfolders, including empty ones
/DST compensate for daylight saving time differences
/FFT tolerate the 2-second timestamp granularity of FAT/NAS file systems
/XO exclude source files older than the destination
/XJ exclude junction points

The same principle copies a music share to an external disk:

robocopy \\192.0.2.20\music F:\Music_back /E /DST /FFT /XO /XJ

Copy to a NAS with rsync from WSL

A Windows symbolic link makes the disk easy to reach from WSL:

mklink "c:\mnt\usb_pcloud" "F:\pcloud_rclone"

Then, in the WSL shell (SSH on a non-standard port, here 222):

sudo rsync --ignore-existing --progress -vr --rsh='ssh -p222' --rsync-path=/usr/bin/rsync /mnt/c/mnt/usb_pcloud/* user@example.com:/backups/pcloud_back

Variant with the -X option (extended attributes), which is not compatible with the NAS ACLs; --dry-run simulates the operation:

rsync -vrX --delete --dry-run --progress --rsh='ssh -p222' --rsync-path=/usr/bin/rsync /mnt/c/mnt/usb_pcloud/* user@example.com:/backups/pcloud_back

Copy a mounted Windows share to an external SSD on Linux:

rsync -azP --delete --ignore-existing --stats --human-readable ~/WindowsShare/pcloud_rclone/ /run/media/user/ssd/backup_pcloud/

rclone scenarios on Linux

Compare while ignoring case (useful when the source is a Windows share):

rclone check pcloud:/ ~/WindowsShare/pcloud_rclone/ --local-case-insensitive
rclone sync pcloud:/ ~/WindowsShare/pcloud_rclone/ --progress --retries 3 --local-case-insensitive --checkers 16 --transfers 8 --exclude ".Trash*/**" --exclude ".davfs.*"

From the cloud to a local SSD (with --dry-run to simulate):

rclone sync pcloud:/ /run/media/user/ssd/backup_pcloud/ --progress --retries 3 --local-case-sensitive --checkers 16 --transfers 8 --exclude ".Trash*/**" --exclude ".davfs.*" --dry-run

From the cloud to a VPS, on a mounted volume (for example a storage box):

rclone sync pcloud:/ /mnt/storagebox/pcloud_copy/ --progress --retries 3 --local-case-sensitive --checkers 16 --transfers 8 --exclude ".Trash*/**" --exclude ".davfs.*"

Mount a remote with rclone mount

On Linux, rclone mount requires FUSE. On some distributions, fusermount3 must exist:

ln -s /bin/fusermount /bin/fusermount3
rclone mount pcloud: mnt/pcloud/ --daemon

Sync a folder to another Windows server

With more parallelism, a larger buffer and multi-threaded transfers:

& .\rclone sync "e:\data" "\\192.0.2.11\e$\data" --progress --transfers 8 --checkers 16 --buffer-size 64M --multi-thread-streams 8 --exclude "tmp/**"
Mastodon LLIMastodon SocialFosstodon