1

Question

How can I rsync uncompressed (but highly compressible) data to a single compressed archive (.tar.gz, .zip, etc) without transferring all the data each time?

Current

This is what I am using now:

rsync -a --delete-during "/home/" "/mnt/external-hdd/backups/"

On the local side (/home/), there will be thousands of small text files that are compressible. On the "remote" side (/mnt/external-hdd/backups/), there will be an exact copy of the local side.

Wanted

  • local side: same as before
  • "remote" side: a homes.tar.gz that contains all of the data from /home, that can be incrementally updated like how rsync works by default

I will not be independently accessing specific files from homes.tar.gz. If an issue occurs, I will restore it to whatever homes.tar.gz contains.

This will significantly reduce storage size since data is compressed.

Prefarably, it should not require tar czf homes.tar.gz backups/; mv homes.tar.gz /mnt/external-hdd/backups/homes.tar.gz. That is, tarring and transferring all the data each time.

Improve this question
4
  • Why rsync? Rsync is not made for working with archives.
    – vidarlo
    Aug 18 at 21:21
  • @vidarlo rsync is just an example tool. I have mentioned it so it is clear what I want (behavior like rsync that can incrementally update the remote, but where the remote stores it as an archive (.tar.gz or other compressed format)).
    – fejyesynb
    Aug 18 at 22:03
  • We don't do product recommendations here, sadly. There's a lot of backup solutions out there that supports both compression and incremental backups.
    – vidarlo
    Aug 18 at 22:06
  • I saw gzip's --rsyncable flag and thought rsync itself might be able to handle my use case.
    – fejyesynb
    Aug 18 at 22:08

1 Answer 1

Reset to default
0

I would use for gzip compression:

tar zcf /mnt/external-hdd/backups/home-$HOSTNAME-$(date +%F_%H%M%S).tar.gz /home

If you need better compression, but slower, use xz:

tar Jcf /mnt/external-hdd/backups/home-$HOSTNAME-$(date +%F_%H%M%S).tar.xz /home

You can speed up a bit xz if you have multi core CPU:

tar cf - /home |xz -T 0 > /mnt/external-hdd/backups/home-$HOSTNAME-$(date +%F_%H%M%S).tar.xz
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .