« Back to list view

blocksync.py

New project: https://github.com/nethappen/blocksync-fast

 

https://github.com/theraser/blocksync
(There are many forks, this seems to be the most complete and maintained)

Tuturial on moving kvm images with blocksync:
https://serverfault.com/questions/434064/correct-way-to-move-kvm-vm/648871#648871

Blocksync can sync only from a device, not a file.
It's perfect to turn a virtual machine e.g. from lvm to a raw file image.

Installation on Ubuntu

Login as root

Options

  • --blocksize  4194304, or 16777216
  • --createdest = create target file
    • should be safe to use in subsequent calls, as truncate() modifies the file only if it's given a different file size than the current one (it either truncates it to a lower size or pads it with zeros if larger).
  • --workers = number of workers to fork 4-8 -> parallel processing -> fast!
  • --splay=250 = add a sleep interval between forks
    • Use in combination with --workers
  • --script = path to the script on the remote side, otherwhise it is copied to /tmp
    • --script=/root/bin/blocksync.py
  • --cipher= you have to give a ssh cipher
    • aes256-ctr  -> widly supported
    • aes128-gcm@openssh.com -> faster?
    • List available ciphers with "ssh -Q cipher"

Examples

  • ~/blocksync.py --blocksize=4194304 /dev/xyz localhost /mnt/myserver.root.img
  • ~/blocksync.py --extraparams="-p 2222" --cipher=aes256-ctr --blocksize=4194304 --workers=4 /dev/xyz root@target.example.com /srv/libvirt_images/myserver.root.img

Workaround for img to img

  • kpartx -al /srv/libvirt_images/myserver.root.img
    • /dev/loop0
  • ~/blocksync.py --extraparams="-p 2222" --cipher=aes256-ctr --blocksize=4194304 --workers=4 /dev/loop0 root@target.example.com /srv/libvirt_images/myserver.root.img
  • kpartx -r /srv/libvirt_images/myserver.root.img