Differences

This shows you the differences between two versions of the page.

--- computers:tricksandtips [2026/01/15 15:30] – [Directory Sync with Unison] david
+++ computers:tricksandtips [2026/03/13 16:43] (current) – [rclone] david
@@ Line 341: / Line 341: @@
 ====== Networking ======
+===== Copy Very Large Disk =====
+  - Rsync with resume-ability <code>
+sudo rsync -aHAXx --numeric-ids --info=progress2 --partial --partial-dir=.rsync-partial --append-verify --rsync-path="sudo rsync" /Volumes/raid/ user@remote-host:/Volumes/raidcopy/
+</code>
+  - Check after done with this <code>
+sudo rsync -aHAXx --numeric-ids --checksum --dry-run --itemize-changes --rsync-path="sudo rsync" /Volumes/raid/ user@remote-host:/Volumes/raidcopy/
+</code>
+Do not do this, if the transfer is interrupted, you start all over again, but this DOES work.
+<code>
+) run some generic command to authorize sudo remotely (pre-auth sudo)
+ssh -t username@remotesystem "sudo -v"
+) then this script will run sudo on your side and remote side
+time sudo ssh username@remotesystem "sudo tar -cpf - /remote/dir/to/copy | /PATH/TO/pigz --processes 16" | pigz -d | sudo tar -xpf - -C /destination/dir
+</code>
 ===== Directory Sync with Unison =====
@@ Line 367: / Line 386: @@
 ignore = Name {.git,node_modules,*.tmp}
 ignore = Path build
+</code>
+===== rclone =====
+If you want to avoid ssh/encryption (say you are moving many terabytes over 10GBe) to keep from slowing down your network.  You can use **rclone**
+**Server Side:**
+  - Start the server and use a non-standard port (using sudo to dig down into permissions of all files)<code>
+sudo /your/local/path/bin/rclone serve http /Directory/to/share --addr :8080
+</code>
+**Client Side:**
+  - **HDD - Spinny Disks on Both Sides**
+    - Use only 4 transfers at a time to keep heads from trashing <code>
+sudo rclone sync RCLONE_CONFIG: /destination/directory \
+    --metadata \
+    --skip-links \
+    --transfers 4 \
+    --checkers 4 \
+    --buffer-size 256M \
+    --multi-thread-streams 4 \
+    --log-file /somedir/log_file.log \
+    --log-level INFO \
+    -P
+</code>
+  - **SSD - Solid State Drive to Solid State Drive**
+    - To pull using SSD many transfers at once are fine: <code>
+sudo rclone sync RCLONE_CONFIG: /destination/directory \
+    --metadata \
+    --skip-links \
+    --transfers 16 \
+    --checkers 32 \
+    --buffer-size 256M \
+    --multi-thread-streams 8 \
+    --log-file /somedir/log_file.log \
+    --log-level INFO \
+    -P
 </code>
 ===== rsync limiting bandwidth =====