Differences

This shows you the differences between two versions of the page.

--- computers:tricksandtips [2025/06/27 17:50] – [Grub Boot Parameters] david
+++ computers:tricksandtips [2026/03/13 16:43] (current) – [rclone] david
@@ Line 342: / Line 342: @@
 ====== Networking ======
+===== Copy Very Large Disk =====
+  - Rsync with resume-ability <code>
+sudo rsync -aHAXx --numeric-ids --info=progress2 --partial --partial-dir=.rsync-partial --append-verify --rsync-path="sudo rsync" /Volumes/raid/ user@remote-host:/Volumes/raidcopy/
+</code>
+  - Check after done with this <code>
+sudo rsync -aHAXx --numeric-ids --checksum --dry-run --itemize-changes --rsync-path="sudo rsync" /Volumes/raid/ user@remote-host:/Volumes/raidcopy/
+</code>
+Do not do this, if the transfer is interrupted, you start all over again, but this DOES work.
+<code>
+) run some generic command to authorize sudo remotely (pre-auth sudo)
+ssh -t username@remotesystem "sudo -v"
+) then this script will run sudo on your side and remote side
+time sudo ssh username@remotesystem "sudo tar -cpf - /remote/dir/to/copy | /PATH/TO/pigz --processes 16" | pigz -d | sudo tar -xpf - -C /destination/dir
+</code>
+===== Directory Sync with Unison =====
+  - Install on both machines<code>
+sudo apt install unison
+</code>
+  - Set up Passwordless SSH <code>
+ssh-keygen -t rsa -b 4096
+ssh-copy-id user@remote_host
+</code>
+  - Command to sync directories <code>
+unison /home/user/local_dir/ ssh://user@remote_host//home/user/remote_dir/
+</code>
+  - Creating a Sync Profile <code>
+# Root directories to sync
+root = /home/user/my_code
+root = ssh://user@remote_host//home/user/my_code
+# Automatic settings
+auto = true      # Skip prompts for non-conflicting changes
+batch = true     # Do not ask any questions at all
+times = true     # Preserve modification times
+# Paths to ignore (supports globs)
+ignore = Name {.git,node_modules,*.tmp}
+ignore = Path build
+</code>
+===== rclone =====
+If you want to avoid ssh/encryption (say you are moving many terabytes over 10GBe) to keep from slowing down your network.  You can use **rclone**
+**Server Side:**
+  - Start the server and use a non-standard port (using sudo to dig down into permissions of all files)<code>
+sudo /your/local/path/bin/rclone serve http /Directory/to/share --addr :8080
+</code>
+**Client Side:**
+  - **HDD - Spinny Disks on Both Sides**
+    - Use only 4 transfers at a time to keep heads from trashing <code>
+sudo rclone sync RCLONE_CONFIG: /destination/directory \
+    --metadata \
+    --skip-links \
+    --transfers 4 \
+    --checkers 4 \
+    --buffer-size 256M \
+    --multi-thread-streams 4 \
+    --log-file /somedir/log_file.log \
+    --log-level INFO \
+    -P
+</code>
+  - **SSD - Solid State Drive to Solid State Drive**
+    - To pull using SSD many transfers at once are fine: <code>
+sudo rclone sync RCLONE_CONFIG: /destination/directory \
+    --metadata \
+    --skip-links \
+    --transfers 16 \
+    --checkers 32 \
+    --buffer-size 256M \
+    --multi-thread-streams 8 \
+    --log-file /somedir/log_file.log \
+    --log-level INFO \
+    -P
+</code>
 ===== rsync limiting bandwidth =====