User Tools

Site Tools


code:work:lvisf:2022:fieldarchive:start

This is an old revision of the document!


Bash Style Archive

Compress All Dirs To A Folder

lvis@lvis-dell-xps:~/bin$ more compress_all_mjd.sh 
#!/bin/bash
 
echo "Usage $0 <input dir> <output dir>"
 
M_INPUTDIR=$(echo $1 | sed '/\/$/! s|$|/|')  # add a slash if necessary
M_OUTPUTDIR=$(echo $2 | sed '/\/$/! s|$|/|') # found online
 
echo Input: $M_INPUTDIR
echo Output: $M_OUTPUTDIR
 
cd $M_INPUTDIR
for f in *; do
    if [ -d "$f" ]; then
        M_BASENAME=$(basename $f)
        re='^[0-9]+$'  # check if this is a just a number (day mjd) (found online)
        if [[ $M_BASENAME =~ $re ]] ; then
           M_TARFILE="${M_OUTPUTDIR}${M_BASENAME=}_raw_data.tar.gz"
           echo "Run on $f which is $M_BASENAME"
           time tar cf - $f | pigz > $M_TARFILE
        fi
    fi
done

Mount Tar As Filesystem

Options we might be interested in

       -o readonly
              disable write support

       -o nobackup
              remove archive file backups
  1. Install if not already on the system
    sudo apt install -y archivemount
  2. Create a mount point
    mkdir /tmp/myfiles
  3. Mount the tar file as read only
    # read / write mount which makes a .orig file
    archivemount myfiles.tar.gz /tmp/myfiles
    
    # read / write mount without any backup
    archivemount -o nobackup myfiles.tar.gz /tmp/myfiles
    
    # read only
    archivemount -o readonly myfiles.tar.gz /tmp/myfiles
    
  4. Read data out of archive as if it were a file
  5. Remove the mount
    cd && umount /tmp/myfiles
code/work/lvisf/2022/fieldarchive/start.1649438248.txt.gz · Last modified: 2022/04/08 17:17 by david