User Tools

Site Tools


code:work:lvisf:2022:fieldarchive:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
code:work:lvisf:2022:fieldarchive:start [2022/04/08 17:49] – [Database] davidcode:work:lvisf:2022:fieldarchive:start [2022/04/08 17:56] (current) – [Database] david
Line 36: Line 36:
   - Just have the file name either ARCHIVE_DRIVESERIALNUMBER.md5.txt for each drive that is an archive (have blown up the md5deep information within each tar?)   - Just have the file name either ARCHIVE_DRIVESERIALNUMBER.md5.txt for each drive that is an archive (have blown up the md5deep information within each tar?)
   - for the orignal and mirror do MIRROR_DRIVESERIALNUMBER.md5.txt with the information (append on any new files not already in the file... touch the file if non-exists)   - for the orignal and mirror do MIRROR_DRIVESERIALNUMBER.md5.txt with the information (append on any new files not already in the file... touch the file if non-exists)
 +
 +
 +Ingest script:
 +  - Check the mount point to see if it is FLIGHT, MIRROR or ARCHIVE (want to expand this to have GPS, APPLANIX also?  Could also have specific drive serial numbers that are known data drives (USB sticks or Applanix CF cards for instance))
 +  - Grab the drive Make and Serial Number to use in file name creation
 +  - Ingest Logic on each file
 +    - Does the meta data exist? MIRROR_DRIVESERIALNUMBER.meta.txt  If not.. just touch it!
 +    - How to avoid double md5deep calculation ?  (if you do the -X it calculates and compares, then we could just run on the output of that... maybe that's a special --switch to make it go faster)
 +      - Do it and then see if entry is already in the file... if so skip, if not add  This will catch files with different times, etc...  I think this makes the most sense, least likely to have an issue and defaults to ingest this data.
 +
 +
 ===== Compress All Dirs To A Folder ===== ===== Compress All Dirs To A Folder =====
  
code/work/lvisf/2022/fieldarchive/start.1649440183.txt.gz · Last modified: 2022/04/08 17:49 by david