Differences

This shows you the differences between two versions of the page.

--- code:work:lvisf:2022:fieldarchive:start [2022/04/08 17:36] – david
+++ code:work:lvisf:2022:fieldarchive:start [2022/04/08 17:56] (current) – [Database] david
@@ Line 25: / Line 25: @@
 later you can add MMMMM_[1-z].tar.gz for any additional files that were missed?
+==== Database ====
+Ideas for this in the field
+  - Can we generate a csv (white space or commas?) data base of all the files that are on original and mirror drives?
+    - Disk Label, Disk Serial Number would be additional fields to the md5deep with size and access time information so we can quickly assess how much of each kind of data we have (oh top level directory type stats also:  gps, imu, camera, rtlvis running stats for all the dirs one down from Mission level
+How about
+  - Just have the file name either ARCHIVE_DRIVESERIALNUMBER.md5.txt for each drive that is an archive (have blown up the md5deep information within each tar?)
+  - for the orignal and mirror do MIRROR_DRIVESERIALNUMBER.md5.txt with the information (append on any new files not already in the file... touch the file if non-exists)
+Ingest script:
+  - Check the mount point to see if it is FLIGHT, MIRROR or ARCHIVE (want to expand this to have GPS, APPLANIX also?  Could also have specific drive serial numbers that are known data drives (USB sticks or Applanix CF cards for instance))
+  - Grab the drive Make and Serial Number to use in file name creation
+  - Ingest Logic on each file
+    - Does the meta data exist? MIRROR_DRIVESERIALNUMBER.meta.txt  If not.. just touch it!
+    - How to avoid double md5deep calculation ?  (if you do the -X it calculates and compares, then we could just run on the output of that... maybe that's a special --switch to make it go faster)
+      - Do it and then see if entry is already in the file... if so skip, if not add  This will catch files with different times, etc...  I think this makes the most sense, least likely to have an issue and defaults to ingest this data.