1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. REMINDER

    Any content, information, or advice found on social media platforms and the wider Internet, including forums such as AP, should NOT be acted upon unless checked against a reliable, authoritative source, and re-checked, particularly where personal health is at stake. Seek professional advice/confirmation before acting on such at all times.

Digital Media Storage Methodologies

Discussion in 'Computer Related Help & Discussion' started by Nuluvius, May 15, 2020.

  1. Nuluvius

    Nuluvius New Member

    I have a mess of digital media to sort out, photos and videos, and I want a robust system to move forward with. I've been doing some research and want to offer up what I've found so far for discussion. I think I like the three tier system but need a final shove to decide. To be clear the top most root would be Pictures or Videos, these types are separate.

    Three Tier
    YYYY->YYYY-MM-DD Event->Photos
    • Flattest, most denormalized methodology
    • Feels like year directories could get very bloated
    • Special events i.e. holidays, that span monthly boundaries would be visually contiguous
    • Complex day events could have further sub directories:
      • Those with many sources such as other people and camera types
      • Events that are long like graduations may be divided into sections
    Four Tier
    1. YYYY->YYYY-MM->YYYY-MM-DD Event->Photos
    2. YYYY->MM->DD Event->Photos
    3. YYYY->MM Name->DD Event->Photos
    • Breaks things down further by month under the year level
    • More nested than Three Tier methodology
    • Holiday events that span month boundaries would be broken up by month so not visually contiguous
    • Same rules possible for complex events and multiple sources as Three Tier
    Five Tier
    1. YYYY->YYYY-MM->YYYY-MM-DD->YYYY-MM-DD Event->Photos
    2. Variants as above
    • Plus, all the naming variants as in Four Tier
    • Deep nesting
    • Could potentially have one or two directories in days
      • These could contain many or few photos
    • Same rules possible for complex events and multiple sources as Three Tier
    General Comments
    • With the Three Tier methodology the concern of ‘keys’ is not there because everything is flat under the year directory.
    • > Three tiers then we might start thinking in terms of a relational database i.e. primary and secondary keys:
      • YYYY is a primary key [PK] at level one but is a foreign key at level two and so on
      • YYYY[FK]-MM[PK]
      • YYYY[FK]-MM[FK]-DD[PK]
    • Do we even want such a bloated naming convention with more than three tiers?
    • It might be unwieldy to deal with anything beyond four tiers, we are humans and not computers after all
    • It is undesirable to rely on additional software such as Lightroom
    • The system must be as future proof as possible
    • It should be intuitive for all that may encounter it – this will be handed down and added to through the generations
    • Metadata is nice but it should be an independent concern that can be dealt with later/separately
    I would greatly appreciate your thoughts and input.
     
  2. GeoffR

    GeoffR Well-Known Member

    I use a simple system that almost works, it doesn't go to event/location level.
    I have a folder for the year in which are other folders for each month. There is then another folder into which the edited versions of the best from the year are saved.
    There is little need to have separate folders in each month for specific events because many are annual and can be found in the same month.
    Your computer's file system will append a date to each individual file but if you use several different cameras adding a folder for each day might be appropriate.

    Before you get too deeply involved with a system it might be worth investigating whether you can modify the file naming from your camera/s to identify which one was used for a particular shot. My cameras prefix each image file with _DSC files then being numbered sequentially, I can change the prefix to almost anything I want. Additionally, I can set the cameras to restart the numbering sequence with each memory card or just keep going irrespective of card changes. Thus, you might have NC1_0001 to NC1_0999 from one camera and NC2_0001 to 0999 on another assuming that you reset the numbering sequence for each card change.

    That may seem complicated but getting the file naming how you need it is actually quite important because otherwise you may end up with your computer adding a suffix if a file name is repeated e.g. _DSC0001 and _DSC0001_01 the _01 being the second file loaded.

    Obviously you can't retrospectively change the file naming except by editing each file individually (unless your computer can "replace" in file names) but this may help for future images.

    If you do adopt a filing system, and it is probably a good idea, it would be really useful to include a text file describing any conventions you use, such as camera names and settings, so that other users can do things in the same way.
     
    Nuluvius likes this.
  3. Stephen Rundle

    Stephen Rundle In the Stop Bath

    OMG..........................

    Whenever I do a shoot for a magazine/company I simply store all their images on a CF card with another as backup. The media is so cheap these days then file them
     
  4. GeoffR

    GeoffR Well-Known Member

    Really? Do you use XQD cards, I wouldn't call them cheap but CF certainly aren't terribly expensive.
     
  5. Stephen Rundle

    Stephen Rundle In the Stop Bath

    I use XQD cards and have done from the start but use CF cards as storage in a filing system, they are cheap(ish) usually what I do is keep then on CF card for a year, then transfer to DVD RAM
     
  6. PeteRob

    PeteRob Well-Known Member

    On Disk I store as YYYY\YYYY_MM_DD\imagefilename.ext

    Where imagefilename.ext is camera given. The underscore separator goes back to the Canon EOS utility default. i started like that and I’ll continue even though it means renaming folders if imported using anything else. I only save raw files now and I have a mix of different ages of Canon and Fuji.

    Originally I catalogued images by putting a jpg on Flickr. This let me put images in sets (now albums) and also attach key words. So in Flickr I could search by event (album) or keyword, find the picture, find the date taken, find it on disk. I can also look at images by date since Flickr implemented camera roll. It is useful for sharing images. Shortly after I started I went to work abroad so it was a way to stay in touch with home.

    Now, even though I use Lightroom for indexing, processing and printing, I still do the Flickr thing. In Lightroom it is easy to create a publish service and within that service create “albums” (collections). It is then trivial to add a photo to a collection, click publish and a jpg appears on Flickr in either a new or existing album and with the keywording from Lightroom attached. i have three basic publish services. Private - used to post sRGB jpg images for AP without a watermark and sized 800 px. Public - sRGB jpg images with a watermark and now sized 800 px. Friends and Family - sRGB images without a watermark and full resolution.

    If I was working commercially I’d probably also use the camera facility to custom name the images for the job. Else you can end up with a lot of ABCDimage001.CR2s
     
  7. Nuluvius

    Nuluvius New Member

    I haven't needed to modify the naming convention thus far since it is essentially a timestamp in the format of YYYYMMDD_HHmmtt for example IMG_20190324_141855.jpg

    The camera software is responsible for the append (mine is anyway) and it will do so if the shot was rapid i.e. within the same second.

    This is an excellent point and I had planned to do just that. Especially important as it will most likely be inherited and survive me.

    Do you have any backups or mechanisms to protect against bit rot/corruption? I would strongly recommend centralised storage on either a Btrfs or ZFS filesystem with a good backup (more than one) and replication strategy.

    The consensus seems to be leaning towards that system.

    That is an excellent point about leveraging the cloud photo storage providers for indexing the file based database. I had arrived at the same conclusion.
     

Share This Page