Discussion in 'Computer Related Help & Discussion' started by swanseadave, Aug 16, 2017.
I am just trying to establish his credentials for dismissing most of the advice in the article.
My credentials are irrelevant, it's an article on the Internet, which means it's also just opinion. So you've now got two opinions. If I'd written my response as a blog post supported by advertising, would it carry any more weight?
And if you want more opinions, read the comments on the original blog,
For those who want to know more there's always Wikipedia https://en.wikipedia.org/wiki/CompactFlash https://en.wikipedia.org/wiki/Secure_Digital and https://en.wikipedia.org/wiki/Flash_memory
But perhaps more important to question the credentials of the article writer...
That`s what I`m beginning to wonder Nick.
Have I opened Pandora`s Box starting this thread?
Seriously,I came across it on Cambridge in Colour and thought it important enough to share.
I didn`t expect such a huge response.It was just the first paragraph that concerned me.The rest,for the most part
told me nothing I didn`t already know.
When I was customer relations manager for a certain car company, our Sales and Marketing Director was brilliant. Knew everything you could possibly need to know about selling and marketing cars, was an absolute master of letting the sizzle sell the sausage, but knew the square root of sweet Fanny Adams about how cars worked. He didn't need to, it wasn't his job. On the other hand, our Service Technical Manager had a complete understanding of all the technologies involved and all the risks and pitfalls associated. What I knew was which one of them to ask about any given customer query. Point is that just working in an industry at a senior level doesn't make you a technical expert in that industry.
BTW, I would advise caution about Cambridge in Colour. There's some really good stuff there, but over the years there have been some rather shocking errors too - don't treat it as completely authoritative.
I've enjoyed it, given I know nothing about photography it makes a nice change to talk about something I know only almost nothing about instead
Nick, I wonder if he is thinking of hard discs where some space is required to defragment and hence filling the disc isn't a good idea. With a memory card, especially one used in a camera, one merely fills the card, copies the content and then formats it. No additional space needed, as far as I can see.
As to not seeing any need for a card reader, P Stoddart, if you are one of those photographers who fills a card, swaps it for a clean one and continues shooting, a card reader means that you can copy those images without having to put the card back in the camera. Also, I have a USB3 card reader but my camera isn't anything like that fast. So using the reader, even into a USB2 computer, makes for a much quicker transfer. Finally, there is no need to deplete the camera battery.
I suspect so, Geoff. It's one of the things that makes me suspect he's not, in fact, a flash memory expert.
I wouldn't claim to be an expert, far from it, but I do know that defraging flash memory isn't a good idea. As with many things, I have gathered enough knowledge to stop other people from doing the wrong thing but I don't necessarily know why.
De-fragmenting flash memory just reduces the lifespan without any benefit. Moving data around, without actually improving the read times. The benefits of de-fragmenting hard drives is clear.
1. When you request data from a spinning hard disk, the head reads data before and after the data you request, and so if you subsequently request the next bit of file, it's already in cache. Having your file in a contiguous on disk lump therefore has obvious benefits.
2. Reading from hard disks is not linear, it can be faster on the outside of the platter than the inside of the platter, so moving important files to the outside edge might result in quicker reads (for important files).
3. The head has to physically travel (seek time), and lots of seeking for a single 'read' can negatively impact performance. Of course, having to move from the inside to the outside every time can also slow things down and so having data on the outside edge doesn't always result in faster use (the read is faster, the total time might not be).
Much of this is mitigated by (and in fact, people spend a lot of time mitigating it) computers which cache filesystems. Linux for example, spends a lot of time filling unused memory with filesystem information so that reads more often come from there than the disk. This is achieved by reading more than necessary when the head is moved, and moving that into memory. It's a guessing game, with the cache trying to stay one step ahead of the applications. The more apps you have accessing more data the harder it is for the cache to be correct.
With flash memory, read speeds for any one piece of data are the same as any other piece of data (by design) regardless of the physical location within the chip so there's no benefit in moving that data round. Equally fragmentation isn't an issue because if a file uses more than one block or unit of flash, the number of reads doesn't change regardless of how fragmented the file is, for the data. For example, if a file is 10 blocks long, the controller has to request 10 bits of information. If they are contiguous on the filesystem (a logical structure) the controller still has to request 10 items. If they are not contiguous, the controller still has to request 10 items, and none of the read speeds vary. It's impossible to tell if the file was fragmented or not.
There are small overheads if a file has a lot of fragments in looking the data up in the FAT table, but those overheads are tiny. They certainly don't outweigh the fact that 'moving' data around increases the write/erase cycles the flash memory experiences, which is the one thing that causes it to wear out.
And again, the physical location of the data within the flash memory is controlled and abstracted by the wear levelling algorithm to prevent a single cell being worn out early through repeated erase/writes. So you might well think you're asking the card to place your data into contiguous parts of the filesystem, but in reality the wear levelling algorithm is placing them somewhere entirely different and then abstracting that out.
But what do I know
Let me know how you're handling de-fragmenting your smart phone storage, or tablet, or how often you don't delete things from your smart phone in case it corrupts the filesystem table.
Or your smart TV.
Or your car radio, satnav, hand-held GPS, jogging computer, multi-function digital watch, etc., etc. Most of the digital things we interact with on a daily basis use some form of NAND storage, and they tend to have a filesystem, and they use embedded software to drive it.
CompactFlash, USB, SD, SSD, they're just containers, the internal technology is the same, the controller is the difference.
If flash memory is the same as when I worked with the technology.
There is fundamental difference at low level between disk based storage and flash memory.
With flash memory you cannot write a individual byte or bit of data you have to erase and block then re-rewrite the change.
With disk based you can just write a byte or bit anytime.
Also flash chip have a write life span ie they fail after writing over and over again.
So forcing the erase and re-write of a flash memory block wears out the flash memory.
So I suspect every time the FAT has to have something delete from its settings ie free data blocks being noted it will require the whole flash memory block to be erased and re-written that holds the FAT not the data blocks of course.
Unlike disk which will simply change the FAT.
As for formatting in camera, I come from the logical position that the team of software engineers have select FAT32 as the best way of using the memory card. Possibly because they know FAT32 is support by even Windows 98 or it works for all the firmware in the camera LOL
So if you go ahead and force the camera to use exFAT and it screws up you only have yourself to blame.
The manuals I have seen on camera operation always say put memory in slot and FORMAT IN CAMERA.
Surely if the firmware development team of the device agreed that exFAT is the better choice they would have the native format be exFAT?
Except the wear levelling algorithms mean there's no direct link between the logical structures and the physical structures - that's the bit everyone seems to miss. The FAT object might say 'I'm in sector 10' but sector 10 isn't a physical location, it's a virtual location to protect from the exact issue you're describing. There's an entire other level of abstraction going on.
So on the one hand, the camera manufacturers know best. And yet, despite giving us the ability to delete files, we're told it's dangerous? You can't have it both ways. If they know what they're doing, and they allow us to delete files, then the risk must be low?
I only ever have myself to blame, I don't blame others for my choices.
They also list the file system formats the camera can handle. Manuals are written to give a baseline behaviour, they're not all encompassing.
Or maybe they just want to have a 'one size fits nearly all' but then why do they support other filesystem types at all.
There's almost no relation between FAT at the top and the actual hardware at the bottom, so filesystem behaviours don't translate to hardware behaviours.
Anyway, caveat emptor.
Someone wrote a blog. I disagree with it. Everyone else is welome to form their own view.
I just read the specs for a typical flash chip used in a SD memory card. That is the IC itself.
They still work on block erase system. In fact in the specs it does warn over time more bad block can occur.
The devices have got smarter since my experience with them. If for example it does erase a block can then copy data back it can detect a block failure and use a valid block.
But the erase system is the same. A whole internal block has to be wiped to change 1 bit of data. Unlike disk where one single bit can have it state changed by the head.without touching any other bit.
Although the chip in question has a 100,000 program/erase cycle and 10 year data retention.
So deleting images and shooting then deleting images adds to the wear and tear to the chip as the memory fragments.
Bear in mind the flash chip blocks are small 128K bytes.
So memory cards might seem like they behave as hard disks or RAM. But at the low level they don't. Changing a byte requires a song and dance of backing up the block, erasing the block then putting the block back with the byte change in place.
Tony, thanks for the information. Regardless of what the linked post says I will continue to format my cars in the cameras, fill them up, download the content to my computer and reformat in the cameras as I have been doing for the last 10 years. As I suspect will everyone else.
He He He....
I know I have related this story on the forum before but it may have been before Andrew's time here.
A few years ago I was having a debate via the "Opinion" page of one of Scotland's national newspapers about the optimum population size for Scotland. The other guy, whom I had known semi-personally for years, was being particularly ignorant yet amazingly arrogant and I was in a delinquent mood.
So I "invented" the "Normark" theory which used the characteristics of Norway and Denmark to explain how small semi-industrial/semi-rural countries could best develop a vibrant and sustainable economy. I then used the "theory" to "prove" that the optimum population for Scotland would be a decrease to 3.8 million.
Now, in fact, the theory was not total rubbish. It was fairly soundly based as a theory but it had no academic credence and would have required years of research and peer review to refine and establish.
But I published it in Wikipedia as if it was an established and well-regarded economic theory, with all the (fictional) references and appraisals.
I then effectively demolished my "friend's" arguments by referring to the theory in the newspaper and referring him to Wikipedia.
Several months later, a similar debate arose in the correspondence columns of Scotland's other national newspaper. I was more than slightly embarrassed when one of the correspondents in that debate referred to the Normark Theory to support his point of view.
I quickly removed the entry from Wikipedia, realising that I should have done it as soon as it had served my purpose.
But, of course, the moral of the story is that you cannot believe an effing thing you find on the internet.
GeoffR that is exactly what I do, I said earlier I once in a while might delete on camera.
The position of the ex memory card person was the FAT being scrambled. My position is deleting just puts more wear on the memory card because once again the camera as to re-write the FAT.
Separate names with a comma.