Discussion in 'General Equipment Chat & Advice' started by huwevans, May 5, 2007.
I'm quite proud of that one, too - Huw had to put a bit of effort into it.
I still thinks that's the best one
I'm sure I can do better if I really put my mind to it. Can I be ar*ed, though? - That's the question.
While I agree with most of what you say, you still need some non varian parts in your tests, how are you going to give a figure about dynamic range if you don't have a rigorous and consistent setup for it?
I agree that shooting charts with resolution lines tells very little, mainly because it only shows how a camera/lens performs with extreme high contrast images, which extremely rarely occurs in real life.
On the other hand, if you test one camera taking pictures of a cityscape in a very dull day, and another using a colorful landscape on a brigh sunny day, you still get very little useful information.
I don't know where you got the idea that I like resolution test charts since I never mentioned them in my previous posts anyway, but maybe a setup like the image that Phil Askey uses for his studio tests is able to tell us something, and includig a Gretag Macbeth chart does not seem such a bad idea to me.
To me a good test has to be a combination of the two, rigorous testing in studio, and real life shooting, and both under different lighting conditions and different lenses.
To that end I find Phil's reviews quite lacking, since he does not seem to do a lot of testing with different lenses, and his outdoors images vary wildly from test to test.
I understand that an exhaustive test will probably require at least a couple of weeks of full time work, both in the lab, in the studio and outdoors; and maybe in today's competitive market magazines cannot afford this kind of expense.
But unless, they do, all reviews are going to be necessarily incomplete, and the only way to have something close to the full picture is to try and put together the results from several reviewers, and maybe ask some questions where the results vary wildly.
I can't see anything wrong with his approach, but some people seem to take this personally and start to use terms like "measurbators", unfortunately this is often how undesirable and avoidable flame wars start.
Have a nice Day
My final thoughts on the issue, and they address a number of posts in this thread - and elsewhere:
Actually, I've got quite a high regard for Phil (and Simon) and his tests - I consider them to be streets ahead of any other web-based tests. Obviously we'll have to see how that pans out under the new ownership - my question there is simply one of how long Phil will remain, and he clearly insists he's there for the long run.
But the simple fact remains that all that really matters is how a camera performs in use. I used to get quite excited by highly technical tests a few years ago, and I'm still interested to an extent, but really all I'm fussed about now is what it can do in practice. Angela has actually explained a fair bit of the AP methodology on the DPR forums, and Damien has previously discussed it here and I think in print - the thing is that they refer to the more formal test processes when it actually makes any difference, rather than wasting page space when it doesn't. I reckon that's a sensible use of space, TBH.
However, I agree completely that it makes sense to look at a variety of tests/reviews when considering a new camera. My favoured sources are AP and the French mag Chasseur des Images in the print world, and the DPR tests online - seems to make a good balance. I have also been known to trust the Steve's Digicams reviews. Other testers do get absolutely obsessed with the minutiae of test figures, and they're the ones I refer to as "measurbators" (I don't much like the term, but it simply seems to fit so well - especially for the amateur reviewers who really don't seem to understand what they're doing) - most of them are online and deeply flawed, but there's one who regularly appears in another weekly publication - I wouldn't call his tests flawed, so much as not particularly pertinent, as they ignore the actual purpose of a camera. The UK monthlies, on the other hand, do appear to me to be guilty of what Angela was accused of on the DPR forums - just taking a few pics in the car park. That's probably rather unfair, but given the limited amount of space they allocate to tests it's rather hard to avoid coming to that conclusion. The one real exception there is Outdoor Photography, which simply lets someone use a camera for a while and then comment on it - perhaps that's too far down the non-measurement route even for me!
Of course for some applications, the ultimate resolution of a camera and lens is important, but for the huge majority of us the only thing that matters is what the results look like. AP, being a generalist mag, uses cameras in the way that most of us would for tests - that means, for example, using lenses that can easily be mounted - how many people use lenses that require mount adaptors? I know I do sometimes (having a collection of Zeiss lenses), but I don't actually know all that many others, and I wouldn't expect AP to do that in a straight test - it's the sort of thing that might make an interesting follow-up article later, perhaps. It also means using a variety of RAW convertors and commenting on which does what, etc. etc. etc. - not using them from the perspective of an expert in that particular camera.
But ultimately if there's a difference between measured results and actual output, although the reason for the difference might be interesting, it's really the actual output that matters, isn't it?
I guess that as with the Leica DMR review, an awful lot of people have heard about the AP review and score and some have chosen to rant about it without reading the reviews or understanding the logic behind the ratings. That is a failure with the way people understand reviews: the score isn't actually the be-all and end-all. People should take time to read and understand reviews, and see which parts are pertinent to them. Sadly, what we've got in these two cases are extreme cases of fanboyism - violent reactions against valid criticism. Anyone who buys a camera - or doesn't - purely based on the test score is crackers, IMHO. For instance, looking at the SD14 test I wouldn't personally care about JPEG quality, because I virtually never shoot in JPEG. But it would be silly of me to say that shouldn't be part and parcel of the test, because other people do. The same applies to most aspects of most tests - you need to take note of the things that matter to you, and ignore the bits that don't. Foveon is a concept I'm very interested in, and I would love it to continue to develop. At the moment, it looks to me as though there's some distance to go before it can compete across the board with the best of the rest, but maybe that's just a firmware upgrade or two away. Whatever, it appears that at present, the SD14 is yet another flawed potential genius.
Good points, however I never tought of the SD14 as a best in class camera, or even a good all-round camera.
To be fair I don't think very highly of any DSLR since I don't like to go around with a brick and a huge lens on my neck, that's why I mainly use Rangefinders and compacts.
In the two years since I had my R-D1 I have almost never touched my 20D, and certainly the R-D1 is not a camera that will make pixel peepers happy, only 6Mpix, and more expensive than a 5D to boot, lots of vignetting with almost every lens, it will probably perform badly in any measurement you can figure out for it!
Still the pictures have a certain quality that I still have to see in any DSLR, maybe it's the Leica lenses and the fact that I only use primes, I don't know, but certainly lab measures were not the reason I bought it.
My most trusted Internet source is Sean Reid, the only drawback is that you have to pay to read his reviews, DPreview is another source I ofter refer to, but it has limited appeal since he doesn't usually test lenses.
The only photo magazines I subscribe to are AP, Black & White Photography, Lensworks and Private so you can see I consider AP up there with the best.
My interest in the foveon sensor is mainly due to the upcoming DP1, a pocketable compact with an APS sized sensor is enough to make my mouth drool, I have no problem in having to use RAW, and having to photoshop my pics, I just want to know if the technology is capable of delivering top quality pictures, and if yes under which conditions.
I have seen great pictures taken with the SD14 so under the right conditions it is definitely capable to perform, but no one seem able to explain which are those conditions.
I have no issue about AP giving it a low score, the camera really has too many quirks to be a good camera for the average amateur, and the fact that is unable to perform consistently relegates it to the status of "niche camera", a bit like my R-D1 that did not receive a good score from AP either; however which niche (besides the hardcore SIGMAphile) does it fit? What subjects is it good at, and what does the user need to to to get the results the camera is capable of at its best? This are the answers that I am looking for, and still those answers keep eluding me.
Yes, I think many of us are interested in the DP1.
I'm not surprised you get on with the Epson - I love the concept of a digital rangefinder. But I can't afford the M8, and frankly I would find the crop factor too limiting on the Epson - I like wideangles too much. So the DP1 would be a wonderful addition to my kit if it were capable of decent results, even if it took some work - as long as it was clear what had to be done, as you say.
As to the SD14, I'm just as unclear as you as to where it fits at the moment. It all rather reminds me of the full-frame Kodak DSLR fiasco - cameras capable of excellent quality at best, but which simply weren't generally good enough.
Oh well isee that the SD14 review in What Digital camera is even worse than the one done by AP. not going to help Sigmas sales at all, may be that we'll see a price crash to about £500 in the near future?.
Lets hope so then I can afford to buy one!
I still think Sigmas are very underrated.At least for a photographer that concentrates mainly in getting high image quality and from a regular user's point of view.
I started my digital photography with an S20pro, then a Minolta 7D and then a S3pro. I have been after a Sigma before I bought an S3pro and the image quality has in fact stayed in my mind ever since I got the S20pro some good years back.So much so, that I ended up tracing one up, bought it second hand with very little use. I am not disappointed at all and image quality is consistent in all situations and sometimes even surprises me.
Coupled with some EX glass, or some adapted Canon glass the image quality is superb.
Separate names with a comma.