Monday, June 6, 2011

Reviewer has problem, needs help

7:40 PM / Blu-ray / Comments16 Comments

Blu-ray

I'm currently working on a review of Universum Film's 80th anniversary German BD release of Fritz Lang's M, kindly supplied to me by Torsten Kaiser, senior producer of TLE Films and the guiding hand behind this new release. Although it should be finished before too long, I must confess that it's giving me more trouble than these sort of things usually do and, while a lengthy conversation with Torsten cleared up the various questions I had about the film's restoration, the one I'm currently pondering is one that only I can really answer.

For a while now, I've been toying with the idea of abandoning the 1-10 numerical scoring I use to rate the image quality of BDs. I've used this system since I started writing reviews in 2003 for the site formerly known as DVD Times (now The Digital Fix) and, while it can be a useful way of giving people a ballpark idea of how a disc looks, I've always found it to be a rather inelegant system. Titles like M demonstrate this admirably: no-one would deny that, even with all the work TLE Films have put into the restoration, M's presentation has "issues" - issues derived from the age, condition and availability of materials and, in cases, issues derived from attempting to fix these problems. Place M alongside some of the discs to which I've handed out 9 and 10 ratings and it understandably falls short, but at the same time we have to bear in mind how good the film could reasonably be expected to look. M was released in 1931, was treated dreadfully by both censors and distributors, and the surviving elements suffer from varying degrees of damage, so it seems altogether unreasonable to compare it to a modern DI-sourced title like THE INTERNATIONAL or UNSTOPPABLE... or even an carefully preserved Hollywood classic like THE SOUND OF MUSIC. And yet that's precisely what having a unified scoring system does: it assumes a level playing field and leads to some tough choices having to be made when the condition of the elements or artistic intent prevent a film from looking like what the AVS Forum's exasperating Blu-ray picture quality tier list would call a Tier Zero title.

10/10

10/10

Another title that recently caused me problems, albeit on a lesser scale, was Darren Aronofsky's BLACK SWAN. Aronofsky chose to shoot the film primarily on 16mm film (with the subway scenes shot with consumer-level Canon digital cameras and processed to look like film). The result is gritty, grainy and, compared to something like ALIENS or the re-release of GLADIATOR, understandably lacking in detail. And yet I doubt that there's anything wrong with the way the film is presented on the disc. At the moment, I consider 20th Century Fox (who released BLACK SWAN) and Sony Pictures to be by far the best of the major studios when it comes to BD releases (isolated blips like PATTON and the re-release of PREDATOR notwithstanding). To be blunt, I doubt they fucked with it. I don't think anything could have been done to make it look any better, short of sending Aronofsky off to reshoot the whole thing on 35mm. I ended up giving the image quality a 9 - a decision that still doesn't rest well with me, and which has been queried by a least one of this site's visitors. Why not a 10? Well, because it has "issues". Intentional issues, but issues nonetheless.

0/10

0/10

Under the current system, I doubt M would score in the top "tier"... and yet that just doesn't seem fair. The question is how to proceed from here. Should I in fact have TWO scoring systems - one a completely objective score that ignores historical context, artistic intent etc. and simply rates a disc based on its value as (ugh) "demo material", and another that tries to take these factors into account? Or should I drop the scoring system completely and just let the text speak for itself? I must admit that I don't particularly want to do that - when you've reviewed a large number of discs, you want some way of telling at a glance how a particular title looks without having to read paragraphs of text.

???/10

???/10

So I'm going to open this up to the floor. Does anyone have any thoughts on this issue... or better yet, any solutions?

 
16 Comments

1. Matthew McKinnon said:

Leonard Norwitz over at DVDBeaver had a 2-part scoring scale (see http://www.dvdbeaver.com/film2/DVDReviews34/bad_santa_blu-ray.htm), wherein there's a number to represent how faithfully the image represents the 'theatrical experience' - or rather, perhaps more usefully, how faithful it is to the original presentation; and another to represent how good it is in absolute terms [like your Gladiator and EFNY scores].

You could do that, and perhaps add a third scoring for your personal feeling about the image? I would prefer that to a single score that has to take everything into account, and loses some specifics in translation.

Hope that helps.

(Posted on Monday, June 6, 2011 at 7:56 PM)

2. Author Profile Page Michael said:

It's an interesting system... but then I suppose the questions does become how I, or Leonard Norwitz, or anyone, can actually determine with any certainty what "the theatrical experience" is if we haven't seen the film theatrically... or indeed whether or not the theatrical experience was itself compromised?

Still, many thanks for your suggestion. I'll definitely bear it in mind.

(Posted on Monday, June 6, 2011 at 8:02 PM)

3. Jules said:

Michael,

I think when there's doubt about the condition of the materials, then I'm not sure a numerical score helps in the circumstances, and a more qualified review listing the pros and cons would seem fairer (a la Gary Tooze on Beaver_ - as you've raised, what more could have been done?

But then in turn that raises the question - how can one find out what the conditions of the materials were? We judge labels' ouput on the material that they have available and they're creative decisions stemming from that. They may also be limited by time constraints, budgetary concerns, demands from the licensors etc.

Personally I'd scrap numeracy altogether - it just does not allow an opportunity for fairness.

I think your problem with M is that you're viewing Torsten's release in isolation, when you really need to see how it stacks up against Criterion's and MoC, to see how much better the version is. I think Robert Harris on Home Theatre forum has compared this against the Criterion and commented that it has better stability; you could act as a counterpoint by buying or requesting a screener from MoC and then compare to see the differences. I'm not sure whether you're region locked but you should ideally check the Criterion as well.

(Posted on Monday, June 6, 2011 at 8:37 PM)

4. antnield said:

My own take has also been that the rating applies to transfer. I've always hated it when reviewers mark down PQ because they dislike a film's cinematography. Some films are supposed to look dull and lifeless; conversely I hate the look of Tony Scott's recent efforts, say, or the Transformers movies - but would never use that as a means to denigrate the transfer *if* they were faithfully rendered.

Likewise older films which are in a less than perfect state. For example, the likes of Italian titles made during the war years (Visconti's Ossessione, say, or Rossellini's Rome Open City) don't have great surviving elements. So it's a case of how well those elements have been transferred. Similarly the British documentaries the BFI have been putting out as extras or as part of themed volumes or their recent L'Age d'or disc. The bottom line is always whether the best obtainable materials have been used and how well they are transferred.

The other aspect worth considering is that I'm sure the vast majority of visitors to Land of Whimsy come to *read* about the transfer, not simply to look at a rating out of 10. I'm also sure that some know how strict you are with approaching Blu-rays and DVDs and may therefore calibrate the rating to their own stance; ie. if they're not as stringent then they may add a mark or two. On top of that, you provide enough information and enough screengrabs to further this lack of need for ratings (or at least to make them considerably less significant). This info and these images should be enough to back up a high rating for M, if of course it deserves one.

(Posted on Monday, June 6, 2011 at 9:12 PM)

5. Argento Fan said:

My two cents:

1. As ANTNIELD suggested, I get the most out of your full-blown reviews, the ones that walk a potential viewer through the strength and weaknesses of a given release. I get even more out of these reviews--often find them invaluable--when they include image comparisons across different releases of the same film (I've pointed more than one person to your image comparisons for the Mondo Vision Zulawski releases, e.g.).

2. In addition, the inclusion of a two-part scoring system, one that includes both an "absolute" value (something akin to the scoring system you currently use in your HD Image Quality Rankings, i.e., the best- looking images regardless) and a "relative" value that takes into consideration any extenuating factors (age of film, availability of quality materials, etc. etc.). I think careful readers/viewers will be able to make discerning decisions based on these two levels of grading/review, and will certainly understand, in general, that a film from 1931 is apt to look qualitatively different from a film made in 2011.

(Posted on Monday, June 6, 2011 at 9:35 PM)

6. Kentai said:

Mark me down as one who'd be okay with there being two ratings - but also favors your always extensive write-ups over a number every time.

I get why review sites tend to rank everything with a number: They get kick-backs from affiliate sales and free screeners from studios, and being less than positive or transparent could bork up both ends of the system. But when you cover mainstream fare alongside genre obscurities and rough 'round the edges classics, it gets pretty difficult to use the same sliding scale. Should The Evil Dead get the same ranking as The Godfather? The level of restoration is surprisingly similar, but the latter will always look - objectively speaking, anyway - a hell of a lot better than the former.

Perhaps the two best categories would be "Materials" and "Transfer"?

The former could go over the ups' and downs' of the source materials - 16mm vs 35mm, DI versus IP, what have you. I tend to think anything that's sourced from anything but the negative is no longer "10" material. If the negative no longer exists and we're looking at a 2nd gen print with a few scenes sourced from dupe elements - hey, that's fine. But that doesn't mean it should instantly get a top score just because you've rescued the one surviving print left and made it as less-ugly as humanly possible.

The latter could then be solely how said materials were handled - you could talk about compression issues, DVNR, signal noise or whatever else warrants a mention. Something like digitally re-created titles would probably go back to the former category, but scratch repair artifacts - though perhaos present because of the source - would be the result of the transfer work done by whatever company restored the print. I guess it's unfair to lump compression with restoration, but both are strictly technical means, and I can't honestly say that a video score needs a third ranking just to talk about wither or not there's any obvious artifacting.

With a system like that you could rate (just as an example) BLACK SWAN as a 8-10. "8" for the naturally grungy 16mm photography, "10" for the perfect handling thereof. Maybe the UK release of SUSPIRIA could get a 9-5, the former for the gorgeous quality of the print used, and the latter for the completely bombed out contrast. Perhaps TAXI DRIVER should be a 9-10, with one point deducted by default for the MPAA shenanigans that have forever made the climax look like chewed up brownish arsehole?

Of course, this two-part system does leave some interpretation open. You can't be 100% sure if something's a DI from the OCN or a 2k scan of several different vaulted IP without someone trustworthy telling you,and it's entirely possible that a gorgeous print was used and utterly destroyed via digital processing ("Friedkinized" or what have you). With more transparency between the labels and consumers this would be the absolute best way to rate image quality, but it involves some guess work that could come back and bite you on the rump if you're mistaken.

But if you want to abandon numbers all together, hey, wouldn't bother me. I'm hear to read your candid, detailed opinions because I think they're valuable, not average the final scores out on a spreadsheet or whatever. :)

(Posted on Tuesday, June 7, 2011 at 1:49 AM)

7. Vincent Pereira said:

Maybe do away with assigning a numerical score entirely and instead summing it up as "Recommended" or "Not recommended" and thus encouraging folks to actually read your full reviews and thus understand exactly where you're coming from?

Vincent

(Posted on Tuesday, June 7, 2011 at 4:50 AM)

8. ChuckZ said:

Stick with the absolute scale of 1-10. A film like M will never look as good as The International. That's okay with me.

Just describe what issues you see with the picture or explain how the restoration efforts, while admirable, can never fully measure up to a modern DI-sourced film.

You're over thinking this way too much.

(Posted on Tuesday, June 7, 2011 at 4:53 AM)

9. Brian said:

How about keeping the numeric system but basing it more on the quality of the encode and on reasonable expectations given when, how, and on what the film was shot. In short, rating more on accuracy rather than qualites for demoing a home theater sytem. For example, I found that Warner's KING KONG was a very nice presentation, though obviously it doesn't look like GLADIATOR.

(Posted on Tuesday, June 7, 2011 at 5:26 AM)

10. Brian said:

Just adding to the last comment...

Accuracy in itself is sometimes hard to determine as right or wrong as has already been mentioned, but there are always things that will stick out like a sore thumb, like what is seen in your capture from ESCAPE FROM NEW YORK.

(Posted on Tuesday, June 7, 2011 at 5:34 AM)

11. Matthew McKinnon said:

@Chuckz,
You're not thinking about this enough!
Many new titles sourced from a DI don't look as good as older titles that have been properly restored from film elements (partly, I feel, because DI is a conveyor-belt process and can result in transfers that are adequate, and 'contemporary', but not as good as they could be).
Making things as simple as not looking 'as good as' just ignores all the mitigating factors we're trying to consider here.

I'm beginning to think just ditch the numerical system and explain. What's a numerical score 'for', anyway? Skim-reading? Who needs that?

(Posted on Tuesday, June 7, 2011 at 7:41 AM)

12. Kram Sacul said:

I say stick with the system you already have (maybe use a 0 to 5 scale) and just go into detail when the need is there.

(Posted on Tuesday, June 7, 2011 at 7:57 AM)

13. Bleddyn Williams said:

My vote is in line with Vincent. I don't think people who use this site are into instant gratification - a quick score and off they go. One of the strengths of this site is your considered reviews - which I think visitors to the site are happy to read. A summary at the end that doesn't have to conform to a numeric scale would work just dandy.

(Posted on Tuesday, June 7, 2011 at 12:08 PM)

14. FoxyMulder said:

For my own site i use a transfer quality system, i try and look into how a film was made and the camera system used before i reach my conclusions, i'm not a fan of rating a film highly just because its eye candy and rating another poorly just because it's super grainy or filmed with the "ugly stick" look.

Obviously such a system requires guesswork at times but its preferable to the "Tier system" found at AVS where a film that has had a superb transfer can still find itself ranked lowly, to me that thread is just about eye candy, i still cannot believe how high they ranked Scary Movie.

Personally i have been thinking of just ditching a rating and going with something like Recommended or Not Recommended, you could list the good points and bad and just say not recommended or highly recommended or recommended with reservations at the end of your reviews.

(Posted on Tuesday, June 7, 2011 at 1:21 PM)

15. Thunderbolt said:

I've never had a problem with your 10 scale rating system. Its relative to the material, how good this movie could possibly look. frankly said, I like that better that the tier system which only rates overall quality. Its obvious to me that a movie of 50 years of age in almost all cases is not going to look as good than a modern DI movie (which studios didnt screw up), I dont need a tier system telling me that.

(Posted on Tuesday, June 7, 2011 at 5:48 PM)

16. LGans316 said:

PQ with respect to accurate reproduction / faithful presentation of source elements (need to rate down if the master is in bad shape) (1-100)

PQ in comparison to Reference / demo quality Blu-rays (1~100)

(Posted on Saturday, June 11, 2011 at 7:23 PM)

 
To combat spam, commenting is automatically disabled on entries older than 30 days.

Did a comment you tried to post accidentally get eaten by the spam filter? It happens from time to time. I get upwards of 200 spam comments every day and unfortunately don't have the time to weed through all of them in case something genuine ended up there by mistake. If one of your posts gets incorrectly flagged as spam, email me at m.r.mackenzie[at]gmail[dot]com and I'll do my best to retrieve it.