Current Post

Proposal for a Standardized Wine Rating System

By Tim on Jun 19, 2007 in Articles

There has been quite a bit of discussion in the wine blogosphere about wine ratings in response to Blake Gray’s piece in the San Francisco Chronicle last week. I’ve blogged quite a bit about this in the past, so I’m not opening up the question of continuing with the 100-point system or not. What I do think is interesting is the idea of some sort of standardization among those of us who blog about wine.

Is it possible for wine bloggers to choose a single rating system?

Given the passion around the 100-point system, I am not going to suggest this as the wine blogger standard. Neither will I propose either the Vinography 10 point or UC Davis 20-point systems. Because, as Ryan said a while back, shouldn’t the web (and wine bloggers) be different?

What makes the most sense to me is the 5-star systems adopted by most of the Wine 2.0 tasting notes sites and fully supported in the hReview standard.

Hear me out here… first, the scale is easy to understand and implement. It can also be used by readers to rate the same wines and the tasting notes sites can more easily extrapolate a composite score. I also think it is imprecise enough for more participants which is the problem with the 100-point system; it’s exclusiveness to just us uber wine geeks.

So I’m going to propose that wine bloggers rally around the 5-star system. I think the addition of 1/2 points will allow plenty of granularity and overlay all the other systems well enough for widespread adoption.

So who’s with me?

Over the next few days I will be posting some reviews on both Snooth and Winelog that will be reposted automatically here. They are the first two sites to implement an easy way for me to post once and then pull into my blog with no effort on my part. I’m hoping other sites follow suit, as well.

And, for the time-being, I will also post my 100-point score for those who like that system along with the new 5-star scale which at some point in the future will be the only scale I will use to rate wines.

Trackback URL

16 Comment(s)

el jefe | Jun 19, 2007 | Reply

I’m sitting here preparing a comment and suddenly out of the corner of my eye there are rubber chickens on your flickr thing….! anyway…

I started writing this whole comment on your five star thing, and then I decided it needed to be a blog post over on El Bloggo Torcido. So you’ll have to read it there.

I could have just made this a trackback, but I really do like the flickr chickens…
el jefe | Jun 19, 2007 | Reply

I hit submit before I remembered to type that the core idea – that of posting one place and having content repost automagically elsewhere using a standardized data format – is pretty huge! Should be fun to see it work!
Ryan | Jun 20, 2007 | Reply

I’m sold! Then again that is precisely why I left the 100pt scale. We need it to be as scale that communicates a few things:
1)Is the wine good
2)is the wine special
3)is the wine worth buying to asses for ourselves

this is why I use it. I love wine so much each bottle comes down to whether or not the wine is interesting enough for me to take the time to try it.

BTW hook a guy up with the auto posting from TN services…is this available? Is it easy to set up?
Philip James | Jun 20, 2007 | Reply

Thanks Tim – I dont have much to add here. Glad it got got done so quickly…
Tim | Jun 20, 2007 | Reply

el jefe: those rubber chicken pictures are pretty distracting; more where that came from next week when I’m in Spain (hope they allow mine through customs). Also, your points over at El Bloggo Torcido are well taken… I’ve posted to your comments.

Ryan: how did I know you would be all over this one? the recipe is pretty simple since we are both using wordpress. just install the FeedWordPress plug-in (http://projects.radgeek.com/feedwordpress) and then subscribe to your RSS feed in Snooth and/or Winelog. then a cron job will aggregate your reviews on those sites to your blog. right now i’m marking those as “private” and inspecting before posting, but when i’m in your neck of the woods next week, i’ll have it post directly here.

Thanks again for your work with Snooth, Philip!
Dr. Debs | Jun 20, 2007 | Reply

OK, I have no reason not to support this initiative, provided that someone tells me where to beg, borrow, or steal little star icons.

BUT. (You knew there was going to be a but, yes?)

I think el Jefe is correct that the 1/2 star thing is wacky and should, I feel, be avoided by individual bloggers. Get off the fence. Is it a 4 or a 5? Make a decision. Then let the averages assign half stars. If we use 1/2 stars, we are essentially using Alder’s 10 point system, with icons. So let’s use the stars. Not half stars, quarter stars, moons, solar systems, or plus signs. Just stars.

If we agree not to use 1/2 stars in individual reviews, this could be quite powerful I think because it is simple and straightforward.

BUT. el Jefe is right again (2 in one post!) that stars without context can be pretty meaningless. My stars would always be given based on varietal character and QPR. I think I would include my QPR ratings, and add a star system if that’s the way the blogosphere goes. But I won’t be able to give up my QPR system in total.Alternatively, who wants to develop the grading rubric (teacher speak) that explains what a 5-star wine is–and that can’t be AWESOME, folks, it has to have some greater meaning attached to it.
Nate | Jun 20, 2007 | Reply

First off – I think the idea of a “star rating system” is right on – people are used to seeing and using this elsewhere (Netflix, etc.) and is completely non-intimidating. I think we should allow the 1/2 stars though, and here’s why: going back to my Netflix example (and I think Netflix is the perfect example, btw, as they were really the first to set up the refreshless 5-star widget on their website back in 1999 that is now widely used across the Internet) – Netflix used a straight 5-star rating system, while Blockbuster started up with the half star system.

Netflix’s ratings were as follows: 5 stars: Loved It; 4 stars: Really Liked It; 3 stars: Like It; 2 stars: Didn’t Like It; 1 star: Hated It.

Blockbuster’s scores go like this: 5 stars: I Absolutely Loved It; 4.5 stars: I Loved It; 4 stars: I Really Liked It; 3.5 stars: I Liked It; 3 stars: It was OK; 2.5 stars: It was so-so; 2 stars: I Didn’t Like It; 1.5 stars: I Really Didn’t Like It; 1 star: I Hated It; 0.5 star: I Really Hated It.

On this HackingNetflix.com discussion, they debate whether Netflix should go to half stars, and most people thought that it should. The gist of the discussion is that there are very few truly 5 star movies by Netflix definition, but they were better than 4s – so they ended up getting a 4. Likewise, a lot of movies were better than a 3, but not quite a 4 – these movies also ended up with 4s. Folks felt this lead to an inaccurate grouping around 4s, whereas a similar thing happens at the bottom of the group.

Also, someone pointed out that in the 5 star system, you end up skewing the results favorably, since there are 3 “positive” answers and only 2 “negative” ones without a neutral “I Don’t Know / Care” answer – poor statistical design in their opinion.

Other folks commented on not liking stars at all, since visually discerning “fractional stars” can be problematic (is that 3 and 2/5ths stars or 3 and 3/5ths stars?) – this contignet just wants a straight numerical system and do away with the whole symbol thing altogether.

So there’s the basic arguments as I see it. My personal opinion is that people like symbols more than numbers. I’d rather see that a movie had 4 stars than a 4 out of 5 (or 8 out of 10). It’s more organic somehow. of course, with wine as our subject we have the perfect symbol for our use – wine glasses! (or bottles, decatners…) My vote is to go to a 5 glass system, allowing for half-glasses. Yes, this is basically Alder’s ten-point system, but more visual (symbols rather than numbers) and more compact as you wouldn’t want to have ten whole wine glasses in a row.

But Dr. Debs brings up a good point: where wines differ dramatically from movies is their cost. A movie pretty much always costs what it costs ($4 to rent, $10 for a theater, whatever). And while some movies are longer than others, it doesn’t make any sense to rate a movie based on its value per minute or something like that. However, I feel we DO need to incorporate that type of rating for wine (like Dr. Debs QPR ratings), because there is such a huge disparity in wine price. Otherwise, some great $12 everyday bottle you just found won’t ever rank above a 3 or 3.5 on an absolute ranking system, but in comparison to other $12 wines it should be a 4.5-5! So I think that incorporating VALUE, not just quality, into the rating system is the most difficult aspect of wine rating, and short of having two different ratings, I’m not sure how we can do it…
Michelle | Jun 20, 2007 | Reply

Well, I’ll probably be sticking with my 5 different Happy Faces on my own scale. It’s because wine is emotional for me, and I feel that my Happy Faces reflect that, as well as my words. However, the Happy Face scale is not really that far off from the star system.
Of course, among my local readers, it’s sort of a trademark at this point and who am I to give that up?
I worry about trying to standardize anything in the blogosphere. There are always so many new and unique voices joining the community that standardization might take that away, or others will completely ignore it.
But, I’m all for a visual scale because I think that pictures express more than numbers, and that words express the most of all. So hooray for your 5 star scale! Just don’t be sad if I hang on to my happy faces. (But if it works out that folks adopt the stars, I’ll probably do a post translating the stars to happy faces.)
Cheers!
Andrew | Jun 20, 2007 | Reply

As Dr Debs says – I can see no reason why not to use it. I use the 100 point at the moment (or a bastardised version of). But why use halves? If you use halves why not just use a ten point system instead?

Basically though you want the bloggers to support something different from the accepted norm? Then why use points/stars at all – cant we come up with something totally different? no, I dont know what it would be either

My ‘100 points’ are based on four qualities (seen on a roll over on the score on spittoon) based on drinkability, interest, value and enjoyment – rather than nose, colour, balance etc. Easy to convert to a 20, a 10 or a 5 point system.

I’m all for setting a new ’standard’ though, so if a 1-5 star it has to be then that’s fine by me.
Tim | Jun 20, 2007 | Reply

Great comments Deb, Nate, Michelle and Andrew!

I still think we need the granularity of half-points (or glasses/bottles/barrels/wine thief’s/rubber chickens, etc.) given a couple of reviews I recently reposted here that have the same point value but should not be rated as such.

I’m pleased the comments are positive to my proposal… who else is on board?
Ryan | Jun 21, 2007 | Reply

1/2’s RULE and I’ll tell you why! I forget the site but they had a way to rate stuff 1-5 stars, hearts, smiley faces(this part as a “grape user doesn’t bother me!) But after so many ratings they started giving a community average which they didn’t round up or down but rather showed the average with decimels. So I couldn’t rate the wines by the 1/2 point but the output of the site had half point in it.

Another point! There is no .5stars and no 5.5 stars, and rarely if ever a 1.5 stars…the only places I actually need to use 1/2 point is at the 2.5(for me below 3 is not good wine or not interesting), this allows me to show that a wine is probably good enough for everyday quaffing, though don’t seek it out, while allowing for a 2 rating where the wine is not flawed but rather not worth the effort.

3.5 and 4.5 are indispensable, as I only want to have a handful of 5pt wines, reserving them for wines that I think are varietally/stylistically the most interesting. Thus I need the gradation. As I see it though and this may make it a bit confusing I have a 8pt scale since the only ratings I give are 1(flawed) 2 and 2.5 (un interesting) 3, 3.5, 4, 4.5(the most used scores) and the mighty 5(which I give based on how the wine rates in a peer group, not the BS that other wine scales do, whereby rating bordeaux against vinho verde against late harvest zinfandel.This is another issue that may or may not need to be discussed here, but all wine sytles deserve a benchmark as to what is prefect for themselves. more later)

that was a mouthful, here are my full rating explanation if you want to see it.
http://catavino.net/ratings/
Redneck | Jun 21, 2007 | Reply

For what its worth I use a 10-point cork system on my site. I’m just a redneck and don’t have any formal reviewing or rating experience. I utilize 1/2 points even though my images only show full corks. I guess that is equivalent to a 20-point scale. Great thread and excellent comments.

I think I remember a time when Netflix had the 1/2 star option and I can’t remember for sure when it went away but I remember being very disappointed. If I were forced to pick one option I would go with 5 points with 1/2 options.
Tim | Jun 21, 2007 | Reply

Ryan and Brian,

Thanks for your support on half-points. I think they are needed to delineate the differences in reviews, but I’m preaching to the choir. Work on Dr. Debs who seems to be wavering
winehiker | Jun 21, 2007 | Reply

Alas, I appear to be the lone holdout. Aw, heck.

I started to comment on this thread about noon today but got interrupted a bit and had to close out before I could finish my thoughts. I later decided to expound by posting my thoughts on my own blog, which you can click to above. (Tim, I see you’ve already visited. Man, you’re quick.)

~Russ
Dr. Debs | Jun 22, 2007 | Reply

What this is evolving into is the 100 point system masquerading under stars. I thought it was going to be different?

I’m sticking to QPR. Before I didn’t see a reason not to support it. Now I do!
Richard | Jun 22, 2007 | Reply

What exactly do we want our Ratings to accomplish? And are those purposes better off served in Tasting Notes rather than a rating?

I use a very simple Rating system, with only 3 categories.

1) Drink & Buy: A wine I recommend as worthy of buying.
2) Drink Not Buy: A wine that is drinkable but not something I would buy myself.
3) No Drink No Buy. A wine I would not recommend at all.

Everything else is said in the Tasting Notes. My goal is to direct people toward certain wines, regardless of price. So, an excellent $10 wine or an excellent $200 wine could both be in the first category. The tasting notes will mention the price, and also whether I consider it a good value.

One significant issue with having 100 point, 20 points, or even 5, is that people will only seek out the highest rated wines. Thus, they miss out on other good wines. Even at 5 points, you are still susceptible to people seeking only the 4&5 point wines. With my 3 point system, all the recommended wines come under the same point. Thus, there is no trophy searching. And maybe more people will drink wines that would only receive good, but not excellent points, in other systems.

6 Trackback(s)

From Snooth Blog » Feed me… | Jun 20, 2007
From It's a wine blog. It's a hiking blog. It's Winehiker Witiculture! | Jun 21, 2007
From Catavino - - Spanish Wine, Portuguese wine and more! » Blog Archive » We had five grapes before there were grapes! What's the Point? | Jun 22, 2007
From Unified Wine Blogger Wine Rating System | Jun 22, 2007
From Wine Ratings Revisted | Jul 25, 2007
From A Passionate Foodie » Blog Archive » Wine Rating Systems | May 23, 2008

Current Post

Proposal for a Standardized Wine Rating System

Recent Comments

Categories

Archives

Subscribe

Let's Get Social

Advocacy Groups

Friends of Winecast

My Other Sites

Wine 2.0

Wine Blogs

Wine Links

Wine Podcasts

Winery Blogs