How is the score/tier/import from imdb system working?

Wondering how Criticker works, or have a question that doesn't seem to fit under requests or bugs?
mathiasa
Posts: 4
5064 Ratings
Your TCI: na
Joined: Wed Feb 05, 2014 1:26 pm

How is the score/tier/import from imdb system working?

Post by mathiasa »

Is there somewehere a help page that explains how exactly stuff is working? Sometimes I when I import a 1/10 imdb movie it gets a score of 10, sometimes one of 30. Does it matter which movies I imported before? Does it matter which other movies I import along with it? I can't put it together.

PeaceAnarchy
Posts: 654
7005 Ratings
Your TCI: na
Joined: Tue Dec 12, 2006 1:47 am

Re: How is the score/tier/import from imdb system working?

Post by PeaceAnarchy »

From what I can tell, if you have no ratings it imports 1 to 10, 2 to 20, etc. But if you already have ratings on criticker it imports 1 to tier 1, 2 to tier 2, etc.

cagedwisdom
Posts: 827
2090 Ratings
Your TCI: na
Joined: Wed Jun 14, 2006 9:44 pm

Re: How is the score/tier/import from imdb system working?

Post by cagedwisdom »

PeaceAnarchy wrote:But if you already have ratings on criticker it imports 1 to tier 1, 2 to tier 2, etc.


This sounds pretty flawed, and entirely contrary to the way Criticker handles tiers. 1/10 or 3/10 should still translate to 10/100 or 30/100. I understand that this may cause problems for people who only use say 1-10 for their rankings on Criticker, but there should at least be an option to choose which method to distribute the rankings. For instance for people whose tier 1 is everything below 50/100, they'll have everything rated 2+/10 on IMDb be over 50/100 on Criticker, even if their tier distribution on IMDb is similar (~10% of films rated below 5/10). That doesn't make any sense.

The most sensible way to do it would probably be for Criticker to analyse and assign tiers to all your IMDb ratings, then port them over onto matching Criticker tiers. That would probably be much more complicated, though.

mattorama12
Posts: 887
3086 Ratings
Your TCI: na
Joined: Mon Feb 13, 2012 3:05 am

Re: How is the score/tier/import from imdb system working?

Post by mattorama12 »

Rufflesack wrote:The most sensible way to do it would probably be for Criticker to analyse and assign tiers to all your IMDb ratings, then port them over onto matching Criticker tiers. That would probably be much more complicated, though.


More complicated, yes, but it shouldn't be prohibitively so. This is really the only way that makes much sense, unless you're starting from scratch on Criticker (in which case adding a zero would make the most sense)

PeaceAnarchy
Posts: 654
7005 Ratings
Your TCI: na
Joined: Tue Dec 12, 2006 1:47 am

Re: How is the score/tier/import from imdb system working?

Post by PeaceAnarchy »

Yeah, when the import function was added I suggested that there be an option to specify how scores would translate.

It should be noted that since a collection is created with the imported rankings, a batch edit of the results isn't too much of a pain to do, but it would be nice to be able to specify it beforehand.

mattorama12
Posts: 887
3086 Ratings
Your TCI: na
Joined: Mon Feb 13, 2012 3:05 am

Re: How is the score/tier/import from imdb system working?

Post by mattorama12 »

PeaceAnarchy wrote:It should be noted that since a collection is created with the imported rankings, a batch edit of the results isn't too much of a pain to do, but it would be nice to be able to specify it beforehand.


That's really handy. I don't know these things because criticker is the first (and only) ratings site I've used, so I never needed the import function.

r.Din
Posts: 10
2885 Ratings
Your TCI: na
Joined: Fri Aug 25, 2006 8:40 am

Re: How is the score/tier/import from imdb system working?

Post by r.Din »

Yeah, I've just run into this. Imported 2000+ ratings from IMDb and find Criticker has skewed my entire rating distribution incorrectly across the ten tiers such that I have 300+ films ranked Tier 1 - Terrible, even though my IMDb ratings for these ranged from 50 (average) downwards...

The obvious way to import is the rating equates to the tier... as this is how most sensible people vote...

I'll have a look at this batch editing to see if I can clear things up a bit...

r.Din
Posts: 10
2885 Ratings
Your TCI: na
Joined: Fri Aug 25, 2006 8:40 am

Re: How is the score/tier/import from imdb system working?

Post by r.Din »

Batch editing such a large collection is a pain, and also like herding cats I guess because as I update a bunch of votes the site will readjust my tiers and I may have to go over things again.

I did update my quips tears to better match the tiers Criticker is using, but again, I imagine they will drift out of sync over time if Criticker makes changes to my tier boundaries.

I wonder if there is a more methodical way to achieve correct tiers? eg import the IMDb data in 10 chunks, once for each rating and then ensure the tier is correct after each import?

Should be easier than it is, that's for sure.

PeaceAnarchy
Posts: 654
7005 Ratings
Your TCI: na
Joined: Tue Dec 12, 2006 1:47 am

Re: How is the score/tier/import from imdb system working?

Post by PeaceAnarchy »

I think you have a misunderstanding of how tiers work. You'll never get scores to match up to tiers the way you want because tiers are simply percentiles. 10%(approx due to rounding) of your ratings will be tier 1, 10% in tier 2 etc. Just make sure the scores match up to what you want, and fix the quips colours to match what ratings mean to you. The tiers work themselves out on their own.

Your scores should have a fixed meaning to you, tiers will float slightly over time but you have 2000 ratings so they probably won't drift too far from here on out unless you drastically change your viewing patterns.

r.Din
Posts: 10
2885 Ratings
Your TCI: na
Joined: Fri Aug 25, 2006 8:40 am

Re: How is the score/tier/import from imdb system working?

Post by r.Din »

I think there's a fundamental flaw with Criticker's tier system...

As you say, it's a percentile thing. Here's the problem - Criticker assumes that my lowest X% are terrible films, eg Tier 1 - when this is simply not the case - Who goes out of their way to watch terrible films? I always try to watch decent stuff, however, I have seen a few stinkers and so there are some very low votes, but generally the votes start around 4/10 and go upwards. So what Criticker thinks is my Tier 1 is actually not, in reality it's a collection of my Tier 1,2,3,4; it's just the tail end of my viewing. And indeed the same problem at the top end - I've seen a few truly exceptional films (Tier 10) and a lot of great/good films (Tiers 8/9), but they all get lumped into Tier 10 due to the % slice size.

The problem is that when it generates a PSI it has clumped so many tiers together that the results are much less useful at the extremes... and the higher Tiers are precisely where I'm looking for the best advice.

It seems Criticker uses a substantially even slicing when it would be better off using a bell curve - or better yet allow the using to specifcy the boundaries. Voting on IMDb is 1-10 - the Tiers are implicit and obvious. When I import I should be able to maintain that Tier distribution.

I can mitigate this somewhat by altering my Quips, but the fundamental issue remains - Criticker's tier slicing seems flawed, which implies that the PSIs are also flawed?

Now I know Criticker will find TCIs for me with similar tastes and similarly skewed distributions, which might also mitigate the issue somewhat, but I'm not totally convinced...

Post Reply