What happens to tiers if a user only rates movies he likes?

Wondering how Criticker works, or have a question that doesn't seem to fit under requests or bugs?
CriticalCat
Posts: 7
330 Ratings
Your TCI: na
Joined: Sun Nov 19, 2017 12:53 am

What happens to tiers if a user only rates movies he likes?

Post by CriticalCat »

If I only rate my most favourite movies on this site and only give out scores like 80 and above, will it totally mess up the tier system? (lowest 10% of movies will still be placed in tier 1 if I understood correctly).

So to use the site to its full value, do I need to rate an equal amount of (in my opinion) good, average and bad movies, i order to get an even distribution and let the algorithms work at their maximum potential?

mpowell
Posts: 3851
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: What happens to tiers if a user only rates movies he likes?

Post by mpowell »

Yes, basically you've understood it right. Tiers are just percentile groups. Regardless of the score, the lowest 10% of your ratings will be in Tier 1.

Criticker does work best for those with a wide variety of rankings.

Bort P
Posts: 1
53 Ratings
Your TCI: na
Joined: Tue Dec 26, 2017 8:08 pm

Re: What happens to tiers if a user only rates movies he likes?

Post by Bort P »

This is an issue, because:

-I try to watch movies that I'll like.
-Criticker suggests to me movies that I'll like.

Thus it seems that the better Criticker works, the more skewed the system will be, because I'll have a ton of liked movies (70+) and few less than 40.

I just signed up today, and I'm observing that because I rate mostly liked movies, the tier system is way off. I'm not sure how this will ultimately impact my suggestions.

BadCosmonaut
Posts: 355
4401 Ratings
Your TCI: na
Joined: Mon Sep 16, 2013 7:08 am

Re: What happens to tiers if a user only rates movies he likes?

Post by BadCosmonaut »

Bort P wrote:This is an issue, because:

-I try to watch movies that I'll like.
-Criticker suggests to me movies that I'll like.

Thus it seems that the better Criticker works, the more skewed the system will be, because I'll have a ton of liked movies (70+) and few less than 40.

I just signed up today, and I'm observing that because I rate mostly liked movies, the tier system is way off. I'm not sure how this will ultimately impact my suggestions.


I don't see an issue with the system here. In your case, if the tiers are skewed it's because you are only rating movies you like. Just rate as many movies as you can that you've seen, including bad ones. There's no reason at all to rate movies you haven't seen.

Also, you've only rated about 50 movies. The system works better with more movies rated.

Midaso
Posts: 24
7453 Ratings
Your TCI: na
Joined: Wed Dec 18, 2013 9:01 am

Re: What happens to tiers if a user only rates movies he likes?

Post by Midaso »

What I found on this site (and I assume other people will be the same) is I've seen far more movies than I thought I had because of all the bad movies you see as a kid and forget fairly quickly

90sCoffee
Posts: 173
2242 Ratings
Your TCI: na
Joined: Sun Feb 17, 2013 5:15 pm

Re: What happens to tiers if a user only rates movies he likes?

Post by 90sCoffee »

I've mentioned this before, the solution for me is to change the tier percentages so that they aren't all equal to 10% of your films each. Tier 1 should be your bottom 4ish% of films, tier 2 should be around 8%, tiers 6 and 7 should be closer to 15ish%, tier 8 should be around 12%, tier 9 should be 7ish%, and tier 10 should be reserved for your top 5% favourite films. Those are rough numbers but you get the idea, they should represent the parabolic shape that most people have with their ratings. By assigning an equal 10% to ratings like it is now, it assumes that everyone has a flat shaped graph of their rating distribution which most don't.

livelove
Posts: 759
67 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: What happens to tiers if a user only rates movies he likes?

Post by livelove »

quote from related thread because the user makes a good point:

philamental wrote:If you decided to rank 100 movies, and it just so happens they are your 100 favourite movies of all time, and they vary in score of 90 to 100, then the movies you ranked the lowest will be your Tier 1 films, red with whatever negative sounding default quip goes along with it. The tier system basically aims to 'separate' the films you've rated and judges them against each other. If 90 out of 100 is the lowest score you have ranked on this site, then that movie will be deemed the worst movie you have ever seen ... because it doesn't know about any other movies you've seen.
In short, you need to rate bad movies too for criticker to understand what movies you see as being good.

livelove
Posts: 759
67 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: What happens to tiers if a user only rates movies he likes?

Post by livelove »

mpowell wrote:Re: What happens to tiers if a user only rates movies he likes?
[…]
Regardless of the score, the lowest 10% of your ratings will be in Tier 1.*
Criticker does work best for those with a wide variety of rankings.
* i.e. bottom 10% percentile

I don't know, but after all maybe it's necessary to take into consideration the user's vote distribution (the curve shape) to some extent, as suggested above by 90sCoffee?

90sCoffee wrote:the solution for me is to change the tier percentages so that they aren't all equal to 10% of your films each.

Tier 1 should be your bottom 4ish% of films,
tier 2 should be around 8%,
tiers 6 and 7 should be closer to 15ish%,
tier 8 should be around 12%,
tier 9 should be 7ish%, and
tier 10 should be reserved for your top 5% favourite films.

Those are rough numbers but you get the idea, they should represent the parabolic shape that most people have with their ratings. By assigning an equal 10% to ratings like it is now, it assumes that everyone has a flat shaped graph of their rating distribution which most don't.


I agree that it would be worthwhile to at least think about what role the graph shape plays and what implications it has for comparing 2 users with different graph shapes to each other (which is the whole point of criticker).

I don't think that assigning fixed (Bell curve induced) values to the tiers (as 90sCoffee suggests), because what happens then to to users with a flat shaped graph (or, generally speaking, anything other than a Bell curve) ?

Also, the Bell curve suggestion above would only mitigate 1 aspect of non-linear graphs, but it doesn't deal at all with the problem highlighted by the OP, namely users who "only rate [their] most favourite movies and only give out scores like 80 and above", regardless of whether these scores from 80-100 have a flat or Bell curved graph.

This type of situation seems in need of a different solution.

Post Reply