Page 1 of 1

What happens to tiers if a user only rates movies he likes?

Posted: Sun Nov 19, 2017 2:57 am
by CriticalCat
If I only rate my most favourite movies on this site and only give out scores like 80 and above, will it totally mess up the tier system? (lowest 10% of movies will still be placed in tier 1 if I understood correctly).

So to use the site to its full value, do I need to rate an equal amount of (in my opinion) good, average and bad movies, i order to get an even distribution and let the algorithms work at their maximum potential?

Re: What happens to tiers if a user only rates movies he likes?

Posted: Tue Nov 21, 2017 3:56 pm
by mpowell
Yes, basically you've understood it right. Tiers are just percentile groups. Regardless of the score, the lowest 10% of your ratings will be in Tier 1.

Criticker does work best for those with a wide variety of rankings.

Re: What happens to tiers if a user only rates movies he likes?

Posted: Tue Dec 26, 2017 9:04 pm
by Bort P
This is an issue, because:

-I try to watch movies that I'll like.
-Criticker suggests to me movies that I'll like.

Thus it seems that the better Criticker works, the more skewed the system will be, because I'll have a ton of liked movies (70+) and few less than 40.

I just signed up today, and I'm observing that because I rate mostly liked movies, the tier system is way off. I'm not sure how this will ultimately impact my suggestions.

Re: What happens to tiers if a user only rates movies he likes?

Posted: Mon Jan 01, 2018 3:09 pm
by BadCosmonaut
Bort P wrote:This is an issue, because:

-I try to watch movies that I'll like.
-Criticker suggests to me movies that I'll like.

Thus it seems that the better Criticker works, the more skewed the system will be, because I'll have a ton of liked movies (70+) and few less than 40.

I just signed up today, and I'm observing that because I rate mostly liked movies, the tier system is way off. I'm not sure how this will ultimately impact my suggestions.


I don't see an issue with the system here. In your case, if the tiers are skewed it's because you are only rating movies you like. Just rate as many movies as you can that you've seen, including bad ones. There's no reason at all to rate movies you haven't seen.

Also, you've only rated about 50 movies. The system works better with more movies rated.

Re: What happens to tiers if a user only rates movies he likes?

Posted: Tue Jan 02, 2018 1:59 am
by Midaso
What I found on this site (and I assume other people will be the same) is I've seen far more movies than I thought I had because of all the bad movies you see as a kid and forget fairly quickly

Re: What happens to tiers if a user only rates movies he likes?

Posted: Tue Jan 02, 2018 3:41 am
by 90sCoffee
I've mentioned this before, the solution for me is to change the tier percentages so that they aren't all equal to 10% of your films each. Tier 1 should be your bottom 4ish% of films, tier 2 should be around 8%, tiers 6 and 7 should be closer to 15ish%, tier 8 should be around 12%, tier 9 should be 7ish%, and tier 10 should be reserved for your top 5% favourite films. Those are rough numbers but you get the idea, they should represent the parabolic shape that most people have with their ratings. By assigning an equal 10% to ratings like it is now, it assumes that everyone has a flat shaped graph of their rating distribution which most don't.

Re: What happens to tiers if a user only rates movies he likes?

Posted: Thu Apr 19, 2018 7:31 pm
by livelove
quote from related thread because the user makes a good point:

philamental wrote:If you decided to rank 100 movies, and it just so happens they are your 100 favourite movies of all time, and they vary in score of 90 to 100, then the movies you ranked the lowest will be your Tier 1 films, red with whatever negative sounding default quip goes along with it. The tier system basically aims to 'separate' the films you've rated and judges them against each other. If 90 out of 100 is the lowest score you have ranked on this site, then that movie will be deemed the worst movie you have ever seen ... because it doesn't know about any other movies you've seen.
In short, you need to rate bad movies too for criticker to understand what movies you see as being good.

Re: What happens to tiers if a user only rates movies he likes?

Posted: Thu Apr 19, 2018 7:36 pm
by livelove
mpowell wrote:Re: What happens to tiers if a user only rates movies he likes?
[…]
Regardless of the score, the lowest 10% of your ratings will be in Tier 1.*
Criticker does work best for those with a wide variety of rankings.
* i.e. bottom 10% percentile

I don't know, but after all maybe it's necessary to take into consideration the user's vote distribution (the curve shape) to some extent, as suggested above by 90sCoffee?

90sCoffee wrote:the solution for me is to change the tier percentages so that they aren't all equal to 10% of your films each.

Tier 1 should be your bottom 4ish% of films,
tier 2 should be around 8%,
tiers 6 and 7 should be closer to 15ish%,
tier 8 should be around 12%,
tier 9 should be 7ish%, and
tier 10 should be reserved for your top 5% favourite films.

Those are rough numbers but you get the idea, they should represent the parabolic shape that most people have with their ratings. By assigning an equal 10% to ratings like it is now, it assumes that everyone has a flat shaped graph of their rating distribution which most don't.


I agree that it would be worthwhile to at least think about what role the graph shape plays and what implications it has for comparing 2 users with different graph shapes to each other (which is the whole point of criticker).

I don't think that assigning fixed (Bell curve induced) values to the tiers (as 90sCoffee suggests), because what happens then to to users with a flat shaped graph (or, generally speaking, anything other than a Bell curve) ?

Also, the Bell curve suggestion above would only mitigate 1 aspect of non-linear graphs, but it doesn't deal at all with the problem highlighted by the OP, namely users who "only rate [their] most favourite movies and only give out scores like 80 and above", regardless of whether these scores from 80-100 have a flat or Bell curved graph.

This type of situation seems in need of a different solution.