From Tiers to Percentiles: Introducing a Big New Change to Criticker

Ideas to improve Criticker and new feature requests, as well as announcements about new enhancements.
TheDenizen
Posts: 1638
3114 Ratings
Your TCI: na
Joined: Thu Apr 13, 2006 12:51 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by TheDenizen »

Love this change, thanks for the continued awesomeness mpowell!

trippingly
Posts: 33
1045 Ratings
Your TCI: na
Joined: Wed Jul 17, 2013 5:18 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by trippingly »

mpowell wrote:
trippingly wrote:
Image

where cℓ is the count of all scores less than the score of interest, ƒi is the frequency of the score of interest, and N is the number of examinees in the sample.


In Criticker terms, that would be:

(# of films with a lower score + half the # of films with the score in question) / total # of films ranked
(multiplied by 100 and rounded down)


We have implemented this. The fact that we weren't accounting for other films of the same rating was a mistake. The percentiles now look better, from what we've seen.

Your percentiles will be recalculated the next time you rank any film.

Thanks so much for taking the time to write this out so clearly!


Wonderful! Thank you! And happy to help in some small way. Seems to be working perfectly. I definitely think the percentiles make more sense now. I've been enjoying comparing with Kumpels.

I wish I'd thought to make a note of the stats from the predictive accuracy page before switching from tiers to percentiles and then again before the tweak to the formula. I can say for sure that when I was using tiers, the predictive accuracy page showed that there were a couple "way off" predictions, and now there are none. I only vaguely remember my average difference before the tweak to the formula as 8-point-something, but it's now 8.01, so I suspect it's probably improved at least a little. So just one user's experience (based on vague recollection), but I'm pretty sure Criticker has become more accurate for me overall.

Also really like how the page for a film you've rated now shows your rating with the equivalent percentile next to it.

Oh, and the blue bar is back to normal.

Thank you for all these improvements and all the hard work that goes into Criticker. The site has really revolutionized my movie watching. I've seen lots of great films I never would have heard of (and I'm sure avoided others i wouldn't like). Really appreciate everything that goes into that!

AFlickering
Posts: 641
2994 Ratings
Your TCI: na
Joined: Mon Feb 26, 2007 6:15 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by AFlickering »

i too want to say thank you for sorting that so quickly, i appreciate how attentive you are to user concerns compared to most other websites. i do think overall this is a great change for criticker, especially so quickly off the back of the games site launching. exciting times for the site.

question: i just renewed my subscription, does that trigger a full TCI regen as well as PSI? and if not, where do i go to buy a full TCI regen? my TCIs are all out as i switched to percentiles before you changed the formula.

edit: nm i found it (ads-free criticker)

AFlickering
Posts: 641
2994 Ratings
Your TCI: na
Joined: Mon Feb 26, 2007 6:15 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by AFlickering »

mpowell wrote:
zevel6 wrote:Also, we've made them much more visible on the Film Information page, when you've ranked a film.


it'd be great if you could do this for kumpel ratings too (preferably on both my own profile's feed and on film pages, but at least the former which you can't even hover over right now), as right now it's just a sea of contextless numbers.

djross
Posts: 1212
5318 Ratings
Your TCI: na
Joined: Sun Apr 16, 2006 12:56 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by djross »

x
Last edited by djross on Thu Jul 20, 2023 9:25 pm, edited 1 time in total.

hellboy76
Posts: 446
6339 Ratings
Your TCI: na
Joined: Mon Sep 20, 2010 1:53 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by hellboy76 »

Noice.

JakeAesthete
Posts: 37
4206 Ratings
Your TCI: na
Joined: Fri Jun 13, 2008 7:41 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by JakeAesthete »

djross wrote:
AFlickering wrote:i too want to say thank you for sorting that so quickly, i appreciate how attentive you are to user concerns compared to most other websites.


Totally.

Also: on a film's page, if my score puts it in the 0th percentile, then the percentile doesn't show up next to the score.

I came here to point this out as well. I think this is the reason my percentiles seem slightly off. A rating of 0 should be included in the bottom 1%, no?

djross
Posts: 1212
5318 Ratings
Your TCI: na
Joined: Sun Apr 16, 2006 12:56 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by djross »

x
Last edited by djross on Thu Jul 20, 2023 9:25 pm, edited 2 times in total.

JakeAesthete
Posts: 37
4206 Ratings
Your TCI: na
Joined: Fri Jun 13, 2008 7:41 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by JakeAesthete »

djross wrote:
JakeAesthete wrote:A rating of 0 should be included in the bottom 1%, no?


Not necessarily, no. If someone gives a score of zero to 10% of the films they see, then it will be the fifth percentile. If they give that score to 20% of their ratings, it will be the tenth percentile. This is because it doesn't just count scores that are lower, but also scores that are the same (but divided by two). This makes sense according to the logic that any particular instance of a particular score should be weighted as falling midway within the set of instances of that score. The previous formula arbitrarily treated a particular instance of a score as falling at the bottom of the set of instances of that score. In other words, a score of zero has a different statistical meaning for someone who gives that score to 10% of their ratings than it does for someone who gives it to 90% of their ratings.

Okay, yes, my mistake. Let me phrase this differently: A score of zero should count as something, and at the moment this appears not to be the case?

djross
Posts: 1212
5318 Ratings
Your TCI: na
Joined: Sun Apr 16, 2006 12:56 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by djross »

JakeAesthete wrote:Okay, yes, my mistake. Let me phrase this differently: A score of zero should count as something, and at the moment this appears not to be the case?


Surely. Must be just a bug I think.

Post Reply