PSI Bug
- lineuphere
- Posts: 10
- 4055 Ratings
- Your TCI: na
- Joined: Mon Jun 27, 2011 4:54 am
PSI Bug
Before changing over to the percentile system, a large number of my films' scores matched their PSI. What happened? Why is the PSI's accuracy failing? I score my movies on a 0-100 scale and I now have zero movies in the database with a PSI over 94, yet I have over 60 films with ratings above 94. Something's is off. I miss the fun of the PSI being more accurate.
- mpowell
- Posts: 3851
- 1201 Ratings
- Your TCI: na
- Joined: Fri Sep 09, 2005 10:22 am
Re: PSI Bug
That does sound like a bug -- the switch to Percentiles should have been more cosmetic, as the same basic backend calculations are used. We'll take a look into your account, and let you know what we find!
- mpowell
- Posts: 3851
- 1201 Ratings
- Your TCI: na
- Joined: Fri Sep 09, 2005 10:22 am
Re: PSI Bug
After looking into this a little, it seems as though the correct calculations are being used. Your score of "94" corresponds to the percentiles 96.45 - 97.25%
If you look at the PSI Explain for As I Was Moving Ahead Occasionally I Saw Brief Glimpses of Beauty, you'll see that it's in your 97th percentile -- nearly perfect, but Lucas M's percentile of 85 brings it down just enough.
I hope this makes sense! It would have (hopefully) been the same under the old system. It's definitely possible that we could enhance this further, and make it even more accurate, and we'll take a look at doing so!
If you look at the PSI Explain for As I Was Moving Ahead Occasionally I Saw Brief Glimpses of Beauty, you'll see that it's in your 97th percentile -- nearly perfect, but Lucas M's percentile of 85 brings it down just enough.
I hope this makes sense! It would have (hopefully) been the same under the old system. It's definitely possible that we could enhance this further, and make it even more accurate, and we'll take a look at doing so!
- lineuphere
- Posts: 10
- 4055 Ratings
- Your TCI: na
- Joined: Mon Jun 27, 2011 4:54 am
Re: PSI Bug
Thanks for investigating. I blocked Lucas M temporarily just to test the rating and I see the PSI jumped up 1 point. However, in addition to that, I ran a test where I blocked all users whose score for As I Was Moving Ahead Occasionally I Saw Brief Glimpses of Beauty was not at 99 percentile. With my top 10 TCIs at 99%, my PSI for the film is still only 96. Why not 99?
So, I understand how this statement works with the percentile system:
Under the tier system, I recall a considerable number of PSIs in the 95-100 range -- and further, as low as 0 (currently, I have no PSI under 10 - not that that is as important; and I understand why, based on the mechanics of the percentile system). The middle-range movies do seem more accurate, but the very high and very low end PSIs seemed to have been shaved off just a hair with the implementation of the percentile system.
Nonetheless, I'm enjoying Criticker as always! Thanks for all the awesome work on the site.
So, I understand how this statement works with the percentile system:
However, I suppose (because I grade on a 0-100 scale), my brain wants to make 96.85 the average score for that percentile. So essentially, a PSI of 94 feels like it should be labeled a PSI of 97 instead.Your score of "94" corresponds to the percentiles 96.45 - 97.25%
Under the tier system, I recall a considerable number of PSIs in the 95-100 range -- and further, as low as 0 (currently, I have no PSI under 10 - not that that is as important; and I understand why, based on the mechanics of the percentile system). The middle-range movies do seem more accurate, but the very high and very low end PSIs seemed to have been shaved off just a hair with the implementation of the percentile system.
Nonetheless, I'm enjoying Criticker as always! Thanks for all the awesome work on the site.
- livelove
- Posts: 759
- 67 Ratings
- Your TCI: na
- Joined: Thu Sep 17, 2015 9:36 pm
Re: PSI Bug
do you mean something like a high-precision normalized score ?mpowell wrote: It's definitely possible that we could enhance this further, and make it even more accurate, and we'll take a look at doing so!
- mpowell
- Posts: 3851
- 1201 Ratings
- Your TCI: na
- Joined: Fri Sep 09, 2005 10:22 am
Re: PSI Bug
We've recently introduced an enhancement to the calculation of PSIs, which makes the "percentiles" a lot more accurate, by storing them as floating point numbers, rather than integers. This allows a maximum "percentile" of 99.9999 as opposed to 99.0, and from our testing, has had the desired affect of greater accuracy.
Thanks again for pointing this out!
Thanks again for pointing this out!