trippingly wrote:I find that when I'm on the page for a particular movie and I'm looking at the sidebar with user ratings and reviews, if it shows users' actual scores instead of something that's been normalized like percentiles or tiers, it's difficult to interpret and compare. Each of those scores only makes sense in the context of that user's individual rating system, which is unfamiliar to me. A score that looks high might actually equate to a low percentile, and vice versa. It's just hard to know what to make of them when they're taken out of each user's rating system but not normalized so they can be meaningfully compared.
perhaps we should just see the percentiles on the film pages. seems like it would have the fewest drawbacks
One more quick thought... When converting from percentiles into PSIs for a given user, would it be possible to interpolate? I think that would further improve prediction accuracy, especially for users who don't assign many different scores.
So for example, a user might only assign 100, 90, 80, and so on, but their predictions could fall in between. Maybe their score of 90 works out to the 80th percentile, and their score of 80 works out to the 60th percentile (just to keep the math simple). A film in the 70th (average TCI) percentile would show up as an 85. (There would be a range of percentiles that converted to 85, another range for 86, and so on.)
I don't know if this would be easy or difficult to implement, and it's definitely not crucial or anything. One can always click on the PSI box and get all the details, and ordering by PSI on the full database page seems to take small percentile differences into account (higher percentiles are listed before lower percentiles even if the PSI shown to the user is the same).
But with interpolation, a user could really tell at a glance if for them a film is a very likely to be a solid 90 or if it's closer to a tossup between that and another one of their ratings.
trippingly wrote:Thought I'd also illustrate with my own scores. I only use six... I imagine most Criticker users like a little more granularity in their ratings, but I like not having to make fine distinctions between movies I liked about the same. Anyway, you can see the distribution below...
Thanks for this detailed breakdown. Actually, now that we have seen this in action, we're already planning on doing something similar. Some users who only rank up to 10, but then have a lot of "10s"... and the resulting percentile "feels" too low.
I'm adding your notes and example into our enhancement report. This is going to be something we get to sooner, rather than later.
i'm glad to hear this because i was going to suggest the same thing. on a scale like mine, which has only nine points, some of the scores are being severely undervalued. for example a score of '75' for me ranges from the 60th percentile all the way up to the 79th, and so my 75s ought to be classed as 69th or 70th percentile (depending on whether you round up or down), not 60th.
Zarkon wrote:- "A score of ... puts this in the ... percentile" line is missing on films with scores at the very bottom (sub-1.0 percentile, maybe).
I think this one is taken care of... it wasn't appearing in the Info Box on the Film Database page for percentiles that were equal to 0. As far as the tool-tips go, I think it should be working now, but it's possible we've missed a spot. If you see this happen somewhere else, please let us know exactly where.
zevel6 wrote:I agree that there does seem to be an issue with the TCI list after changing to percentiles. For example, my top 12 TCIs currently show a TCI score of under 3 yet when i click the users the TCI shows 100x the number from the TCI list. It's as if the TCI list still calculates it per tier on some users.
This definitely sounds like a bug. It's possible there's some place in the code which is still using tiers instead of percentiles for TCIs, even for those accounts which have already switched. We'll take a look and see if we determine where it's happening.
I can confirm this is still happening and is affecting all of my PSIs.
zevel6 wrote:I also have this same issue. The percentiles are not visible anywhere on the profile page. In fact, the only place the percentiles are visible for me is when clicking on the info box on a film under "All Your Rankings". Yesterday when I clicked into a specific film its percentile was visible but that seems to have disappeared today.
Alright, we've added percentiles to the scores found on the User Profile pages, much in the same way they are present when using Tiers. Also, we've made them much more visible on the Film Information page, when you've ranked a film.
