From Tiers to Percentiles: Introducing a Big New Change to Criticker

Ideas to improve Criticker and new feature requests, as well as announcements about new enhancements.
mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

ramynoodle wrote:Found a little bug. When I regenerate my PSIs in the full database, the PSI for every movie becomes 30.


This is a bug we're aware of... something similar is happening on other pages, where you can regenerate PSIs. We'll be solving this soon. Thanks for the note!

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

mattorama12 wrote:Anyway, one thing that I think is lost with the new system is a quick visual way to see things in the "all rankings" view. When we had tiers, there was a divider for each one. Having a divider for every percentile would be too busy, but I think it would be nice to have a divider for every 10th percentile or even 5th percentile.


This is something we are definitely considering.

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

Zarkon wrote:The info box isn't accessible on Profile pages, and the mouseover popup doesn't work on the scores there, either.

Yeah, the info box is only on the database page. The mouseover, however, should work anywhere the percentile appears. It seems to be working for me on the profile pages... can you confirm that it's not appearing? We'll check it out, if you can reproduce the behavior.

Zarkon wrote:Honestly, the mouseover popup is more busy, slower to get information out of (especially with the fade-in), and less mobile-friendly compared to the always-visible "T#" numbers that it replaced.

This was a concern. There's a couple places where we've replace the "T#" with a number like "45%" or "45th". But in places where both score and percentile are shown, it's kind of confusing to see something like "92 93%" The mouseover seemed like a good way to show the information in a straight-forward way. It should load fast, because it's a single database lookup. Our server is under a bit of strain at the moment, with the number of users switching (and fully regenerating their TCIs/PSIs). But if the lag continues, we'll take a look.

Zarkon wrote:Other things:
- "A score of ... puts this in the ... percentile" line is missing on films with scores at the very bottom (sub-1.0 percentile, maybe).

Okay, I can totally see why this might be happening. It's probably just a stupid coding oversight, we'll look into it.

Zarkon wrote:I think it's about time the score distribution chart is changed to a bar graph or histogram. The line graph isn't appropriate when most users only use a subset of the 0-100 scoring.-

On that same page, it would be nice to have a percentile (y-axis) vs score (x-axis) graph.

We'll consider both of these. Most users actually employ the full range in their rankings (the average is actually around 50 unique scores for users w/ more than a hundred rankings). And for them a bar chart would be ugly. But it would make sense for users that have only, say, 20 or less distinct scores. There's no real reason we can't do both. The percentile/score chart is an interesting idea, too. We've written both down.

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

zevel6 wrote:Also, there appears to be an issue with extracting the results since the change to percentiles (i get the following error message). Would it be possible to include the percentiles in the extract?


This should be fixed now!

zevel6
Posts: 42
4227 Ratings
Your TCI: na
Joined: Wed Mar 16, 2011 3:20 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by zevel6 »

mpowell wrote:
zevel6 wrote:Also, there appears to be an issue with extracting the results since the change to percentiles (i get the following error message). Would it be possible to include the percentiles in the extract?


This should be fixed now!


Much appreciated :D

zevel6
Posts: 42
4227 Ratings
Your TCI: na
Joined: Wed Mar 16, 2011 3:20 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by zevel6 »

mpowell wrote:
Zarkon wrote:The info box isn't accessible on Profile pages, and the mouseover popup doesn't work on the scores there, either.

mpowell wrote:Yeah, the info box is only on the database page. The mouseover, however, should work anywhere the percentile appears. It seems to be working for me on the profile pages... can you confirm that it's not appearing? We'll check it out, if you can reproduce the behavior.


I also have this same issue. The percentiles are not visible anywhere on the profile page. In fact, the only place the percentiles are visible for me is when clicking on the info box on a film under "All Your Rankings". Yesterday when I clicked into a specific film its percentile was visible but that seems to have disappeared today.

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

zevel6 wrote:I agree that there does seem to be an issue with the TCI list after changing to percentiles.
For example, my top 12 TCIs currently show a TCI score of under 3 yet when i click the users the TCI shows 100x the number from the TCI list. It's as if the TCI list still calculates it per tier on some users.


This definitely sounds like a bug. It's possible there's some place in the code which is still using tiers instead of percentiles for TCIs, even for those accounts which have already switched. We'll take a look and see if we determine where it's happening.

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

zevel6 wrote:I also have this same issue. The percentiles are not visible anywhere on the profile page. In fact, the only place the percentiles are visible for me is when clicking on the info box on a film under "All Your Rankings". Yesterday when I clicked into a specific film its percentile was visible but that seems to have disappeared today.


Okay, we'll take a look. I think I was misunderstanding the issue before, but now I have a more clear idea.

mpowell
Posts: 3853
1201 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

quartier wrote:But my regenerate PSIs button are not working.


I believe this should now be working again, both on the New Releases and Film Database pages. Please let us know if it's not!

trippingly
Posts: 33
1045 Ratings
Your TCI: na
Joined: Wed Jul 17, 2013 5:18 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by trippingly »

I know there are lots of improvements underway at the same time, but just to concur with Zarkon on this one...

mpowell wrote:
Zarkon wrote:Honestly, the mouseover popup is more busy, slower to get information out of (especially with the fade-in), and less mobile-friendly compared to the always-visible "T#" numbers that it replaced.

This was a concern. There's a couple places where we've replace the "T#" with a number like "45%" or "45th". But in places where both score and percentile are shown, it's kind of confusing to see something like "92 93%" The mouseover seemed like a good way to show the information in a straight-forward way. It should load fast, because it's a single database lookup. Our server is under a bit of strain at the moment, with the number of users switching (and fully regenerating their TCIs/PSIs). But if the lag continues, we'll take a look.


I find that when I'm on the page for a particular movie and I'm looking at the sidebar with user ratings and reviews, if it shows users' actual scores instead of something that's been normalized like percentiles or tiers, it's difficult to interpret and compare. Each of those scores only makes sense in the context of that user's individual rating system, which is unfamiliar to me. A score that looks high might actually equate to a low percentile, and vice versa. It's just hard to know what to make of them when they're taken out of each user's rating system but not normalized so they can be meaningfully compared.

The mouseover option does help some, but it means you have to "look into" each score to figure out what it means (and as Zarkon pointed out, this is not so convenient on the phone).

Also, are the scores in that sidebar color-coded in a "normalized" way that indicates the percentile? That seems to be the case, which is helpful too, but I guess I haven't been relying on the colors too much because I wasn't sure if they were normalized or if they reflected each user's color choices, since they're configurable.

I think the sidebar would probably be easier to really take in if the percentiles were primary and the original user scores were secondary or optional. One possibility would be to switch the current arrangement and show the percentiles in the sidebar and have the original user scores in the mouseover.

Or the percentiles could be large and colored and the scores could be small, the way they are on the PSI page for a particular movie (the page that comes up when you click the box with a movie's probable score). I think that works really well. I like how the percentiles are emphasized, and the scores are there but smaller, in case one is interested. (To my eye, "45%" might be a bit easier to read than "45th", but that's a minor detail.)

And a third option would be to show scores that have been converted not only from original score into percentile but then also into the rating system of the user who's viewing the page. Maybe that would seem too "adjusted", considering that these are other users' ratings that appear next to their name, but on the other hand, numbers that have been converted into the rating system of the user viewing the page are probably the numbers that are most immediate and meaningful for that user. Just a thought.

Post Reply