More PSIs

[angela79at]

Hi,

I have an issue with how - or rather when - PSIs are created. If I understood correctly, the PSIs are created from the min. 3/max. 10 Top 1000 TCIs. If you ask me, this is an unnecessary limitation and a PSI should be created even if you don't have 3 people in your Top 1000 that have seen the movie. A PSI created from an "outside the Top 1000 group" is better than none at all, and I currently get PSIs for only about a third of movies anyway.

As a concrete example, there's a movie that is pretty obscure (87 votes on IMDB) and there are only three ratings shown on Criticker. That's IMDB (in my Top 1000) and two others that aren't in my Top 1000. However, I have 554 and 770 movies in common with these two and I clearly share the same interest in the actor with at least one of them, if not both. So a PSI would definitely be meaningful here.

I would imagine that this thing becomes more of an issue the more obscure a movie is and the more movies you've rated (minimum films for TCI), so it probably hurts the most active users the most. I'd like to hear your thoughts on that.

Best wishes,
Angela

[mattorama12]

One thing you can do is lower the minimum number of films for a TCI. You have a healthy number of rankings, so you could probably lower the minimum percentage without hurting the effectiveness.

[angela79at]

Thanks for the reply, but unfortunately I'm already at the minimum number of films in common.

[PeaceAnarchy]

Actually, what you want is to raise the minimum number of films in common. It might seem counterintuitive, but what you want isn't more and lower TCIs, but rather to restrict your TCIs to those who have a good number of films in common even if their tastes are not as close to yours. You do this by raising the minimum, thus getting rid of people who may well be very compatible with you but who've only seen 200 films of which you've already seen 150. That person just isn't going to be giving you many recommendations so it's best to raise the minimum and get rid of them in exchange for more people on the end who've seen more films in common and still are kind of compatible. Just don't raise it too high or you'll start getting recommendations from people who have significantly different tastes.

[mattorama12]

Actually, what you want is to raise the minimum number of films in common. It might seem counterintuitive, but what you want isn't more and lower TCIs, but rather to restrict your TCIs to those who have a good number of films in common even if their tastes are not as close to yours. You do this by raising the minimum, thus getting rid of people who may well be very compatible with you but who've only seen 200 films of which you've already seen 150. That person just isn't going to be giving you many recommendations so it's best to raise the minimum and get rid of them in exchange for more people on the end who've seen more films in common and still are kind of compatible. Just don't raise it too high or you'll start getting recommendations from people who have significantly different tastes.

Interesting. You're quite right that it may be the case that raising the minimum could help alleviate the problem. That's assuming that there are lots more than 1000 people with whom Angela shares the minim number of films in common. By raising the minimum, she's trading those people with the minimum number in common for people who are currently her 1001+ TCI and who have more films in common with her, even if their tastes are more different from her than the current top 1000. Of course it may also be the case that there just aren't that many more people, so raising the minimum will simply lower her "top 1000" group to some number under 1000. If that's the case, then it wouldn't make sense to raise it. Anyway, per PeaceAnarchy's suggestion, Angela should monkey around with the minimum number and see if raising it helps.

[angela79at]

I know what you mean and I could certainly try it, but I'm not sure it's a good idea considering my best TCI is only 2.446 already. I'm actually wondering how seemingly so many people can have TCIs of 1.xxx while I'm nowhere near that. Or maybe people with bad best TCIs don't post much? I don't know.

Anyway, your answer leads me back to my initial thought: if the system took more than just the top 1000 into consideration, we wouldn't have this trade-off between good TCIs and more PSIs, right?

And another thing is that raising the minimum number of films in common wouldn't necessarily include the people who have watched any particular movie that I don't get a PSI for. In the sense that two people may share a certain interest, for a particular genre or for a particular actor, director, etc... while they don't have much in common besides that, and thus haven't got so many movies in common either.

[PeaceAnarchy]

Actually, what you want is to raise the minimum number of films in common. It might seem counterintuitive, but what you want isn't more and lower TCIs, but rather to restrict your TCIs to those who have a good number of films in common even if their tastes are not as close to yours. You do this by raising the minimum, thus getting rid of people who may well be very compatible with you but who've only seen 200 films of which you've already seen 150. That person just isn't going to be giving you many recommendations so it's best to raise the minimum and get rid of them in exchange for more people on the end who've seen more films in common and still are kind of compatible. Just don't raise it too high or you'll start getting recommendations from people who have significantly different tastes.

Interesting. You're quite right that it may be the case that raising the minimum could help alleviate the problem. That's assuming that there are lots more than 1000 people with whom Angela shares the minim number of films in common. By raising the minimum, she's trading those people with the minimum number in common for people who are currently her 1001+ TCI and who have more films in common with her, even if their tastes are more different from her than the current top 1000. Of course it may also be the case that there just aren't that many more people, so raising the minimum will simply lower her "top 1000" group to some number under 1000. If that's the case, then it wouldn't make sense to raise it. Anyway, per PeaceAnarchy's suggestion, Angela should monkey around with the minimum number and see if raising it helps.

Yeah, that's why I added that last line. That said, my minimum films in common is 11% (606) and I have all 1000 TCIs and plenty of people not in my top 1000, so there's plenty of room to manoeuver. I can't imagine most people would have an issue getting 1000 TCIs at 15% for example, and that will make a huge difference from the minimum

[PeaceAnarchy]

I know what you mean and I could certainly try it, but I'm not sure it's a good idea considering my best TCI is only 2.446 already. I'm actually wondering how seemingly so many people can have TCIs of 1.xxx while I'm nowhere near that. Or maybe people with bad best TCIs don't post much? I don't know.

When you rate on what is effectively a 10 point scale your TCIs will naturally be higher because of the missing tiers. It affects everything evenly, so it shouldn't be a big deal in terms of PSIs, but it does make everyone seem less compatible. Of course you may also have a more ideosyncratic taste than most. A better gauge would be what your 1000th TCI is.

Anyway, your answer leads me back to my initial thought: if the system took more than just the top 1000 into consideration, we wouldn't have this trade-off between good TCIs and more PSIs, right?

Yes, but I guess the added database strain isn't considered worth the tradeoff.

And another thing is that raising the minimum number of films in common wouldn't necessarily include the people who have watched any particular movie that I don't get a PSI for. In the sense that two people may share a certain interest, for a particular genre or for a particular actor, director, etc... while they don't have much in common besides that, and thus haven't got so many movies in common either.

No, it won't solve everything, there will always be people you may be compatible with who haven't seen many movies in common, or that one person who's rated very few films but happens to have watched a rare one you may be interested in. Or people who are only compatible in certain genres or for certain directors. But on the whole raising up from the minimum will help a lot, the tradeoff is worth it, as long as that 1000th is still decently useful.

[livelove]

Angela should monkey around with the minimum number and see if raising it helps.

The problem is, that when monkeying around with the minimum number, you don't see any immediate effects (see: viewtopic.php?f=10&t=4982#p50264 ) so you don't know whether you have found a good value or maybe even worsened your situation.

It's like shooting on a target in complete darkness without seeing if you hit it. Aiming higher or lower isn't of much use under these circumstances ...