How Criticker fits into my personal movie recommendations...

Introduce yourself to the community or chat with other users about whatever is on your mind
Quicky
Posts: 451
786 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

How Criticker fits into my personal movie recommendations...

Post by Quicky »

Hi all,
I stumbled on Criticker.com a few weeks ago, and I must say I like it, despite some flaws that would make it even easier to do what I want.

I wanted to tell you something about how I personally look for movies to watch on DVD or in the theater, and how Criticker aids me in that search.

In the past few years I've been developing a movie database in Excel that makes scores (much like PSI here) that tell me which movies I'll probably like and which I probably won't. During those years, the database has gradually gotten bigger and more complex. Before I came across Criticker.com it was calculating a PSI-like score for each movie on 6 parameters: 1) IMDB-score (for 18-29 year olds), 2) Length of the movie, 3) Popcorn-rating (more on that below), 4) Age of the movie, 5) Genre, 6) Awards.

The algorithm is quite complex, but in the end the score is based on these 6 parameters, each having their own weight. I calculate these weights with a program I wrote in IDL so that the final scores most accurately agree with how I rated my movies myself after I saw them (putting more emphasis on the movies I rated high). Now that I know about Criticker, I added a 7th parameter: the PSI shown to me here on criticker.

The weights for each of these parameters in my final score is now as follows:
  1. IMDB: 29.0 %
  2. Criticker PSI: 22.8 %
  3. Length: 13.4 % (apparently longer movies have a better chance at pleasing me... maybe because they have more depth?)
  4. Popcorn: 12.1 % (I calculate a popcorn rating based on the ratio between box office income and IMDB rating. If a movie has a high box office income, but a low IMDB, I claim that it has a higher 'popcorn' rating, i.e. it's less alternative and is able to reach the wide audience. For example, Spider-Man has a high popcorn rating, Donnie Darko has a low popcorn rating)
  5. Age: 10.6 % (Don't ask me why, but I tend to like more recent movies better than older ones. This might be a psychological effect whereby I 'forget' how good an old movie actually was...?)
  6. Genre: 7.8 % (I like history, sport, war better than family, comedy, horror)
  7. Awards: 4.3 % (the more awards a movie has won, the better)

As you can see, Criticker PSI now provides me with a good chunk of new information. It definitely is not perfect (otherwise the parameters would be 100% for criticker and 0% for all the others), but it helps quite well.

Below you can see the top recommendations for me of all movies that I haven't seen yet and that are supposed to be available on dvd at my local video store.

Image

Most of you probably think I'm crazy by now, which is pretty much true, considering that this hobby got way out of hand ;). I also suspect a lot of you will think I should just read reviews online instead of basing my recommendations on numbers. This is probably true as well, however I consider it a challenge to be able to come up with these recommendations solely based on numbers and constantly try to improve the algorithm. I'm a scientist (well... going to be) for a reason ;).

Nevertheless, I was hoping that some of you might be inspired by this post to think of new ways to get their movie recommendations. If you have any questions, suggestions, requests concerning this project, I'd really appreciate it :). If you would be willing to take the time to provide me with your own PSI's on a big list of movies, I'd actually be willing to provide you with your own top recommendations like the one for myself in the picture ;).

And now for the developers here... It would be really helpful if you wouldn't hide PSI's for movies you've already seen :P. I had to delete all of my scores and resubmit them one by one in order to find my PSI for each of these movies so I could test how well PSI conforms to my ratings...

td888
Posts: 835
3955 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

Hahaha, those crazy dutch..... ;)

(but i'm dutch too....)

Quicky
Posts: 451
786 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

Actually, I'm Belgian. I've merely been studying in the Netherlands ;).

td888
Posts: 835
3955 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

My apologies. But this explains of lot... (just kidding)

MmzHrrdb
Your TCI: na

Re: How Criticker fits into my personal movie recommendations...

Post by MmzHrrdb »

Okay. :|

Grit
Posts: 181
799 Ratings
Your TCI: na
Joined: Mon Mar 19, 2007 5:31 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Grit »

I don't really think it is a system that would work for everyone as length, genre and age ratings would differ from person to person making them somewhat irrelevant. Personally I'd see them as irrelevant anyway, to me they are some of the least important qualities and least consistent measures of a good film, particuarly age more than the others. If it is a good film it will shine through, regardless of these qualities.
However as your individual personal guide, it seems to work well. I can't imagine the hours you have put in, you must have been pretty determined to make such a system!

KGB
Posts: 746
1335 Ratings
Your TCI: na
Joined: Thu Jun 15, 2006 12:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by KGB »

So you have a lot of free time on your hands, huh? ;)
Well, I can't say I could find this useful to me, it just seems to mechanic to base the films I'll watch on a study. Yeah, I'll make crappier choices then you, but what good are good films if there aren't bad film to make you see how good the good ones are?
Still, I must say I'm very impressed by this study of yours.

Quicky
Posts: 451
786 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

td888 wrote:My apologies. But this explains of lot... (just kidding)

;)

Hells_Zargon wrote:I don't really think it is a system that would work for everyone as length, genre and age ratings would differ from person to person making them somewhat irrelevant.

First of all, let me say that I never intended this system to work for everybody, and certainly not when you use weights identical to mine. I started the project when I got annoyed at seeing stupid movies and once it started working really well for me personally, I started wondering whether it could be of any use at all to somebody else. The weights used in the system have to be calculated on a personal basis, as like you said, people have different tastes in length, genre and age. I do believe my system could have benefit to everybody, however I think for most people the time wasted (?) in calculating weights and updating the database with new movies, by far outweighs the benefits over something like Criticker (unless I make it into an online application to rival criticker in the year 2064 ;)).

As an example for how tastes differ: the genre rating is automatically calculated using the ratings that you fill in. Yesterday I entered data from my girl friend's movie ratings on facebook, and for her the system favors animation, fantasy and mystery, whereas for me it favors history, sport and war. Our taste in genre is different and that is reflected in the scores. She also feels much more strongly about genre, than I do. Furthermore, whereas I apparently favor longer movies, my girl friend favors shorter movies and where I tend to prefer more recent movies, my girl friend pretty much doesn't care. What surprised me the most however, was that the higher the IMDB-rating for a movie, the lower my girl friend rates it. Clearly, my girl friend doesn't seem to have a 'mainstream' taste. So for my girl friend, Criticker is actually the most successful parameter (highest weight) in determining recommendations.

Hells_Zargon wrote:Personally I'd see them as irrelevant anyway, to me they are some of the least important qualities and least consistent measures of a good film, particularly age more than the others. If it is a good film it will shine through, regardless of these qualities.

True. I never intended this project to judge movies based on cinematographic qualities, acting performances, or whatever. The challenge I set for myself was to determine which movies have the highest probability of satisfying me based on numbers that are readily available online. And as my calculations have shown, for me personally, length, age and genre do have a certain influence on whether I'll like the movie or not. If movie A and B have the same IMDB-rating and the same PSI, but A is longer than B, it's more likely that I'll prefer A, than the opposite.

Hells_Zargon wrote:However as your individual personal guide, it seems to work well. I can't imagine the hours you have put in, you must have been pretty determined to make such a system!

**nods**, I think by now it's in the hundreds of hours... **shrugs**. Maybe I should have just watched movies in all those hours ;). To each his own ridiculous hobby, I guess? :D

KGB wrote:[...] but what good are good films if there aren't bad film to make you see how good the good ones are?
Still, I must say I'm very impressed by this study of yours.

You've got a point there ;).

Philip
Posts: 70
541 Ratings
Your TCI: na
Joined: Fri Dec 28, 2007 6:35 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Philip »

This seems like a very interesting study, but I'm curious to know how you calculated the importance of each factor. If you'd like to explain it more in depth I'd love to hear it.

Quicky
Posts: 451
786 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

Philip wrote:This seems like a very interesting study, but I'm curious to know how you calculated the importance of each factor. If you'd like to explain it more in depth I'd love to hear it.

I've written a program in IDL that optimizes the parameters. This pretty short program basically consists of two parts: 1) evaluate a specific set of paramaters, and 2) provide the first part with new parameters to improve the fit.

I generate sets of parameters by dividing the 7-dimensional space of parameters into 5 points in the direction of each parameter, going from 0 to 100. This means that I evaluate the parameters in 5*5*5*5*5*5*5 = 5^7 = 78125 points. So for example, it would evaluate the parameters at [20, 40, 0, 0, 80, 80, 60], but also at [0,100,60,60,60,40,80]. Then I make a new set of parameters around the best of those 78125 points, but zoomed in. So if [20, 40, 0, 0, 80, 80, 60] was the best point, it then would evaluate it the 78125 points that are pretty close to this point in the 7-dimensional space of parameters, like [22,40,4,2,78,76,62] for example. I do this 12 times (so that the program evaluates about a million sets of parameters), until the 'resolution' is better than 0.01 when the sum of the weights of the parameters is normalized to 100. Granted, this is probably not the best or fastest way to optimize the parameters, but I haven't taken any specific courses on this matter and I couldn't really find something better and/or easier online.

Above I showed you how I generate the sets of parameters. Of course, these have to be evaluated somehow in order to determine which set is better than the others. Each set of parameters corresponds to a certain order of my movies. A set of paramaters that strongly favors old movies will put the old movies that I've seen at the top and will put the more recent ones at the bottom, for example. I then correlate the scores of my movies in this order to an array that goes from 10 to 1. If the scores are ordered quite well from high to low, this correlation will be high. If the scores are not really going up or down on average, the correlation will be pretty much 0, and if the scores are going from low to high, then the correlation will be negative.

So, the better the parameters, the better the scores will be lined up from high to low and the higher the correlation will be. I made the array from 10 to 1 logarithmically, so that it puts more emphasis on the high end of the scale than on the low end, meaning the order of the higher rated movies will be more important than the ones lingering at the back of the grid.

If I've gone over all this too quickly or if you have more questions, let me know. I have no idea what kind of background you have, so I didn't really know how exactly to approach you. I've tried to be complete, yet clear. I hope I succeeded ;).


(Before I wrote the program, I did it by trying to optimize the parameters one by one within Excel, which was obviously a tedious, boring, and not very accurate job.)

Post Reply