Simple formula for scoring rank

Ed

Ed

Soldato
Joined
28 Apr 2004
Posts
4,979
Location
Hastings
Hi,

I'm after a simple formula expression to give rank to items based on their number of votes and their score from one to five.

------------
Item #1
Votes = 1,000
Score = 3.9 out of 5
------------
Item #2
Votes = 1
Score = 4 out of 5
------------

Should item #2 rank above item #1? I'm looking for a simple way to flatten item scores out to a number which is meaningful as the owners of the items could find this highly contentious.

Thanks.
 
Soldato
Joined
28 Oct 2006
Posts
12,456
Location
Sufferlandria
There's lot of different calculations you could use to order them but nobody can answer your question for you: does item #2 rank above item #1?
You need to decide on the answer to that first, then you can find a formula which fits with what you want.

The most common ways I can think of would be:
  • Rank by average of votes
  • Rank by average of votes once items have more than a certain number of votes
  • Artificially add some 1* votes to each item (not visible to users) which will favour items with more votes.
 
Last edited:
Permabanned
Joined
9 Aug 2009
Posts
12,236
Location
UK
Need more sample data.

A basic solution:
ORDER BY (Votes * Score) DESC

Basically making popular things more likely to stay popular tho.
 
Last edited:
Soldato
Joined
6 Mar 2008
Posts
10,078
Location
Stoke area
Honestly, I wouldn't even rank an item until it had over 100 votes, and even then I'd still consider leaving it out. If you go by average votes a single voted 5-star item could rank top even if it's poor.

Questions that you need to consider is what quantity of votes is each item willing to get?

Are you having a single score board? You could have items with 100-1000 votes in one scoreboard and anything over 1000 in a separate one.

As a quick idea I would just having something like:

IF NO_OF_VOTES < 100 THEN IGNORE
ELSE ORDER BY AVERAGE_SCORE
 

Ed

Ed

Soldato
OP
Joined
28 Apr 2004
Posts
4,979
Location
Hastings
Need more sample data.
The sample data above is only in principle; it's the point of determining position given those two scenarios, and what's deemed fair.

There's lot of different calculations you could use to order them but nobody can answer your question for you: does item #2 rank above item #1?
You need to decide on the answer to that first.

The most common ways I can think of would be:
Rank by average of votes
Rank by average of votes once items have more than a certain number of votes
Artificially add some 1* votes to each item (not visible to users) which will favour items with more votes.
They are good points and highlight the meaningless of rank in this context which is what I'm battling with.

Honestly, I wouldn't even rank an item until it had over 100 votes, and even then I'd still consider leaving it out. If you go by average votes a single voted 5-star item could rank top even if it's poor.

Questions that you need to consider is what quantity of votes is each item willing to get?

Are you having a single score board? You could have items with 100-1000 votes in one scoreboard and anything over 1000 in a separate one.

As a quick idea I would just having something like:

IF NO_OF_VOTES < 100 THEN IGNORE
ELSE ORDER BY AVERAGE_SCORE
I'm thinking of going down this route. Numbers of votes having a certain weight would certainly be "fairer" than overall score but it just needs to be balanced.

I'll play with the data and see what looks "right" and get some layman feedback too as it's all subjective anyway.
 
Soldato
Joined
28 Oct 2006
Posts
12,456
Location
Sufferlandria
I'm thinking of going down this route. Numbers of votes having a certain weight would certainly be "fairer" than overall score but it just needs to be balanced.
I'll play with the data and see what looks "right" and get some layman feedback too as it's all subjective anyway.

If you do something like:

rank = added total of vote scores + 10 / total number of votes + 10

You've effectively added 10 1star votes to each item. It wont have much of an effect on the item with 1000 votes but will significantly reduce the rank of the item with 1 vote. The more votes an item gets, the smaller the effect this has on it.

However, if the item owners are likely to get annoyed at your ranking system, I'd avoid any sort of manipulation of the numbers. Purely just use average vote.
 
Permabanned
Joined
9 Aug 2009
Posts
12,236
Location
UK
Need more sample data.

A basic solution:
ORDER BY (Votes * Score) DESC

Basically making popular things more likely to stay popular tho.

OK I think I was on the right track here. To resolve the issue we just need to convert both factors to a fraction. We can do so by dividing them by their largest possible value. The value for score is presumably 5. The value for votes can be dynamic, based on which item has had the most votes, so you don't need to adjust this formula if your site is more/less popular than you think.

ORDER BY ( (Votes/MaxVotes) * (Score/5) ) DESC

So to run some test data:
0tp3ibf.png
 
Soldato
Joined
28 Oct 2006
Posts
12,456
Location
Sufferlandria
OK I think I was on the right track here. To resolve the issue we just need to convert both factors to a fraction. We can do so by dividing them by their largest possible value. The value for score is presumably 5. The value for votes can be dynamic, based on which item has had the most votes, so you don't need to adjust this formula if your site is more/less popular than you think.

ORDER BY ( (Votes/MaxVotes) * (Score/5) ) DESC

I think you need to put a lot more weighting on the score compared to the number of votes.
The 2nd item in your list only scores 1 star but because so many people voted for only 1 star, it ends up higher up the list.
 
Soldato
Joined
28 Oct 2006
Posts
12,456
Location
Sufferlandria
Assuming that vote options are 1 - 5 and 1 is lowest rating, I'd say that having 600 votes at lowest rating is very damning for that item and it should probably be last in order.
 

Ed

Ed

Soldato
OP
Joined
28 Apr 2004
Posts
4,979
Location
Hastings
Thank you for everyone's contribution; I knew it would be a tricky subject.

Or you can use the votes to sort of stretch out the scale, so more of a high score makes it higher, more of a low score makes it lower, like so:

QwiXNEA.png

As long as it's a reasonable expectation of position given the numbers, this would do it.

I did see this but didn't fancy the task of PHP'ing it. It's something I can turn to later on to refine the service.
 
Back
Top Bottom