So, as I told you in the chat I think we can easily adapt the ELO system.
You can found here the mathematical explanation of this system : http://fr.wikipedia.org/wiki/Classement_Elo
(this time I found the french page to be clearer on the mathematical principles). In a nutshell the ELO system is :
- A kind of first-order differential equation, which should converge to some point if we always play against the same opponent (we cannot claim that it converges when we play with different opponent, except if we restrict the set of opponents to a very special class)
- A model in which some function of the probabilities can be factorized as a ratio of functions that only depends on the skill of one of two opponents (this is not something that can be proven, it is actually an arbitrary model very far from reality, since in reality we cannot associate the players with a complete order relation satistfying basic properties)
So we see there is not a lot of theoretical stuff, that's mainly arbitrary stuff we can change.
So the master equation is
and we just have to choose a scoring system for W and a p(D) model. And if we want to complicate, the K could be caracteristic of the importance of the game, for instance bigger for games with lots of players.
So what to choose as W and p(D) ?
The simplest would actually be to simply keep the system of chess, only keeping the victorious/loser status and dropping the rank and score information.
On the contrary here's a way to keep the score (then I drop the victorious/loser info, but we can keep it with a linear combination or for instance mapping the winners to [0.5,1] and the losers to [0,0.5]) : mapping the scores to the [0,1] intervall through an affine transformation (or something more complicated if it is needed for more numerical stability). For instance if one player has 1000 points, another 700, and the last 500, the first has W=1 (he totally wins), the last has W=0 (he totally loose), and the second has W=(700-500)/(1000-500)=0.4 (a kind of slightly-losing draw). And we can keep p(D) to the same function since all of its shape is arbitrary except some basic properties such as p(0)=0.5 and p(inf)=1
Actually maybe an important feature of the ELO system is that the sum of the ELO points of the two opponents is conserved in a match : Its variation is K*(W-p(D))+K*(W'-p(-D)) = K*(1-p(D)-p(-D)) = 0
This ensures that the system doesn't converge to a trivial point (except if we have on open system, with a steady input of new noobs). So if we want to keep this interesting property we have to ensure that the W of all players sum to 1.
Therefore I have to drop my suggestion of mapping the scores to [0,1], we should instead map on [0,a] where a is chosen so that all the W sum to 1. For instance in my former exemple 1000/700/500 we now have a/0.4a/0 where 1.4a=1, so the W are finally 0.71/0.29/0