Terraforming-mars: Feature: Ranking

[ssimeonoff]

After playing some chess last week, it kept me thinking about extrapolating the simple Elo into TM. - nothing fancy as TrueSkill or Glocko or any other complex system.

The math

The 3 ideas:

• use the "simple" 1vs1 Elo
• Partial losses - use the tournament scoring as example
• "winner takes it all"

Elo

Suggesting starting rating of 1500 and K=20 (pretty standard stuff).
K=20 means that if two equally rated players play 1vs1, a win will give them 10 rating and a lost will cost -10. A draw gives them 0.
So in Elo, you have 3 possible discrete results that are thrown in the big formula:
-a win passes 1 in it
-a draw passes 0.5 in it (50% win and 50% loss)
-a loss passes 0 in it

Partial losses

In the ongoing tournament, a non-wining player can get a set of points based on how far behind from the winner is.
Let's use that logic to fragment the 3 elo states (1, 0.5, 0) even more.

If a player is within X points from the winner, use 0.4 (40% win, 60% loss)
If a player is within 2X points from the winner, use 0.3 (30% win, 70% loss)
If a player is within 3X points from the winner, use 0.2 (20% win, 80% loss)
If a player is within 4X points from the winner, use 0.1 (10% win, 90% loss)
If a player is over 4X points from the winner, use 0 (100% loss)

Many "1vs1" into one game

Compare the winner to every other player and calculate the ratings based on that.
Example: 3 players game , everyone has the same rating - 2nd player is within 2X and 3rd player is in 4X.
Being in 2X results in 0.3 win and 0.7 loss, which will result in losing 4 rating.
Being in 4X results in 0.1 win and 0.9 loss, which will result in losing 8 rating.
The winner takes it all: 4+8 = 12 (zero-sum game)

Define X

In the Tournament X=5 -a constant. But we need to do better.
Points distribution is more condensed with higher player count (Pcount).
So we can do something simple: X = 8 - Pcount.
E.g. X=4 in 4p games and it is X=5 in 3p game.
E.g. in a 3p games, if you are 20 points behind the winner, it is a 100% loss.

Values for X are just example, maybe we should use bigger values, maybe we should use smaller, up for a debate.

Concerns and side-effects

As in the Tournament scoring, same here, players won't care about their standing - being 2nd or 3rd, but only how far behind they are from the leader. This will cause the effect of everyone vs the leader, which I find a very nice mechanic. Also if a player is snowballing, everyone has the same incentive to cut the game short.

---

Implementation

Of course the best option is to have user authentication. Also using the same server/db is a must.
But since security is not concern while playing (sharing player links freely) and trust is decent (everyone assumes that the others won't look at their links/cards), we can get by with a much cheaper implementation.

Frontend/UI

When creating a game, instead of typing the players' names, have the option to choose a "new player" or "existing player"

• "new player" just lets you type free text (input validation if that name does not already exist). At the end of the game, a new player name and id will added.
• "existing player" pull the list from the database (the same way it is done with the seeds) and you can the ids and names of all existing players

Backend/database

A simple table with id(key), player name and player rating.
At the end of the game, crunch the ratings numbers and update the corresponding table rows. (a new player results in a new table row).

Stats page

Just a basic static sorted table is enough for start.

Concerns

Again - trust. Everyone can create a game with anyone's name and mess up my rating, for good or worse (since we dont use auth). But I doubt this to be a problem for the time being.

[kberg]

Before that I'd think we would want to manage logins -- instead of doing all of this rankings can be stored elsewhere -- especially if servers are changing regularly.

In fact, to me the biggest concern is the server's reliability requirements would change from this -- instead of only needing to be reliable through the life of a game, it needs to be reliable ongoing.

It's a good idea, but I'd rather see it managed in another backend entirely.

[ssimeonoff]

The "math" part is the main idea, how the implementation is done - I leave it to the pros :)

[kberg]

I did come up with an engineering idea that doesn't need logins, but I don't love it. Separate service that does manage logins, and is the ranking host. When you want to play a ranked game, you go to the ranking host, log in, and get a one-time key. Then our app connects that one-time key to your player id. At the end of the game, the server delivers the results to the ranking host back-end.

/shrug.

[bafolts]

One service for managing the login and the results should work fine one way or the other. I assume it will need to be moderated as it would be easy to cheat and people will most likely forget their logins.

[samyelias]

Another thing to consider is the different combinations of setups: which expansions are used, how many players, etc. It might be a good idea to have a separate ranking for each. I wouldn't say it's a must, but it's something to consider. I'm not sure what I'd prefer; I've seen it done both ways with other games.

[bafolts]

My largest concern is what @kberg mentioned. We have a good thing going that no one has to keep some central place running which will have a cost associated with it if it is reliable. This centralized tool would need to be kept up as if it went down people wouldn't get ranked.

Brainstorming

At a minimum we need the data point of the key that @kberg mentions. I already pay for the postgres database on the heroku demo site and could keep the logins and rankings there. Could most likely provide write access to administrators to work out any issues that will arise with people claiming there was cheating. Once you were generated a key you couldn't lose it. At the start of a game you would need to be able to provide this key from game options. Ideally there would be nothing stopping anyone from running this token generation and score tracking service if they wanted to run their own ranking server. Could one off this ranking service as a separate npm run start-ranking-service or similar. When starting a game you would have to provide the ranking server host. If not provided then we wouldn't display the option to provide the player key.

[vsrisuknimit]

Another idea: a Discord bot function, $score @vsrisuknimit 100 @nwai90 150 @chosta 110. The bot asks for confirmation. All players mentioned in the command need to react to the bot message for verification. Bot then calculates the ELO and update everyone's rank.

Good

• No need for authentication, Discord already does that.
• Submitting score at the same place many players are finding opponents.

Bad

• Not available to non-Discord user.
• We need to migrate the bot to somewhere else. The free pylon bot that I am using now has many limitations. A major one is limited persisting memory.

[kberg]

Discord bot isn't a bad idea at all.

>

[kberg]

In fact the app could give data for the bot to parse. Could be an
unreadable string or whatever.

[duckmammal]

I don't like how it encourages playing for 2nd, 3rd instead of playing to try to win.

It'd be good if we had winner, within x% of winner (say 5-10%), and losing by a greater fraction.

[ssimeonoff]

@duckmammal ,
Huh?! It doesn't encourage playing for 2nd or 3rd, since it doesn't matter if you are 2nd or 3rd.
The only thing that matters is how far behind from the winner a player is to determine the level of loss.

[duckmammal]

Oh, I misunderstood something when I was reading through the implementation! Thanks for clarifying!

[kevinb9n]

I agree the discord bot seems like enough. I'm not sure it strictly needs to get confirmation from all players so long as each player is notified.

I would strongly discourage doing anything complex and innovative here. I highly, highly doubt it's worth it. Use a well-studied formula off-the-shelf. Just score each matchup within a multiplayer game as its own game (AvB, AvC, BvC, etc) and all that matters for each is which player won between the two of them, ignoring the others. (I think this is what BGA does and I suspect it's a very standard approach.)

[EDIT: it's not just that taking more data into account than this is unnecessary; it also kinda corrupts the game. If I'm winning I should place the last ocean, not think "but I can win by more points if it goes another generation".]

The one thing I do think is essential is to have the concept of a "provisional rating" for players new to the system. Losing to a new-here but very skilled player shouldn't have the same effect on your rating as losing to someone like me. If all new players are treated as unskilled it puts a lot of chaos into the ratings, and also new players will find no one wants to play with them.

[kevinb9n]

You can have rated solo games too. But the player has to commit to a rated game before they can start it, and if they don't finish within [time limit] it's a loss. Take each possible configuration for a solo game (with no bans) and call that a player. A rated solo game equals a match against that player. Voila. Over time it will become clear how much harder some solo configurations are than others. (This can't accommodate custom card bans, so there should be one agreed-on set of cards that can be banned as a block or nothing.)