Open Geo-Guessing Rating Standard (OGRS) 1.0

1 Introduction

1.1 The problem

A geo-guessing round has a hidden answer (a point on Earth) and one or more guesses. The game turns the error, a great-circle distance, into points. Rounds make up formats: duels, where two players lose health by the gap between their scores; free-for-alls of 3–100 players ranked by total points or by elimination; team games; and solo runs against the places themselves.

Ratings in this genre fail in three recurring ways:

1. Starvation. Only one exact rule set counts, typically a world map with set movement, clock and health. Everything else, including most games played with friends, is "casual". On geoguess.es before this standard, one of a player's 51 finished games had been rated. Their rating could not settle because it was almost never fed.
2. Opacity and drift. Undocumented formulas, scales that drift, and no way to tell a settled rating from a guess.
3. Incomparability. A "1500" in one game, mode or format means nothing in another.

1.2 Goals

1.3 Non-goals

OGRS does not prescribe a game's imagery, scoring curve, map or user interface. It does not rate locations for anything except solo play. It does not detect cheating, though it says what must happen when cheating is confirmed (§9.4).

1.4 Conventions

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be read as described in BCP 14 [RFC 2119, RFC 8174] when, and only when, they appear in capitals. Ratings are printed on the familiar Elo-like scale (1500 at the centre). Glicko-2's internal scale uses $\mu, \phi$. Logarithms are natural unless written $\log_2$.

2 Terminology

Round

One place shown to every player at once, then answered. It yields a score per player.

Match

A sequence of rounds with a result for every seat: a placement, or win, draw or loss.

Place

One round's subject: a panorama or photograph with a known position. In solo play a place is rated like a player (§5.4).

Pool

A separate rating kept per player: one per time class for duels, plus Team, Battle and Solo (§6).

Player record

The triple ($r$, RD, $\sigma$): rating, rating deviation and volatility, plus the time of the last update and a game count.

Established / provisional

A record is provisional while its current RD is at least 110, and established otherwise.

People-only match

A match in which every seat is a person: no bots, no stand-ins.

Standard rules

A game's own reference rule set for a class (for example its ranked queue). Under OGRS it identifies a queue. It does not decide whether a game is rated (§7).

3 Round scores

OGRS rates results, not points. A game MAY use any scoring rule, provided it is (a) a monotone non-increasing function of the error distance for a given place, and (b) bounded in $[0, S_{\max}]$. Only solo play (§5.4) uses the score itself, as the fraction $s = \text{points}/S_{\max}$.

A game without a scoring rule of its own SHOULD use the reference rule:

with $k = 1000$ km for the whole world. For a smaller map, $k = \operatorname{nice}(D_{98}/20)$, where $D_{98}$ is the 98th percentile of pairwise great-circle distances between the map's places and nice() rounds to 1, 2 or 5 times a power of ten. At $k = 1000$ km a guess 25 km off scores 4877, 100 km off 4524, and 1000 km off 1839. In the geoguess.es bot model, the separation in score between neighbouring skill levels peaks for $k$ between 750 and 1000 km on a world map (reference implementation notes, §14 [GG]).

4 The rating model

4.1 Glicko-2

OGRS uses Glickman's Glicko-2 system [Glickman 1999, 2001, 2022], a dynamic paired-comparison model in which skill drifts as a random walk with a per-player volatility. It is preferred to Elo [Elo 1978] because a rating carries its own uncertainty (G2, G5). It is preferred to TrueSkill-style factor graphs [Herbrich et al. 2006] because one closed-form engine serves every format on one familiar scale (G4, G6). For a rating $r$ with deviation RD:

Every observation $j$ is an opponent (or a place, or a composite side) with $(\mu_j, \phi_j)$, a result $s_j \in [0,1]$ and a weight $w_j > 0$ (§5). The update for one match is:

The new volatility $\sigma'$ is the root of Glickman's function, solved by the Illinois method with tolerance $\varepsilon = 10^{-6}$:

With every $w_j = 1$ and $t = 1$, this is exactly Glickman's algorithm. Vector V1 in Appendix B reproduces his published worked example.

4.2 Continuous time

Players expect a new rating the moment a match ends, so OGRS applies the update per match, not per batch period. The inflation step uses the real time since the player's previous rated update, counted in rating periods:

This value is lichess's, chosen so that an RD of 60 grows to 110 over a year at $\sigma = 0.06$ (vector V4). Ten matches within one period add up to one period of inflation, as in the batch algorithm. An opponent's deviation in $E$ and $g$ MUST be their current deviation:

4.3 Parameters

A conforming implementation MUST use these values. It MAY publish a different value for a parameter marked as tunable in Appendix C, but then it MUST name the parameter and value beside every rating it publishes (for example "OGRS 1.0, $\tau$ = 0.5").

5 From formats to evidence

Every format reduces to the update in §4 with a list of observations $(\mu_j, \phi_j, s_j, w_j)$. All players in a match are updated simultaneously from their pre-match records.

5.1 Duels

One observation against the opponent: $s = 1$ for a win, $\tfrac12$ for a draw, $0$ for a loss, $w = 1$. The format that decides the winner does not matter: health to zero, most points after $n$ rounds, first to a target.

5.2 Free-for-all

A free-for-all of $N$ players is $n = N - 1$ pairwise games per player, each scored by placement (better 1, equal $\tfrac12$, worse 0). Counting them as $n$ independent games would over-count, because one match's luck (a lucky round, a bad connection) is shared by all of a player's pairings. The information saturates [Ebtekar & Liu 2021]. OGRS weights each pairing so the whole match is worth $W(n)$ duels:

This gives $W(1) = 1$ (a duel), $W(2) = 1.5$, $W(7) = 2.33$ (eight players), $W(99) = 2.94$ and at most 3. A better placement always earns more (vector V5). Ties in a placement are draws.

5.3 Teams

Each side $T$ of $m$ members is a composite with the mean strength and the uncertainty of a mean:

Each member plays one observation against every other side's composite. The expectation uses the member's own side's composite $\mu_T$, because the team won or lost together. The update uses the member's own $\phi, \sigma$, so an uncertain newcomer moves more than an established teammate. The weight is $w = \frac{W(n)}{n}\cdot\frac{1}{\sqrt{m}}$, where $n$ is the number of other sides. A strict chain rule would give $1/m$, but then a member of a team of four would need about 40 games to be placed. $1/\sqrt m$ is the compromise (a 2-v-2 win is worth about 72 % of a duel win: V2, V6). Uneven teams (2 v 1, 3 v 2) are allowed: the composite already accounts for size.

5.4 Solo play: players against places

In solo play there is no opponent, so the place is the opponent. Each place carries a difficulty rating $\beta$, defined as the Solo rating at which the expected round score is half the maximum. This is the Rasch idea [Rasch 1960] in Glicko form, as used for adaptive learning [Pelánek 2016] and chess puzzles. A round is one observation with a fractional result:

A whole solo game is one update (one Glicko-2 rating period) containing all of its rounds. The order of the rounds therefore does not matter, and a five-round game counts once for inflation (vector V7).

Scale. One round decides much less than a whole duel. In simulation, mean points against rating run about 2.3 times flatter than the duel win curve. The Solo pool and place ratings therefore use a doubled scale, $2q = 347.4356$ (Elo base 800), so a Solo rating means roughly the same skill as the same number in Rapid duels.

Places learn slowly and only from established players. A place starts from a prior by difficulty tier (RD 300) and is updated, with result $1 - s$, only by players whose Solo RD is at most 150. Place parameters: $\tau = 0.3$, starting $\sigma = 0.03$, RD floor 30, ceiling 300, no inflation with time. An implementation MAY also teach places from a duel class it has calibrated to the Solo scale (geoguess.es uses Rapid and Classical rounds). It MUST NOT teach them from provisional players.

What a Solo game may vary. Clock, movement, region and difficulty tiers MAY vary: each place's own difficulty absorbs the region and tier. The view rules and the clock make a place harder. OGRS 1.0 accepts that bias in exchange for G1. Version 1.1 is expected to add a per-condition offset (§13).

6 Rating pools

A record is kept per (player, pool). A duel's pool is its time class, from the viewing time per round $T$ (seconds):

The reference implementation does not yet offer rated Daily duels. A chess-clock bank of $B$ seconds over $R$ rounds with increment $I$ is classed by $T = B/R + I$. Movement rules do not split pools. When a pool is opened for the first time, the start is the inverse-variance mean of the player's established people pools with RD 250, or the self-assessed level (§8.1) if none is established.

7 What is rated: the people-only rule

Why. Custom rules change the game for both sides equally, and Glicko-2 measures relative performance. Restricting rated play to one rule set made the rating starve (§1.1), and a starved rating never settles. A game MAY still keep a separate leaderboard for its standard rules. Players MAY choose a casual game, and the choice MUST be shown before the game starts.

7.2 What keeps a game unrated. A match MUST NOT be rated when:

• any seat is a bot or an automated player (OGRS 1.0; see §13);
• any player had assistance the others lacked: reference panels, hints, guides;
• the places could be known in advance: a replayed set, or a user-made map whose author is playing;
• it was aborted before the first round closed, or voided (§9.3);
• a player is under a rated-play ban (the game MAY then go ahead casual);
• the same two players have already played the pair limit of rated games in the window (§9.1).

7.3 Solo. A solo game is rated in the Solo pool when its rounds are panoramas or photographs answered with a pin and scored by distance. Card, quiz, city-name and country-only rounds, and places without a difficulty rating (user-made maps), are not rated, because a place's difficulty is defined for pin rounds on rated places.

7.4 Calibration statement. An implementation SHOULD publish how its rating relates to accuracy, as rating points per halving of the median error, estimated as in §10.4.

8 Starting, provisional status and display

8.1 Self-assessed start

A new player MAY answer one question at sign-up. The answer sets the start, always with RD 350, so a wrong answer costs only a few games:

8.2 Display

• A provisional rating MUST be marked, for example "1534?", with a tooltip saying it is still settling.
• An established rating SHOULD be shown with its 95 % interval, $r \pm 2\,\mathrm{RD}_{\text{now}}$, at least on profiles.
• Before a rated game, the stakes (rating change on a win, draw or loss) SHOULD be shown. Win chances shown to players MUST combine both uncertainties: $E\big(\mu_a,\mu_b,\sqrt{\phi_a^2 + \phi_b^2}\big)$.
• Displayed ratings are rounded to integers and never digit-grouped. Records MUST keep full precision.

9 Integrity

9.1 Pair limit. Win-trading is bounded by a pair limit: the 6th and later rated games between the same two players within 24 hours are casual (geoguess.es: 5 per rolling 24 h).

9.2 Aborts and disconnects. A player who leaves before the first round closes aborts the match (unrated). Repeated aborts SHOULD earn a queue cooldown. After a disconnect, the server clock keeps running and missed rounds score 0. A resignation is a loss.

9.3 Voiding. A voided match is removed by marking its rating events void, restoring each player's record from just before it, and replaying their later rated matches in order. Refunds and adjustments made after it keep their deltas.

9.4 Refunds. When cheating is confirmed, every person who lost a rated game to the cheater within the past 30 days MUST get those points back as a separate refund event.

9.5 Leaderboards. A record SHOULD appear on a leaderboard only when RD ≤ 80, it has at least 20 rated games (10 of them against people), it has been active within 90 days, and the account is at least 7 days old.

9.6 Idempotence. Rating a match MUST happen at most once, atomically, keyed by the match. Applying it a second time MUST change nothing.

10 Simulation study

10.1 Method

All studies use the reference engine and a seeded generator (mulberry32). The script, its output and the seeds are published with this standard (§12.3), and any implementation can re-run them. Ground truth: every simulated player has a fixed true skill $\theta \sim \mathcal N(1500, 350^2)$, and a duel is won with probability $1/(1 + 10^{-(\theta_a - \theta_b)/400})$. Matchmaking pairs neighbours in the current ranking with noise (standard deviation 120), or finds opponents within ±150 of the player's rating (§10.2). The Elo baselines start everyone at 1500 with a fixed $K$.

10.2 How fast a rating settles (S1)

The number of games until a new player (1000, RD 350) is no longer provisional, over 400 players per row:

Matchmaking matters: evenly matched games carry the most information, because $E(1-E)$ peaks at $E = \tfrac12$. The Solo pool places a player as fast as matched duels do. This is why OGRS rates solo play (G1, G2).

10.3 Accuracy and stability (S2, S4)

1000 players, 100 rated duels each. The error is measured after removing the population-wide offset (a rating scale has no absolute zero):

Error = root-mean-square of rating minus true skill, in rating points. Rank corr. = Spearman correlation with true skill.

Elo has to choose between speed and calm: a larger $K$ converges faster but swings more. Glicko-2's deviation gives both. Once players are established, the median standard deviation of a player's rating over their last 40 games (true skill fixed) is:

OGRS is calmer than Elo K16 once settled, and in the same simulation it converged faster than Elo K40. This is G2 and G3 together.

10.4 From guesses to results (S3)

To check that a logistic win model suits the genre, 20 000 simulated duels were played from distances. Each player has a median error $m$ (km). A round's error is log-normal around $m$ (log-SD 0.9) and is scaled by the place's difficulty (log-SD 0.5). Points follow the reference rule (§3). Duels are played to 6000 health with ×1, ×1.5, ×2 and ×3 damage bands over at most ten rounds. Fitting $P(\text{win}) = \operatorname{logistic}(c\,x)$ by maximum likelihood, where $x$ is the difference in $\log_2 m$, gives $c = 2.44$. One halving of the median error is then worth about 424 rating points in this model.

The logistic fits within 4 percentage points everywhere. It is slightly over-confident for close matches, because health duels add luck that skill alone does not explain. The 424-point figure depends on this noise model. Each game SHOULD estimate its own figure from its own data (§7.4).

10.5 Free-for-all weight (S5)

The winner of an 8-player free-for-all among equals (all 1500, RD 350) gains 267 points and ends at RD 244. Two duel wins in the same situation give +250 and RD 256: one eight-player win is worth a little more than two duels, as $W(7) = 2.33$ intends.

10.6 Limitations

The simulations assume a fixed true skill, while real players improve. $\tau = 0.75$ exists for that, but the study does not measure tracking of a moving skill. Real matchmaking is thinner than ±150 at quiet hours, which sits between the "matched" and "unmatched" rows of S1. The solo study draws places from a population with known difficulty. In practice place ratings are learned, and their own error adds to the player's. These are the measurements the reference implementation will publish from live data.

11 Data exchange and comparability

11.1 Records and events

An implementation that exposes ratings SHOULD use these JSON shapes (numbers at full precision; times as milliseconds since the Unix epoch, UTC):

// a player record in one pool
{ "standard": "OGRS/1.0", "pool": "rapid", "rating": 1534.2, "rd": 71.9, "vol": 0.0601,
  "lastAt": 1794761234000, "games": 48, "provisional": false }

// one rating event (append-only; one per player per rated match)
{ "standard": "OGRS/1.0", "matchId": "7D3ZN60XZ6", "pool": "battle", "at": 1794761234000,
  "format": "ffa", "fieldSize": 8, "placement": 2, "result": 0.857,
  "before": { "rating": 1520.0, "rd": 80.1, "vol": 0.06 },
  "after":  { "rating": 1533.6, "rd": 77.4, "vol": 0.06 }, "delta": 14, "voided": false }

result is the player's mean $s_j$. For solo games, fieldSize is the number of rounds observed.

11.2 Comparing games

Ratings from two games that both conform to OGRS share a model, parameters and starting levels. They are comparable only to the extent that their populations overlap. A game SHOULD NOT present another game's rating as its own. Games MAY link scales through players who are rated on both, by publishing the mean offset between their ratings with its standard error and the number of shared players.

12 Conformance

12.1 Levels

A game claims conformance by stating the level and version ("OGRS 1.0 Full") and any tunable parameter it changed (§4.3).

12.2 Passing the vectors

An implementation passes a vector when its output matches Appendix B to within 0.01 in rating, 0.01 in RD and 0.00001 in volatility (stakes match exactly).

12.3 Published material

• ogrs-reference.mjs: the reference engine (pure JavaScript, no dependencies; MIT).
• simulate.mjs and results.json: the study in §10 (node simulate.mjs ./ogrs-reference.mjs).
• vectors.mjs and vectors.json: Appendix B.
• parameters.json: Appendix C.

13 Versioning

A version number is MAJOR.MINOR. A minor version only adds requirements or tunable parameters, so a 1.0 rating stays valid under 1.1. A major version may change parameters or the model, and ratings move to it by replaying the event history. Every version stays published at its own address.

Open questions for 1.1:

1. Bots as anchors. Fixed-rating bots could tie the scale to fixed reference opponents and give quiet hours a rated game. One option: half-weight observations ($w = 0.5$, as lichess halves rating changes against bots) with a daily cap.
2. Solo condition offsets. A learned offset per clock and movement class for place difficulty (§5.4).
3. Measured calibration. Replacing the simulated figures in §10 with live-data estimates, including how well a moving skill is tracked.

14 References

1. Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3/4), 324–345.
2. Ebtekar, A., & Liu, P. (2021). Elo-MMR: A rating system for massive multiplayer competitions. Proceedings of The Web Conference 2021, 1772–1784.
3. Elo, A. E. (1978). The Rating of Chessplayers, Past and Present. Arco.
4. Glickman, M. E. (1999). Parameter estimation in large dynamic paired comparison experiments. Journal of the Royal Statistical Society, Series C (Applied Statistics), 48(3), 377–394.
5. Glickman, M. E. (2001). Dynamic paired comparison models with stochastic variances. Journal of Applied Statistics, 28(6), 673–689.
6. Glickman, M. E. (2022). Example of the Glicko-2 system. Boston University (revised 22 March 2022). glicko.net/glicko/glicko2.pdf
7. Herbrich, R., Minka, T., & Graepel, T. (2006). TrueSkill: A Bayesian skill rating system. Advances in Neural Information Processing Systems 19.
8. Pelánek, R. (2016). Applications of the Elo rating system in adaptive educational systems. Computers & Education, 98, 169–179.
9. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Danish Institute for Educational Research.
10. Bradner, S. (1997). RFC 2119; Leiba, B. (2017). RFC 8174. Key words for use in RFCs to Indicate Requirement Levels. IETF.
11. lichess.org rating implementation (scalachess and lila, open source): τ 0.75, starting σ 0.09, σ cap 0.1, 0.21436 periods per day, provisional RD 110.
12. [GG] geoguess.es (2026). Reference implementation and research notes: rating engine, bot calibration and scoring scale studies.