Handicapping
Posted: Wed Jan 04, 2012 12:35 pm
Thought I'd put this up for discussion and see what other peoples thoughts on the concept is. I realise it is likely controversial.
Jim is about to run his third online tourney with an empire handicap system. I am one of those who was intrigued with this, and in principle liked the idea. However, there are a couple of bits I don't think are working so well (No offense meant Jim, and you are probably aware of what I'm saying anyway, it's not like a haven't commented before
).
If I look at some examples of things I think could be improved on:
1. At the start Lyrans were given bonus points even though they had just won at Origins whilst Feds were given a handicap even though the previous online tourney had shown they were the one of the worst performing empires,
2. Orions were going to be left at 1.05 even though no one had taken them because Jim saw them as Feds on steroids. He did change his mind and drop them to 1.00, before they went straight back to 1.05 when they were suddenly very popular.
3. Last touney saw the worst empire not adjusted because they were played by 'poorer' players whilst the best were not adjusted as they were the played by the 'best'.
The above examples highlight 3 different issues.
1. Subjectivity. For a handicap system I like there to be a clear, easily understood and objective scoring system. We all have our ideas on how good certain empires are, but if the data we say we are using is showing something else then we should go with the data not a subjective opinion, everyone should be able to know up front what the 'decision' on the next set of handicaps will be.
2. Large handicap steps. Orions were not popular at 1.05 but at 1.00 they became very popular. A finer grained scale might help with some empires. It might not for some of course, but going from 0 to a 22 point handicap is quite a jump.
3. Good/Bad players. A system based on adjusting empires is always going to struggle with those empires that are either difficult to master (but awesome when you do) or very popular for 'cultural' reasons that therefore result in more of the not so awesome players taking them, even though they are otherwise excellent in the current tournney.
So what was I thinking I would do differently?
Well the most radical is to introduce a player based handicap alongside an empire based one, so the points you get to spend is based on both the empire you choose and your own handicap. Then amend the empire based handicappinig to be clearer and more objective.
The player based handicap is likely controversial, I have no ideas how the likes of Paul and Patrick would see that - they may be playing in a tourney as the 2 best players, but be disadvantaged on that basis.
IMHO it adds 2 useful things (maybe even 3).
One is that it allows, for example, the likes of Feds to be handicapped based on their results whilst the player based handicap mitigates against the best players then finding that an empire they think is powerful gets bonus points (if that is where the empires handicap went).
The second advantage IMO (and I recognise that others may well see this as the opposite) is that it would over time maybe prevent just 2 or 3 people always getting the top spots. Entering tourneys knowing you have no chance of getting into the top half is lilely to discourage a few people over time (Or put off some even bothering to enter). I've read comments about the old SFB tourney scene becoming known as the 'shark tank' or something similar, i.e. a group of very good competitive players chewed up anyone else who dared to enter, and that this may have contributed in some part to the decline of the SFB tourney scene (I may have misundertsood what I was reading, but that is how it came across).
The third possible advantage is that whilst the best players may feel a bit let down that they may not win tourneys because they are so good, they have something IMO much better - a big handicap. A handicap says more about your consistent ability than any tourney.
So what about specifics, well as a first stab I'd change the empire handicap to a simple +/-1 for each game lost/won.
If the Feds played in 12 games and won 4 and lost 8 then they end up with a +4 handicap. That means that for the next tourney they get 450 + 4 = 454 points to spend before player handicap. The scale is pretty smooth, no sudden 22 point jumps that might mean that an empire is either hugely popular or never used. It also allows for anyone to see pretty instantly what it means, especially if the number of games actually played is tracked as well, e.g. if I see a handicap of +4 (12) then I know straight away that they have lost 2/3rds of their games so far.
The player handicap I'd handle more or less the same way, probably multiplied by 2. The multiplier is due to empires could be played in a large number of games due to being picked multi times, but a player can't be choosen multi times! So If Paul played 8 games and won all 8 he would have a -16 handicap. If I played 8 games winning 3 losing 5 I would have a +4 handicap. At some point Paul would hit a point where he would struggle to keep winning and the handicap would stabilise.
This combined player and empire handicap would determine the points to spend. So if we choose the above Feds, Paul would have 438 points to spend, where as I would have 458 points.
I'm mainly intrigued on what opinions are on the player handicap idea. I've only explained the rest as I was thinking of the 2 things working together.
Jim is about to run his third online tourney with an empire handicap system. I am one of those who was intrigued with this, and in principle liked the idea. However, there are a couple of bits I don't think are working so well (No offense meant Jim, and you are probably aware of what I'm saying anyway, it's not like a haven't commented before
). If I look at some examples of things I think could be improved on:
1. At the start Lyrans were given bonus points even though they had just won at Origins whilst Feds were given a handicap even though the previous online tourney had shown they were the one of the worst performing empires,
2. Orions were going to be left at 1.05 even though no one had taken them because Jim saw them as Feds on steroids. He did change his mind and drop them to 1.00, before they went straight back to 1.05 when they were suddenly very popular.
3. Last touney saw the worst empire not adjusted because they were played by 'poorer' players whilst the best were not adjusted as they were the played by the 'best'.
The above examples highlight 3 different issues.
1. Subjectivity. For a handicap system I like there to be a clear, easily understood and objective scoring system. We all have our ideas on how good certain empires are, but if the data we say we are using is showing something else then we should go with the data not a subjective opinion, everyone should be able to know up front what the 'decision' on the next set of handicaps will be.
2. Large handicap steps. Orions were not popular at 1.05 but at 1.00 they became very popular. A finer grained scale might help with some empires. It might not for some of course, but going from 0 to a 22 point handicap is quite a jump.
3. Good/Bad players. A system based on adjusting empires is always going to struggle with those empires that are either difficult to master (but awesome when you do) or very popular for 'cultural' reasons that therefore result in more of the not so awesome players taking them, even though they are otherwise excellent in the current tournney.
So what was I thinking I would do differently?
Well the most radical is to introduce a player based handicap alongside an empire based one, so the points you get to spend is based on both the empire you choose and your own handicap. Then amend the empire based handicappinig to be clearer and more objective.
The player based handicap is likely controversial, I have no ideas how the likes of Paul and Patrick would see that - they may be playing in a tourney as the 2 best players, but be disadvantaged on that basis.
IMHO it adds 2 useful things (maybe even 3).
One is that it allows, for example, the likes of Feds to be handicapped based on their results whilst the player based handicap mitigates against the best players then finding that an empire they think is powerful gets bonus points (if that is where the empires handicap went).
The second advantage IMO (and I recognise that others may well see this as the opposite) is that it would over time maybe prevent just 2 or 3 people always getting the top spots. Entering tourneys knowing you have no chance of getting into the top half is lilely to discourage a few people over time (Or put off some even bothering to enter). I've read comments about the old SFB tourney scene becoming known as the 'shark tank' or something similar, i.e. a group of very good competitive players chewed up anyone else who dared to enter, and that this may have contributed in some part to the decline of the SFB tourney scene (I may have misundertsood what I was reading, but that is how it came across).
The third possible advantage is that whilst the best players may feel a bit let down that they may not win tourneys because they are so good, they have something IMO much better - a big handicap. A handicap says more about your consistent ability than any tourney.
So what about specifics, well as a first stab I'd change the empire handicap to a simple +/-1 for each game lost/won.
If the Feds played in 12 games and won 4 and lost 8 then they end up with a +4 handicap. That means that for the next tourney they get 450 + 4 = 454 points to spend before player handicap. The scale is pretty smooth, no sudden 22 point jumps that might mean that an empire is either hugely popular or never used. It also allows for anyone to see pretty instantly what it means, especially if the number of games actually played is tracked as well, e.g. if I see a handicap of +4 (12) then I know straight away that they have lost 2/3rds of their games so far.
The player handicap I'd handle more or less the same way, probably multiplied by 2. The multiplier is due to empires could be played in a large number of games due to being picked multi times, but a player can't be choosen multi times! So If Paul played 8 games and won all 8 he would have a -16 handicap. If I played 8 games winning 3 losing 5 I would have a +4 handicap. At some point Paul would hit a point where he would struggle to keep winning and the handicap would stabilise.
This combined player and empire handicap would determine the points to spend. So if we choose the above Feds, Paul would have 438 points to spend, where as I would have 458 points.
I'm mainly intrigued on what opinions are on the player handicap idea. I've only explained the rest as I was thinking of the 2 things working together.
