DOWN THE LINE GRADING ALGORITHM
Introduction and Guiding Principles
When we at MCMS designed our algorithm we did so with the following key objectives in mind:
The MCMS algorithm has been designed to identify the likely range that a shooter's score will fall between the next time they shoot an event. It does not indicate the likelihood of them winning the event, simply the likely score they will shoot in the event. This means that only event scores are considered (not shoot-off scores).
To ensure the statistical relevance and comparability of scores and maximise the scores available to be reviewed all event scores will be recorded in legs of no more than 30 targets (35 targets for sporting). This means a 50T event will be recorded as two legs of 25T's thus providing two reference scores for the algorithm (the MCMS user interface now enforces this when setting up an event). To ensure comparability any event legs with less than 20T's will be ignored by the algorithm.
To further ensure the relevance and comparability of scores the algorithm will separately review and calculate a grade for each skill within a discipline. The highest skill grade within a discipline will then be used for that discipline when shooting an event. Disciplines include: Trap, Double Rise, Skeet, Sporting, Ball, Tower, ISSF Trap and ISSF Skeet. Skills include double barrel, single barrel and point score within the Trap discipline and 12G, 20G, 28G and 410 within the Skeet discipline. Skills are reviewed separately as the difficulty to achieve a score is different for each.
This means that if I am A grade based on my double barrel scores, AA grade based on my point score scores and B grade based on my single barrel scores my "trap" grade will be AA and I will shoot in AA for all trap events (DB, SB and PS) within a shoot program.
It is the score for each skill that is used when the algorithm runs. For most skills the score equals broken targets but in skills where this is not the case (eg point score) it is the score that is used by the algorithm. This is because the score is more indicative of ability in relation to that skill. Using the score also minimises the data needed to be captured and entered by the volunteer club administrator.
So that as many scores as possible can be used within the algorithm calculation we are using scores even where they have been shot under non-standard conditions such as continental for trap or doubles for skeet. To maximise comparability we uplift the scores for each skill by a specified difficulty factor before including them in the algorithm. The percentage uplift of the difficulty factors will be based on data collected within the MCMS system and will be reviewed by the Rules Supervisors from time to time.
As it is scores and only scores that truly indicate the likelihood of what a shooter will shoot at their next event, it is only scores that are used by the algorithm. This means there are no manual intervention or discretionary aspects to the algorithm. Hence a shooter cannot choose to stay in a grade nor can a handicapper choose to exclude a particular score.
The best indication of a shooter's next score is their most recent scores. Thus the algorithm will complete a grade review each time a score is received. The resulting output will stand on its own with no reference to previous calculations. This means that a shooter could move from AA to C grade if they shoot sufficient scores at a lower level (but this will take a significant period of time as detailed below).
Whilst a shooters grade must be able to move freely over time based on their scores, that movement should be easier when going up and harder when going down. To achieve this the algorithm will have both a long term ("Anchor") and a short term ("Recent") calculation for each skill. The higher of the two will become the shooter's grade for that skill. This means a form slump may take the Recent grade down but until the Anchor also moves down the shooter's grade will not change. Alternatively finding form will quickly increase the Recent grade above the Anchor grade which will lead to an immediate uplift in the shooter's grade.
Reconfirming the Key Principles
A discipline (eg Trap) is made up of skills (eg DB, SB, PS) and those skills are shot under difficulty factors (eg continental).
When calculating a skill's grade the algorithm will identify an anchor (ie long term) grade and a recent (ie short term) grade and the higher of those two grades will become the shooter's skill grade.
A shooter's highest skill grade for a discipline will become their grade for that discipline when competing (regardless of the event being shot). So if your DB grade is AA, your SB grade is A and your PS grade is A then your "trap" grade will be AA and so in a trap event you will compete as a AA shooter whether you are shooting DB, SB or PS.
Non-Indicative Scores
The algorithm will automatically exclude non-indicative scores. To facilitate this calculation the algorithm will use the cut-off percentages for each of the standard grades plus cut-off's for a notional D and E grade.
When calculating a shooter's anchor grade for a skill all scores below the E grade percentage cut-off will be excluded as they are "below the floor" and therefore non-indicative of a shooter's long term ability.
When calculating a shooter's recent grade for a skill, all scores below the grade, that is two below the shooters anchor grade for that skill will be excluded as they are "below the floor" and are therefore non-indicative of a shooter's short term ability. So if a shooters DB anchor grade is A grade then all scores shot below the C grade percentage cut-off will be excluded as non-indicative.
"Anchor" Grade Calculation
The algorithm first identifies all scores shot by the shooter for the skill being reviewed that are above the E grade "floor" percentage for that skill.
Once those scores are identified they are ordered by most recent first and then by highest percentage within scores shot on the same day.
The algorithm then identifies the most recent 50 scores within the scores identified and ignores the rest. It then identifies the 25 scores with the highest percentage (after uplift for difficulty where appropriate) and then averages them. The result of that averaging process is compared to the grade cut-offs for the skill to determine the skills anchor grade.
Where the shooter does not have 50 scores that meet the requirements the number of scores included in the "best of" calculation will be pro-rata'd. eg if 29 scores are available then only the best 15 scores would be averaged to compare to the grade cut-offs for the skill to determine the skills anchor grade.
Once a shooter has reached 50 available scores any new score received will displace the oldest of the 50 previously available scores.
To allow for the fluctuations in scores for new shooters the "floor" will not kick in to exclude scores as "non-indicative" for a particular skill until the shooter has shot a minimum number of scores for that skill (of any percentage).
"Recent" Grade Calculation
The algorithm first identifies all scores shot by the shooter for the skill being reviewed that are above the "floor" percentage identified by reference to that skill's already calculated "anchor" grade. That is, if the shooters SB anchor grade is AA then the floor for their SB recent grade calculation will be the B grade percentage cut-off.
Once those scores are identified they are ordered by most recent first and then by highest percentage within scores shot on the same day.
The algorithm then identifies the most recent 20 scores within the scores identified and ignores the rest. It then identifies the 10 scores with the highest percentage (after uplift for difficulty where appropriate) and averages them. The result of that averaging process is compared to the grade cut-offs for that skill to determine the skills recent grade.
Where the shooter does not have 20 scores that meet the requirements the number of scores included in the "best of" calculation will be pro-rata'd. eg if 17 scores are available then only the best 9 scores would be averaged to compare to the grade cut-offs for the skill to determine the skills recent grade.
Once a shooter has reached 20 available scores any new score received will displace the oldest of the 20 previously available scores. However should the new score cause a change in the shooter's anchor grade this may cause a change in the available scores due the floor percentage changing.
To allow for the fluctuations in scores for new shooters the recent calculation will not kick in until the floor for the shooter's anchor grade has triggered and the shooter has shot a minimum number of scores above the floor determined by their anchor grade..
Review Trigger
Every score that is entered for a shooter will trigger a review of that shooter's grade for the relevant skill and its associated discipline. If this review results in a discipline grade change it will be immediately reflected in a shooter's "membership" grade record.
However, any grade change will not impact the grade that a shooter competes in for an "in progress" shoot program unless the club administrator manually changes their grade in line with any governing body rules in place at the time.
Current ACTA Grade Percentage Cut-Offs
The Rules Supervisors, with the support of the ACTA Executive, have asked MCMS to set the grade percentage cut-offs to achieve an approximate split of 20% of shooters in AA Grade, 30% in A Grade, 30% in B Grade and 20% in C Grade. They further requested that this should be done in the context of only those shooters who compete regularly (in order to ensure the significant proportion of ACTA members who only shoot irregularly do not overtly affect the distribution of competitors).
The current cut-offs running in the algorithm can be found in the latest version of the ACTA Shooting Rules Book which is available on the ACTA website (under Referees & Rules). The cut-offs will continue to be regularly reviewed in light of captured scores but it is expected that cut-offs will be updated no more than once per year in consultation with the Rules Supervisors and the ACTA Executive.
Where Can I view my calculated grade?
Your grade calculation is available within the MCMS Shooter's app in the "ME" section. You will need to register with the app and link your membership to see your grade and score information.
Explanations for the icons shown within the user interface can be found here.
Frequently Asked Questions
Will I still only be regraded after 200 targets?
No. Every score recorded for a shooter that meets the parameters of the algorithm will trigger a review of the shooter’s grade at the time it is received. This will ensure the grade a shooter nominates in for a program is reflective of their ability at that point in time.
Can I request to stay in a grade?
No. There are no discretionary elements (from the shooter or the handicapper) within the algorithm. Only your scores will influence your grade.
Will ACTA's “AA Elite” remain?
We understand that as a recognition award it will remain, but this is a question for the ACTA. Within the MCMS grading algorithm your grade (and handicap) will be based on the scores you shoot so whether you reached a particular grade (or handicap) in the past is irrelevant to what your current grade (and handicap) is.
Can I move more than one grade?
Yes. After each score is entered into the MCMS system your grade will be reviewed using your most recent scores. Whatever that review determines to be your grade then that will be your grade with no reference to any previous reviews. However, with the currently specified parameters (best 25 of last 50 and best 10 of last 20 with exclusions for non-indicative scores) it would be almost mathematically impossible for one day of shooting to cause a review to a grade two above or two below the grade you were in when starting the day.
Why is there both a Recent and an Anchor calculation?
In line with the guiding principles of the MCMS algorithm a shooter's grade should be able to move in line with their ability, but in a stable manner. This means all scores that are not non-indicative (ie not below E grade) should be considered over the long term to allow a shooter to move down. On the flip side if the shooter has proven they can shoot at a certain level it should take a significant run of poor scores before they can move down. This is what the 50 score Anchor is for.
However, as a shooter can very quickly “strike form” there is a need to be able to reflect this in their calculated grade more quickly and also to ensure that once the shooter reaches a higher grade the scores regarded as “non-indictive” are also higher. This is the purpose of the 20 score Recent calculation.
The Anchor and the Recent work together to create a stable grade for a shooter over the long term that can eventually fall with consistent poor shooting but will rise quickly in response to good shooting.
Introduction and Guiding Principles
When we at MCMS designed our algorithm we did so with the following key objectives in mind:
- to minimise the data needed to allow it to calculate, thus reducing the administrative burden on club volunteers
- to maximise the acceptance of the outputs (ie grades) calculated by the algorithm by using a data driven approach to inputs and parameter setting
- to facilitate the setting of cut-offs for grades based on maximising competition rather than on perceived ability (ie provide the ability to rebalance the number in each grade to maximise the competition within each grade)
The MCMS algorithm has been designed to identify the likely range that a shooter's score will fall between the next time they shoot an event. It does not indicate the likelihood of them winning the event, simply the likely score they will shoot in the event. This means that only event scores are considered (not shoot-off scores).
To ensure the statistical relevance and comparability of scores and maximise the scores available to be reviewed all event scores will be recorded in legs of no more than 30 targets (35 targets for sporting). This means a 50T event will be recorded as two legs of 25T's thus providing two reference scores for the algorithm (the MCMS user interface now enforces this when setting up an event). To ensure comparability any event legs with less than 20T's will be ignored by the algorithm.
To further ensure the relevance and comparability of scores the algorithm will separately review and calculate a grade for each skill within a discipline. The highest skill grade within a discipline will then be used for that discipline when shooting an event. Disciplines include: Trap, Double Rise, Skeet, Sporting, Ball, Tower, ISSF Trap and ISSF Skeet. Skills include double barrel, single barrel and point score within the Trap discipline and 12G, 20G, 28G and 410 within the Skeet discipline. Skills are reviewed separately as the difficulty to achieve a score is different for each.
This means that if I am A grade based on my double barrel scores, AA grade based on my point score scores and B grade based on my single barrel scores my "trap" grade will be AA and I will shoot in AA for all trap events (DB, SB and PS) within a shoot program.
It is the score for each skill that is used when the algorithm runs. For most skills the score equals broken targets but in skills where this is not the case (eg point score) it is the score that is used by the algorithm. This is because the score is more indicative of ability in relation to that skill. Using the score also minimises the data needed to be captured and entered by the volunteer club administrator.
So that as many scores as possible can be used within the algorithm calculation we are using scores even where they have been shot under non-standard conditions such as continental for trap or doubles for skeet. To maximise comparability we uplift the scores for each skill by a specified difficulty factor before including them in the algorithm. The percentage uplift of the difficulty factors will be based on data collected within the MCMS system and will be reviewed by the Rules Supervisors from time to time.
As it is scores and only scores that truly indicate the likelihood of what a shooter will shoot at their next event, it is only scores that are used by the algorithm. This means there are no manual intervention or discretionary aspects to the algorithm. Hence a shooter cannot choose to stay in a grade nor can a handicapper choose to exclude a particular score.
The best indication of a shooter's next score is their most recent scores. Thus the algorithm will complete a grade review each time a score is received. The resulting output will stand on its own with no reference to previous calculations. This means that a shooter could move from AA to C grade if they shoot sufficient scores at a lower level (but this will take a significant period of time as detailed below).
Whilst a shooters grade must be able to move freely over time based on their scores, that movement should be easier when going up and harder when going down. To achieve this the algorithm will have both a long term ("Anchor") and a short term ("Recent") calculation for each skill. The higher of the two will become the shooter's grade for that skill. This means a form slump may take the Recent grade down but until the Anchor also moves down the shooter's grade will not change. Alternatively finding form will quickly increase the Recent grade above the Anchor grade which will lead to an immediate uplift in the shooter's grade.
Reconfirming the Key Principles
A discipline (eg Trap) is made up of skills (eg DB, SB, PS) and those skills are shot under difficulty factors (eg continental).
When calculating a skill's grade the algorithm will identify an anchor (ie long term) grade and a recent (ie short term) grade and the higher of those two grades will become the shooter's skill grade.
A shooter's highest skill grade for a discipline will become their grade for that discipline when competing (regardless of the event being shot). So if your DB grade is AA, your SB grade is A and your PS grade is A then your "trap" grade will be AA and so in a trap event you will compete as a AA shooter whether you are shooting DB, SB or PS.
Non-Indicative Scores
The algorithm will automatically exclude non-indicative scores. To facilitate this calculation the algorithm will use the cut-off percentages for each of the standard grades plus cut-off's for a notional D and E grade.
When calculating a shooter's anchor grade for a skill all scores below the E grade percentage cut-off will be excluded as they are "below the floor" and therefore non-indicative of a shooter's long term ability.
When calculating a shooter's recent grade for a skill, all scores below the grade, that is two below the shooters anchor grade for that skill will be excluded as they are "below the floor" and are therefore non-indicative of a shooter's short term ability. So if a shooters DB anchor grade is A grade then all scores shot below the C grade percentage cut-off will be excluded as non-indicative.
"Anchor" Grade Calculation
The algorithm first identifies all scores shot by the shooter for the skill being reviewed that are above the E grade "floor" percentage for that skill.
Once those scores are identified they are ordered by most recent first and then by highest percentage within scores shot on the same day.
The algorithm then identifies the most recent 50 scores within the scores identified and ignores the rest. It then identifies the 25 scores with the highest percentage (after uplift for difficulty where appropriate) and then averages them. The result of that averaging process is compared to the grade cut-offs for the skill to determine the skills anchor grade.
Where the shooter does not have 50 scores that meet the requirements the number of scores included in the "best of" calculation will be pro-rata'd. eg if 29 scores are available then only the best 15 scores would be averaged to compare to the grade cut-offs for the skill to determine the skills anchor grade.
Once a shooter has reached 50 available scores any new score received will displace the oldest of the 50 previously available scores.
To allow for the fluctuations in scores for new shooters the "floor" will not kick in to exclude scores as "non-indicative" for a particular skill until the shooter has shot a minimum number of scores for that skill (of any percentage).
"Recent" Grade Calculation
The algorithm first identifies all scores shot by the shooter for the skill being reviewed that are above the "floor" percentage identified by reference to that skill's already calculated "anchor" grade. That is, if the shooters SB anchor grade is AA then the floor for their SB recent grade calculation will be the B grade percentage cut-off.
Once those scores are identified they are ordered by most recent first and then by highest percentage within scores shot on the same day.
The algorithm then identifies the most recent 20 scores within the scores identified and ignores the rest. It then identifies the 10 scores with the highest percentage (after uplift for difficulty where appropriate) and averages them. The result of that averaging process is compared to the grade cut-offs for that skill to determine the skills recent grade.
Where the shooter does not have 20 scores that meet the requirements the number of scores included in the "best of" calculation will be pro-rata'd. eg if 17 scores are available then only the best 9 scores would be averaged to compare to the grade cut-offs for the skill to determine the skills recent grade.
Once a shooter has reached 20 available scores any new score received will displace the oldest of the 20 previously available scores. However should the new score cause a change in the shooter's anchor grade this may cause a change in the available scores due the floor percentage changing.
To allow for the fluctuations in scores for new shooters the recent calculation will not kick in until the floor for the shooter's anchor grade has triggered and the shooter has shot a minimum number of scores above the floor determined by their anchor grade..
Review Trigger
Every score that is entered for a shooter will trigger a review of that shooter's grade for the relevant skill and its associated discipline. If this review results in a discipline grade change it will be immediately reflected in a shooter's "membership" grade record.
However, any grade change will not impact the grade that a shooter competes in for an "in progress" shoot program unless the club administrator manually changes their grade in line with any governing body rules in place at the time.
Current ACTA Grade Percentage Cut-Offs
The Rules Supervisors, with the support of the ACTA Executive, have asked MCMS to set the grade percentage cut-offs to achieve an approximate split of 20% of shooters in AA Grade, 30% in A Grade, 30% in B Grade and 20% in C Grade. They further requested that this should be done in the context of only those shooters who compete regularly (in order to ensure the significant proportion of ACTA members who only shoot irregularly do not overtly affect the distribution of competitors).
The current cut-offs running in the algorithm can be found in the latest version of the ACTA Shooting Rules Book which is available on the ACTA website (under Referees & Rules). The cut-offs will continue to be regularly reviewed in light of captured scores but it is expected that cut-offs will be updated no more than once per year in consultation with the Rules Supervisors and the ACTA Executive.
Where Can I view my calculated grade?
Your grade calculation is available within the MCMS Shooter's app in the "ME" section. You will need to register with the app and link your membership to see your grade and score information.
Explanations for the icons shown within the user interface can be found here.
Frequently Asked Questions
Will I still only be regraded after 200 targets?
No. Every score recorded for a shooter that meets the parameters of the algorithm will trigger a review of the shooter’s grade at the time it is received. This will ensure the grade a shooter nominates in for a program is reflective of their ability at that point in time.
Can I request to stay in a grade?
No. There are no discretionary elements (from the shooter or the handicapper) within the algorithm. Only your scores will influence your grade.
Will ACTA's “AA Elite” remain?
We understand that as a recognition award it will remain, but this is a question for the ACTA. Within the MCMS grading algorithm your grade (and handicap) will be based on the scores you shoot so whether you reached a particular grade (or handicap) in the past is irrelevant to what your current grade (and handicap) is.
Can I move more than one grade?
Yes. After each score is entered into the MCMS system your grade will be reviewed using your most recent scores. Whatever that review determines to be your grade then that will be your grade with no reference to any previous reviews. However, with the currently specified parameters (best 25 of last 50 and best 10 of last 20 with exclusions for non-indicative scores) it would be almost mathematically impossible for one day of shooting to cause a review to a grade two above or two below the grade you were in when starting the day.
Why is there both a Recent and an Anchor calculation?
In line with the guiding principles of the MCMS algorithm a shooter's grade should be able to move in line with their ability, but in a stable manner. This means all scores that are not non-indicative (ie not below E grade) should be considered over the long term to allow a shooter to move down. On the flip side if the shooter has proven they can shoot at a certain level it should take a significant run of poor scores before they can move down. This is what the 50 score Anchor is for.
However, as a shooter can very quickly “strike form” there is a need to be able to reflect this in their calculated grade more quickly and also to ensure that once the shooter reaches a higher grade the scores regarded as “non-indictive” are also higher. This is the purpose of the 20 score Recent calculation.
The Anchor and the Recent work together to create a stable grade for a shooter over the long term that can eventually fall with consistent poor shooting but will rise quickly in response to good shooting.
