Welcome! This forum has a treasure trove of great info – Scouters helping Scouters! Just a heads up, though - all content, information, and opinions shared on this forum are those of the author, not the BSA.
BOY SCOUTS OF AMERICA

Scouting Forums

JTE 2021 Fixes

So there’s one HUGE math error in JTE that gives high scores to shrinking units than growing units. Hopefully these can be fixed for 2021 before the scoring guide comes out

Retention is obviously about keeping members year after year until they age out of the program.

Here’s last year’s formula:
"Number of youth members on the most recent charter renewal (A) divided by the number of youth registered at the end of the prior charter year (B) minus any age-outs ©. Total = (A) / (B-C). "

The formula forgets to remove NEW scouts from the formula.

Let’s say your unit starts with 30 Scouts and 5 age out of the program. You retain all 25 Scouts and you gain 5 more in the fall before charter renewal

A = 30, your charter renewal
B = 30, your prior year
C= 5, your age offs
= 30 / 30 - 25 = 120%. Which is clearly impossible

The math problem is that RETENTION needs to remove the number Scouts who were not registered in the prior year from A.

Here’s a better formula for 2021 but it’s still wrong
(Total staying on the 2022 charter (A), minus the total of anyone who signed up anytime during the 2021 charter year(D)) divided by (the total on the 2021 charter (B)minus any who age out of the program ©

30 - 5 / 30 - 5 = 100% So that’s honest for this unit

What about a unit that ages off zero Scouts and we start to look at losses
2021 charter: 30 Scouts
2022 charter: 30 Scouts
total lost: 20 Scouts
total aged off = 0
total recruited: 20 Scouts

30 - 20 / 30 - 0 = 10 / 30 = 33% retention. That is correct for the unit

What if we combine both?

2021 charter: 30 Scouts
2022 charter: 25 Scouts
total lost: 20 Scouts
total aged off = 5
total recruited: 20 Scouts

25 - 20 / 30 - 5 = 20% retention. But we kept 16%.

There’s no simple math formula that actually shows retention for all units and it’s much simpler than that. We want to retain Scouts gained over the course of the year and end up with a bigger unit in the end. Units need to retain and recruit.

Drop the retention standard. Keep the recruitment standard as is. It makes a lot more sense, units can find whatever way to maintain their size whatever works. If they shrink through bad efforts overall and drop below 40 they have to start growing again or retain better.

I wanted to preface this with the comment that, despite being an engineer, I’m not a huge proponent of rigid metrics for measuring “success”. Success take a lot of different forms, some of which may not be measured by data we can measure as a unit (e.g. long-term positive impact on a scout’s life due to an even short-term involvement in scouting, or a scout who was present in one unit for a short time who continues in another unit after moving due to positive experiences with the first unit).

I guess from a certain point of view, it depends on what exactly is the desired statistic as to what the appropriate math would be. I tend to agree that there’s an issue with the calculation if the intent is to count the fraction of scouts who are already in the specific unit’s program who did not leave (excluding those who aged-out of the program). In that case, it seems like the math would be:

     (S' + Delta S)/S' = (B-C-D)/(B-C)

where “D” is the number of specific youth who left the unit between recharter date B and the date of submittal for JTE, and B and C are as previously defined. I might call “S’ = B - C” the “age-corrected membership”. The number of scouts chartered in the succeeding year doesn’t come into the calculation. This method requires units/evaluators to cross-check actual names, not just count noses, but I believe actually computes how many members a unit retained, excluding those who could not be retained due to age restrictions.

On the other hand, if the question is simply whether or not any particular unit is shrinking or growing net of all arrivals and departures, then the math being done now makes more sense. I still think that there is an issue with how it’s calculated for that purpose. It seems like one calculation might be more like:

    (S + Delta S)/S = A/B = fractional change in membership size

which is a straight ratio of how many you have “now” vs how many youth the unit had last year. It’s a measure (in a sense) of whether or not a unit is exceeding, matching, or not reaching replacement rate for a given unit size. This metric could unduly penalize (or benefit) units which are weighted toward a particular age cohort, depending on where that cohort falls in the spectrum of ages served. At the same time, it could benefit units which have high recruitment during the year. It kinda conflates recruitment efforts, retention, and losses due to age-out.

Another approach might seek to explicitly take into account the recruitment that occurred during the period between measurements A and B. For example:

        (S' + Delta S)/S' = (B-C+R-D)/(B-C)

where “R” is the number of scouts registered during the time between recharter submittal B and recharter submittal A, all other terms as defined before. If R exceeds D, then a unit is “growing”. If R is smaller than D, then the unit is “shrinking”. As before, this omits the effect of scouts aging-out, which can lead to “real” unit shrinkage. This effect is masked by the metric as proposed, since, in order to maintain a given total membership, the unit actually needs to achieve:

          A = B - C - D + R

To grow in real-space requires R > (C + D). Shrinkage would occur any time that R < (C + D).

I suspect there are a million ways to try to metricize unit membership, each with its own set of drawbacks and advantages. I don’t know that it’s a reason to drop the idea of measuring something like retention or replacement rate for units. It just requires deciding what actually is going to be measured and what value that information brings to understanding the unit’s practices (either for the unit leadership, unit parents, district/council/national staff, or some mixture of the above).

On a separate note, it looks like you have a typo in the first part of your math. I think you meant:

to say “30 / (30 - 5) = 30/25 = 120%”, based on the formula you originally posted.

I’m also not clear on how you’re arriving at the 16% retention number in your last calculation. For the following, I’m assuming your 20 “lost” includes both drops and age-outs.

  1. If you start with 30 and lose 20, then that’s (30-20)/30 = 33% retention, neglecting the effects of age-outs.
  2. If you are discounting the 5 age-outs from your losses, then your retention percentage would be [30 - (20-5)]/30 = 50% retention. This assumes retention is relative to the initial size of the unit, and considers only the number of losses due to anything other than aging-out.
  3. If you assume retention is relative to the maximum potential retention, you have (30-20)/(30-5) = 40% retention. This considers the total number of losses (including age-outs) in relation to the total number that the unit could possibly have kept.
  4. If you assume that the aging-out number should be eliminated from both the number lost and from the maximum potential retention (since they couldn’t have stayed anyway), then the math becomes [30 - (20-5)]/(30 - 5) = 60% retention.

Maybe I’m missing something obvious. Are the 5 “aged-out” not included in the 20 “lost”? I think I can make the math work if I assume that:

    (30 - 20 - 5)/(30) = 16.7%

although that penalizes the unit for “failure” to retain scouts that they could not retain. It seems like that calculation should actually be:

   (30 - 20 - 5)/(30 - 5) = 20%

since the unit couldn’t have retained those 5 scouts anyway. Although this numerically matches the percentage calculated by (A - D)/(B - C), it’s actually calculating (B - E - C)/(B - C), or the “retainable scouts” minus losses divided by the total number of scouts “retainable”.

3 Likes

The formulas used for JTE are a balancing act between what “they” want to measure and what the folks filling out the form are able to reliably do. The number of last year’s scouts, the number of age outs, and the number of this year’s scouts are all easily found and entered.

You are correct, the formula as written does not measure only retention of last year’s scouts. The formula allows new members who recharter to benefit the unit, while not penalizing the unit for new members who drop.

I agree the formula does not match the description of what it is measuring. But that doesn’t make it a bad measure. The key would be to see what this measure shows about all the units in the council. Is it a good diagnostic tool to help in figuring out which units are providing better scouting to their members?

2 Likes

@RickHillenbrand - something to consider.

1 Like

On the Excel spreadsheet on which this is calculated, it allows you to enter the numerator. I only include Scouts who are rechartering who were on the roster at the start of the year. I have to do a manual count in order to get that number. Since it makes no sense to include all teh Scouts who are rechartering, I presume that’s why this field requires data entry rather than being automatically calculated.

However, I agree that the explanation of how this is calculated should be more explicit. Based on what I’ve seen people posting in online about JTE, I think there are many who just enter the number of Scouts on their recharter in this field. So, if a troop starts the year with 10 Scouts, all of them are dropped, the troop recruits 15 Scoutsandfour of thos are dropped, they have 11 Scouts and are calculating the retention rate at 110%, when it is actually 0%.

It depends on your definition of “is”. :slight_smile:

If National doesn’t care about an individual, but only enrolment numbers (paid Scouts), then who recharters is fungible and as long as the number goes from 10 to 11. Thus, 110%. This isn’t good for long term health, but from a total number of Scouts in the country, it can be seen as growth. It is sort of like same store sales vs. total sales.

1 Like

@Matt.Johnson - If National actually wants it calculated as simply a raw increase in the number of Scouts, why do they call it “retention”? There is another JTE objective for growth in youth membership. I think the retention JTE objective is widely misunderstood, and I think the biggest reason for that is the lack of clarity in the explanation as well as on the Excel spreasheet that gets distributed.

In 2019, my pack started the year with 10 Scouts and ended with 19. Two of the 10 Scouts on our roster on January 1, 2019, graduated to a troop, One transferred to another pack during the year. Four were dropped at recharter in December 2019. Our retention percentage was 37.5%. We kept three of the eight Scouts who did not age out. It’s hard for me to see how that could be the reality and result in a retention percentage of over 100%.

Our growth percentage was 90%. That make a lot of sense.

If we were to calculate retention based on total available youth, we would get a different result:

Scouts on January 1, 2019 - 10
New Scouts joining - 35
Scouts graduating - 2
Transfers out - 2
Scouts dropped - 22
Scouts on December 31, 2019 - 19

So, total available Scouts were 10 + 35 - 2 = 43. We kept 19 of them or 44.2%. However, theway they tell you to calculate retention ingore Scouts who join and then do not reregister for a second year. Neverthless, this would be a decent metric to replace what they currently have. The problem is that the way recruiting was done in 2019, the council got Scouts to register online, telling families that they could give Scouting a try by just paying a $15 registration fee for a few months. One of those 35 Scouts we recruited never even showed up, and her mother never responded to emails, voicemails or texts. In many ways, the higher fees eliminate the time and effort required to deal with these folks who often come to two meeting and then vanish.

I agree with your analysis.

The whole thing works if you change were you subtract from.

30 scouts on charter minus those aging out leaves 25. Now divide the 25 by whatever number recharter and you have retention. To me the real problem is a combination of transfers. If they transfer out of council they will never disappear from my roster. And if they transfer within council 60% of the time they will not be on my roster at recharter.

Of course that doesn’t address any of the whys. Fortunately all of my transfers have been because the scout moved. Most out of state, the others a couple hours away. And all of them did complete the transfer.

1 Like

@KirkWood - When you say:

do you mean all the Scouts on your recharter? If so, A unit starting with 30 Scouts, having five age out and the other 25 it began the year with quit but recruiting 30 new Scouts has a 125% retention rate, but the unit didn’t really retain anyone.

If you look, the words are “Re-register x% of eligible members.” Thus if your unit has 30 scouts showing on the roster, with 5 aging out - you have 25 eligible members to renew. How you got to that 25 is immaterial to the question.

@KirkWood - I don’t understand what you’re trying to say. Thye denominator in the calculation is pretty clear, particularly since the Excel spreadsheet calculates it for you. Are you saying that if I have 25 eligible members, and they all quit, it is nevertheless possible that my retention percentage can be greater than 100%? Os that it can be greater than 100% in any circumstance?

“Guys” … I’m struggling just trying to get approval from the owner of the JTE process to simply update the 2020 unit scorecards to 2021. I don’t own this process, but in the name of unit service I am willing to have the commissioner technology team make the changes to put the 2021 unit scorecards and worksheets in place. (We have been ready to disseminate since before January.)

Here is the issue with introducing the above discussion about the way retention is calculated: It has taken me a month of baby steps to get where I am today in this process. Having the owner of JTE consider and make a decision on this change might add weeks to the process of getting the products completed. The calculation of retention (and automating it) has been an annual issue.

2 Likes

@RickHillenbrand - I don’t have an ounce of doubt that you’re doing your best.

Our beloved organization simply has too many fiefdoms and too many silos. That’s why there are inconsistencies, ambiguities and adverse outcomes for the Scouts in the information that gets pushed out to unit-level volunteers.who need to do their best to administer the program as intended.

You frequently comment here and face the music, trying to apply first aid, but the BSA needs some preventive and long-tem care to resolve these issues.

1 Like

I hear you and agree… and can empathize as a Committee Chair of large Cub Scout pack. Only this week my council decided to throw back my charter (Turned in more than a month ago) because they failed to have us execute the latest version of the Annual Charter Agreement. (We used the older one they included in the recharter package.) Years ago, executing the Annual Charter Agreement was an expectation of every DE… then slowly we “allowed” commissioners to present the Annual Charter Agreement to the chartered org on behalf of the council. These days it is the unit’s job to represent the council and have “the conversation” with the chartered org.

2 Likes

The number in your denominator is wrong, which is where the confusion lies. The total number registered at the end of the charter year INCLUDES the age outs, they don’t come off the roster until after the recharter, which is why you have to subtract them in the equation. In your example the math is 30/(35-5)= 100%

1 Like

This - please stop making things hard. And for the love of all things good so NOT take any software (including spreadsheets from BSA or other sources) as a source of truth.

I write software for a living and ALL SOFTWARE HAS BUGS. We need to not allow software to define what “should” be and we need to teach our youth critical thinking which includes don’t trust software.

The stuff is great - but IT HAS BUGS

If you really want to make a difference lets get a re-organization of all of this. Technology should be a service to the others. And honestly - it is time to follow the world in providing more of a platform than locking people into using proprietary software.

Here is an idea - a web page that queries the user for the input and outputs an appropriate PDF ready for either direct submission or if you just can’t release the past - print and sign. The source code could be hosted in an open repository (pick a platform) and anyone with the skill set can help out.

One last comment - just saw this morning they have updated them. The spreadsheets that would calculate this are gone. The PDF is gone. We now have a spreadsheet that will (presumably) print correctly.

#ThisIsIncrediblyBACKWARDS

@KirkWood - revised forms (PDFs) have been created, and should be posted shortly. The Excel files that were posted did NOT do calculations.

The “workbooks” for 2021 have also been created and in final review with the National JTE Committee. The “retention calculation” discussion here on the forum has made them do some re-thinking.

2 Likes