Advent of Code Statistics - 2022
I have really been enjoying the 2022 Advent of Code created by Eric Wastl. For the unfamiliar, this is the combination of an advent calendar and Leetcode programming puzzles mixed with a lot of imaginative storytelling. This is my first year participating and I am clearly not contending for the leaderboard but I have been able to keep up.
I have done a number of programming puzzles but I find that they incentivize quick and dirty solutions to just get to the next one. The forced slow pace of a single two part puzzle per day allows you to take time and not feel forced to create the bare minimum.
Besides the actual puzzles I was intrigued to see that there are Leaderboard and Stats pages which provide puzzle completion times and puzzle completion counts for each day since the calendar began in 2015. I don't have much else to say here but I have had some fun with numbers below.
A consistent colour scheme is kept for years through the plots. This year, 2022, is red and the mean of all of the results is white. Completion of the first part of the puzzle is represented by the solid line and the second part is represented by the dashed line.
Total Completions
- The white 'mean' line represents the average decline in players multiplied by the players who completed the first puzzle in 2022.
- 2022 has had higher retention than average so far though the gap is closing after a difficult day 7 puzzle.
- Day 16 of 2022 featured a devastatingly difficult challenge which is evident by the dramatic drop in completion for the day. Notably, the completion rate did not recover on the 17th as has been seen with some difficult puzzles in previous years.
- Though this 'non-recovery' will likely diminish with time as people complete puzzles in future years.
- Day 16 was followed by an equally difficult day 17 which continued the participation decline. It will be interesting to see if players return to try the remainder of the challenges or lose motivation.
- There was a slight recovery after the day 16 and day 17 gauntlet but the player count never recovered to the expected count.
- The player count reached a sort of plateau after day 16 and the player count dropped on difficult days but recovered the next day when the committed players returned.
Total Completions - Final Days
- This is just a focus on the final half of the graph above.
Normalized Player Retention
- This is a modified version of the player count plots where each year has been normalized to the player count of the first day of the year.
- This year was going really well but began to drop below the average on day 9, mostly due to the difficult day 7 puzzle.
- Note that the first year(2015) represented by the dark green line is abnormally high, likely due to the smaller beginning population and fewer casuals just dropping by for the first puzzle and forgetting about it.
- The difficultly of both the day 16 and day 17 challenges are highlighted here as the normalized participation is the lowest ever for the given advent day. Nearly lower than the minimum ending participation.
- Similar conclusions can be drawn from the normalized player plots but it is a bit more clear that day 16 and day 17 had a larger than average impact on the player count. The normalized player count is fighting for the second lowest for the last 10 days.
Normalized Player Retention - Final Days
- This is just a focus on the final half of the graph above.
Puzzle Part Completion Ratio
- This is the ratio of completion of the first puzzle to completion of the second puzzle. A higher ratio(higher number) will represent a problem with a more difficult second part.
- This shows a significant pattern of difficult second parts on day 7 as well as on days 21-23.
- I suspect the large spike on the 25th is a result of people not wanting to commit too much time on the holiday though it could be a significantly more difficult second part.
- So far in 2022 the second part has been relatively similar to the first part but it seems the common factor / modulo trick of day 11 caught a few people out.
- Notable completion ratios for both day 16 and day 17 which can be explained by their difficulty. Note that the ratio for day 17 should decrease as days pass, the day 16 ratio has dropped significantly since yesterday.
- The ratio stayed relatively low except for another spike on the 22 which is typically the day with the most difficult puzzle.
These next plots include times that are pulled from the top 100 leaderboards. They are only representative of the absolute highest echelon of contendors but it is all the data I can get. The times are still useful indicators of the difficulty of the problems though time consuming problems don't always mean difficult problems.
Completion Time: 2019-2022
- The white line and 'ribbon' show the average times as collected over all years. The centreline is the average completion time of the top 100, the ribbon is the average shortest time to the average longest time.
- The effects of ChatGPT can be seen in the first six days of 2022 though this has eased off a bit as the input parsing has become more complex.
- A clear trend of longer completion times can be seen as the calendar progresses, especially for the second part of the puzzle.
- The short completion time on the 25th supports the idea that the final problems aren't difficult, people just don't get around to the second part.
- If all times were logged then the short average time on the final day could be attributed to only the quick and dedicated players completing the puzzle and bringing the average time down but since only the top 100 players are saved and they are all try-hards this is not a factor.
- The completion times stay within normal ranges except day 16 and day 19. This is quite the feat that is achieved by the designer Eric Wastl. I can't imagine being able to consistently create puzzles that (loosely) fit the story and are of appropriate difficulty.
Completion Time 2022
- This is an expanded view of the completion times for the year 2022. The line is the average completion time of the top 100 while the ribbon shows the quickest and slowest of the top 100.
- The fastest completion times are just outrageous in the first days with automated submission and the use of ChatGPT.
- Day 18 provides some much needed reprieve after the demanding previous days. I am curious if the completion numbers will recover with this simpler day or if previous days have irrevocably turned people away.
- The first puzzle completion time of day 19 quickly dashed hopes of easy puzzles and was quite anomalous, only matched by a few puzzles from 2021.
Completion Time History
- This plot is identical to the previous plot but covers all years of the challenge since 2016.
- It really shows how remarkably consistently there are at least 100 people that can complete at least the first challenge in under 20 minutes.