Though Hudl's Football Assist offering has largely had positive feedback, we were regularly hearing negative feedback regarding tagging offensive formations. Formations are a tricky breakdown component to get right, as each team has their own methodology and mental model of offensive formations. Currently, analysts tag a formation, which is matched to a database of known formations (which do not include backfield or strength), and the default formation is returned to the coach. The coach is them prompted to name the returned diagram, and that name is used when that formation is tagged throughout the season.
At the end of Football season, Assist Operations conducted multiple surveys to determine the success of Assist, which informed prioritizing offensive formation improvements. When asked which columns coaches would like to see added, Strength and Backfield were the most frequently requested. Therefore, we knew entering the project that supporting Strength and Backfield would be a key part of our improvements.
In addition to the known issues on the coaches end, Assist Analysts did not enjoy tagging formations. Currently, tagging formations involves clicking specific points in the video, which auto-populates a formation diagram, while the majority of Analyst workflow uses the keyboard.
- Determine how coaches determine formation names.
- Understand what coaches are communicating via formation names.
- Understand how formation names vary between members of the same staff.
- Determine if there are prevalent common themes in formation naming amongst teams.
- Understand why formations are important to coaches, and what information is dependent upon formations.
- Determine how coaches use formation data for their own team, and how they use it with opponents.
- Understand how coaches organize formations.
- Determine if coaches would be willing to alter how they use formations to get the most out of Hudl Assist.
Most coaches need formation, backfield, and strength broken out individually in order for formations tagged by Assist to be useful, and are willing to do the work or adapt their breakdown techniques to name each separately.
- Coaches think about strength, backfield, and formation when thinking of “formation,” even if they don't break those components out explicitly.
- Formation naming is highly variable, and how the coach communicates those three data points varies greatly -- but most coaches are communicating strength, backfield, and formation, and the mental model of a singular formation contains that information.
1. Internal and External User Interviews
Neither myself nor the other designer on the project have played or coached Football, and neither of us felt we had enough in-depth knowledge around formations to speak confidently about them to our end users. Therefore, we started our research by interviewing three internal Football coaches. This allowed us to evaluate and iterate on our questions, and to gain a greater understanding of formations to ensure we could effectively lead external user interviews.
Our external interviews were largely informed by the research objectives, however, we wanted to conduct external user interviews before sending a survey, as we wanted to ensure that the survey confirmed any assumptions we validated during the comparatively small set of user interviews.
The goal of the survey is to capture a large amount of data regarding coach's mental model of formations, and how they install formations on their team. While user interviews help us understand the issue in depth, how teams use formations is highly variable. We wanted a large amount of data to back up our assumptions and key takeaways from user interviews, as interviewing enough users to feel confident about established tendencies among our tens of thousands of users was outside of scope.
3. Data Analysis
As Hudl supports 98% of High School Football teams in the country, we have a large amount of existing data that we can manipulate and use to inform our assumptions and conclusions. I ran multiple queries to better understand some trends within the existing data, and help inform our research objectives.
- I compared the amount of Assist formation names that included an indicator of strength (left/L/LT, right/R/RT, Strong, Weak) to the number of non-Assist formation names that included an indicator of strength over the same time period. As Assist users were explicitly told not to include Strength in their formation names, I was seeking to determine how amenable coaches are to changing their workflows around formation naming.
- Of the 23,217 Assist Formations named between 1 Sept 2017 and 15 Sept 2017, 7,545 or 32.5% had a strength indicator.
- Of the 563,359 non-Assist Offensive Formation column data values filled out between 1 Sept 2017 and 15 Sept 2017, 263719, or 46.8% had a strength indicator.
- Number of formation names that included common strength indicators (left/L/LT, right/R/RT, Strong, Weak), to determine the percentage of coaches that consider strength a part of the formation name, as opposed to a separate column (as Hudl has columns available for offensive formation, strength, and backfield).
- As mentioned above, 46.8% of formations included strength indicators.
- Number of coaches who use the offensive formation column alone, offensive formation + strength column, offensive formation + backfield column, or all three.
This research is ongoing -- the survey is still being conducted, and further user interviews and prototype testing will be conducted.