Spellbook · projects
Wizard Card · about
Arithmancy · bend the gridOwl Post · nowRecords · resume
Avyaya's Machine

Spellbook · projects

×
p. 5

spell no. 003

Exercise Form Checker

upload a workout video and get your joints tracked frame by frame, plus rule-based feedback on your angles, alignment, and range of motion. the rules are mine and haven't been checked against a coach

askcould a pose model actually tell you what's wrong with your squat, not just that you're squatting

ingredients
computer visionYOLOv8OpenCVStreamlit
state
buildingtool
note
runs locally, no hosted demo yet
cast it
github
annotation

version one could tell you what exercise you were doing. it had zero idea if you did it well

p. 6

✦ casting notes

01The first attempt

the first version was a scrappy 137-line Streamlit app in an old repo. it ran a YOLO model on every frame, drew boxes on the video, and guessed which of 15 movement types you were doing (the 15 Penn Action classes). it could name the exercise, sure, but it had nothing to say about how well you actually did it

02Where it got interesting

recognizing an exercise and judging it turned out to be totally different problems. the rebuild fine-tunes a YOLOv8 pose model to track 13 keypoints, then feeds those into a separate biomechanics layer, around 1,300 lines on its own, that works out joint angles, alignment, and range of motion for each move, compares them against thresholds i picked (like a squat knee angle under 90° counting as good depth), and averages the results into a 0-10 score

03Where it landed

fine-tuned YOLOv8n-pose on the Penn Action dataset (163,841 frames across 2,326 videos). it handles 15 movement types (the Penn Action classes), everything from pull-ups and squats to tennis serves, golf swings, and guitar strumming, gives you back annotated video, and exports JSON and CSV reports. to be clear about what that score is: it's how many of my rules a video passes, not a validated measure of good form. nobody's compared it with a coach's ratings, it averages every frame (so the standing part of a squat counts as "too shallow"), and a few distance checks have a known bug where they compare pixel coordinates against thresholds meant for normalized ones. the README lists all of it

04The next question

how much of good form can you even judge from one camera angle, and at what point do you actually need a second one