Project 06 · Computer vision · Fall 2025

The headline number was 91.95%. The one I'd defend is 82.8%.

SWISH predicts whether a basketball free throw goes in, using only the shooter's 3D skeleton at the moment of release, before the ball is visible in flight. It works. It works less well than our abstract says, and the gap between those two numbers is the part worth writing down.

My roleRelease detection, data pipeline, models, demo
TeamThree, COMS 4731
StackYOLOv8-pose, SAM3D Body, PyTorch
ModelKeyJointNet, 72K params
82.8%five-fold CV, ±5.6
75.9%majority-class baseline
0.84 vs 0.58AUC, pose vs trajectory
01

What we built

Free throws are the most standardized shot in basketball and they go in about 75% of the time. The question was whether the body tells you anything the ball hasn't yet. Prediction has to happen at the release frame. Once the ball is in flight the problem is physics and everyone can see it.

The pipeline runs in four stages. YOLOv8-pose finds the release frame by watching for an arm angle between 100 and 145 degrees with an elevated wrist. SAM3D Body extracts a 70-joint 3D skeleton from that frame and the three around it. Velocity and acceleration across the four frames give a tensor of nine channels by four frames by seventy joints. A small attention-plus-temporal-CNN called KeyJointNet reads fifteen upper-body joints and predicts make or miss.

I owned the release detector, the data pipeline, the model architectures and the demo. That meant the YOLOv8 detection heuristics and their confidence scoring, the annotation tool we used to review candidate frames, the curation and feature engineering that produced the training set, the architecture comparison across KeyJointNet, ST-GCN and the two baselines, the trajectory-only models, and the Streamlit app that runs the whole thing on an uploaded clip.

Getting usable data was most of the work. We started with 2,352 candidate clips across two sources and finished with 174. Broadcast footage is a bad input for pose estimation: the shooter is often occluded, the camera angle moves, and the ball is frequently invisible at the moment it matters.

StageCandidatesSurvived
Basketball-51, after manual validation1,332139
Basketball-51, after pose extraction QC139102
NBA Play DB, after auto-detection1,020985
NBA Play DB, after confidence filter985340
Merged training set—174

The result that holds up best has nothing to do with the headline. I trained trajectory-only models on the full 1,020-sample set using ball tracking and physics features. Random Forest got 75.0%. XGBoost got 75.0%. Always predicting “make” gets 75.3%. Early ball trajectory provided no edge at all over guessing, with three times the training data. Pose reached an AUC of 0.84 against trajectory's 0.58. The body really does say something the ball hasn't yet.

02

The number

Our abstract leads with 91.95% accuracy at 0.97 AUC. Table 2 of the same paper reports the same model at 82.8% ±5.6% under five-fold cross-validation. Both numbers are in the submission. Only one of them estimates how the model would do on a shot it hasn't seen.

The decision threshold and the probability calibration were both fit on the data they were then scored against. The confusion matrix sums to 174, which is the entire dataset.

What went wrong

Three things inflate the headline, in descending order of how much they bother me.

  • The threshold was tuned in-sample. Moving the decision boundary from 0.50 to 0.64 bought five points of accuracy, chosen by looking at the same 174 samples it was evaluated on. Platt scaling has the same problem.
  • The baseline is higher than it looks. The merged set is 132 misses to 42 makes, so always predicting “miss” scores 75.9%. The honest 82.8% sits about one standard deviation above that. The class balance is also inverted from real basketball, where roughly 75% go in.
  • I merged in 72 samples from a second source, all of them misses. If those clips differ systematically in camera angle or pose-extraction quality, then “which dataset is this” becomes a usable proxy for “miss” across more than half the miss class. It improved our numbers, we wrote it up as a win, and I never tested for the confound.

That last one matters most for the finding we were proudest of. The model called 88 of 88 high-confidence misses correctly and only 87.9% of 33 high-confidence makes, and we wrote that up as a biomechanical result: bad form is distinctive, good form is necessary but not sufficient. It might be. A shooting coach would tell you the same thing. But a source confound produces exactly that pattern, and the experiment that separates the two is one I designed the dataset for and didn't run.

What I'd do differently

Hold out a source. Train on the curated 102 and test on the MHR70 samples, or report the 108-sample clean subset on its own. Tune the threshold inside each CV fold rather than on the full set. Split folds by shooter instead of by class, since NBA broadcast clips repeat players and stratified CV doesn't care. None of that is hard. We ran out of semester, which is a reason and not an excuse.

The paper also frames the asymmetry as a betting edge, with an expected value calculation and a claimed 110% ROI. I'd cut that. It rests entirely on the in-sample threshold, and a result that fragile shouldn't be dressed as a strategy.