dhruva.lifecontact

← blog

[Jun 22, 2025 · 3 min read][research]

A Coach That Watches

Let's be honest, jiu-jitsu can be a boring sport to watch.

Take what's supposed to be its ultimate form - no time limit, submission only, the big ADCC matches Gordon Ryan has made a career of winning. Two of the best grapplers in the world spend the better part of an hour trying to drain each other's gas tank, and then suddenly something happens and somebody wins. If you don't train, it looks like two people lying on a mat. It reminds me of test cricket back in the day - long stretches where almost nothing happens, and you'd better love the game to sit through it.

Which is why the match everyone talks about is so different. Craig Jones set up his invitational specifically to make jiu-jitsu entertaining, and out of it came Andrew Tackett against Kade Ruotolo. Two wrestlers, two ridiculous athletes, going at a pace that makes the sport look like a different sport. Scramble after scramble, nobody stalling, nobody resting. It's widely called the greatest grappling match ever, and it was only a semi-final.

I've watched it more times than I can count, and I still can't fully parse it. For someone like me, who's used to slower jiu-jitsu, there's just too much happening - so many moves and counters and transitions, so dense, so fast. And somewhere around my tenth rewatch, I had a thought I couldn't shake. Could an AI understand this? Actually understand it, move by move, the way a coach would?

So I've spent a few evenings finding out. I pointed open vision-language models at it and at other footage. I tried LLaVA, I tried Qwen2.5-VL, I cut clips short and long, I asked nicely and I asked very specifically.

Honestly, it didn't work. Not even close. The answers came back fluent and confident and grammatical, and a lot of them were just wrong. Which leg belongs to which person, who's on top, whether that's a guard or a pass in progress - the moment two bodies are tangled up, the models lose the plot.

In fairness, that's the hardest thing you could ask them. Jiu-jitsu is two people wrapped around each other, half the limbs hidden at any given moment, and most of what matters is invisible in a single frame. A grip that's about to break. Weight that's about to shift. It's chess with your body as the pieces, except in 4D, because the fourth dimension is time. A position doesn't mean much on its own, it's what came before it and what's about to come after that tells you whether you're winning. Show a model one still frame and ask it who's winning and you might as well be showing it a single chess move with the board covered.

There's a selfish reason I care, too. Every time I roll, there are moments I can't explain afterwards. I got swept and I don't know how. I passed and I don't know why it worked that time and not the ten before. You can film your rolls, but rewatching them alone only helps so much when you don't know what you're looking for. What I actually want is a coach who's watched every roll I've ever done and can tell me the pattern I keep falling into. When I first tried to put it into words, the best I could come up with was "an AI jiu jitsu critique and mentor basically." Not very poetic, but that's the idea.

Seeing the models fail made that idea more interesting to me, not less. If they were already great at this, it'd be a product problem, and there'd be ten apps doing it by now. The fact that they fall apart on two people in contact means there's something real and unsolved in there, something about bodies and occlusion and time that these models just don't have yet. That's the kind of problem I can't leave alone.

I want to be clear with myself that this is curiosity, not a company. It's an itch. I'll probably keep poking at it the way I poke at a position I keep losing in - try something, fail, go home, think about it in the shower, try something slightly different next time.

The mats taught me that the stuck feeling is usually right before something opens up. I'm going to keep pulling on this thread.