UFC 333 predictions graded: fans vs our model vs the odds

Three opinions went into UFC 333 in Abu Dhabi on Saturday: the sportsbook consensus, the 868 people who made picks on our pick'em, and our AI model. All week I printed what each of them believed. Fourteen fights have now been settled (Arnold Allen's withdrawal took his bout off the card, and Aaron Pico's fight against Allen's replacement counts), so it is only fair to grade all three the same way, using the numbers they had before the first bell.

The short version: the market picked the most winners, the model put the best probabilities on the board, and I picked worse than all three of them. None of them did much better than a coin.

The scoreboard

Every number below is the last one each side had before the early prelims began. Fans is the share of pick'em entries on the first-named fighter. Model is our model's win probability for that fighter from its final pre-fight run, in the early hours of fight day. Line is the 18-book consensus at the first bell, turned into a probability with the bookmakers' margin removed.

Counting winners first. The closing favorite won 10 of the 13 fights that had one; the main event closed at -109 on both men, which is as even as a line gets. The fans' majority won 8 of 14. Our model picked the winner in 8 of 13, with Dawson vs Aliev at 49.9 percent, which I'm not counting either way.

Counting winners treats a 50.1 percent call the same as a 95 percent one, so I also scored the probabilities. The Brier score is the average squared gap between the forecast and what happened; lower is better, and a forecaster who said 50 percent on every fight scores 0.250. The model scored 0.229, the fans 0.233 (reading their vote share as a probability), the closing line 0.237. Log loss, which punishes a confident miss harder, ranks them the same way: 0.645, 0.651 and 0.674, against 0.693 for the coin.

Before anyone frames that as the model beating the market, one fight does all the work. Take out Thicknesse vs Chopurov and the line scores 0.203, the fans 0.221 and the model 0.229. The market was the best forecaster in 13 fights and the worst in one, and that one was the biggest miss of the night by anybody.

The title fights: the crowd told me so

Photo: NAPARAZZI / CC BY-SA 2.0

Friday's preview ran under the headline that the fans backed both champions and our model didn't. Both champions kept their belts. Alexander Volkanovski knocked out Movsar Evloev at 4:54 of the second round, the first loss in Evloev's 21 pro fights, and Petr Yan beat Merab Dvalishvili by unanimous decision over five rounds.

Our model gave Volkanovski 33.5 percent and Yan 42.9. Those were its two largest misses on the card, and they came in the two fights everyone was watching. The market had the main event at a pick'em and Yan at 58.5 percent, so it took no side in one and got the other right.

The fans had both right, but less loudly than I wrote on Friday, when 62.5 percent of main-event entries were on Volkanovski. Among entries made before the early prelims started, the share was 58.1 percent. Entries made or changed after that went 132 to 100 for Evloev, which is how the final count landed at 319 to 290. Yan's support thinned the same way, from 63.8 percent on Friday to 55.6. The late crowd drifted toward the model. The early crowd was right.

Nobody saw the finish coming. Of the 41 Volkanovski entries that named a method, 21 said knockout. That was the closest anyone came to calling the night's biggest result, and it was 21 people out of 609.

When the model stood alone

Photo: Photo by Tommy Gilligan/USMA PAO / Public domain

Three times on Saturday the model sided against both the fans and the market. It won all three.

The loudest was Colby Thicknesse against Asaf Chopurov, who made his UFC debut at -620. The line had Thicknesse at 17.5 percent, the fans at 37.3, our model at 53.3. Thicknesse won a split decision, 29-28 on the cards of David Lethaby and Ben Cartlidge, with Darryl Ransom scoring it 29-28 for Chopurov. In the prelims piece I filed it under "the number I would not sign", and I had reasons: Chopurov had no UFC fights for the model to read, and no height or reach on file when it ran. That piece quoted 60.8 percent from the October 19 run. By the final run the model had come down to 53.3, still on Thicknesse. I also wrote that the six biggest gaps between the model and the line since mid-July had all gone to the betting favorite. This one, wider than any of them, did not.

Dominick Reyes was the second. Our model gave him 58.4 percent, against a line that had Azamat Murzakanov at 63.6 and a crowd that had 58.3 percent of its entries on Murzakanov. Reyes stopped him at 2:40 of the first round. The third was the quietest: Daniel Santos at 54.6 percent against Bolaji Oki, whom the line had walked from +113 to -110 in five days. Santos won 29-28 on all three cards.

Wednesday's piece noted that since mid-July, when the model backed a betting underdog at 58 percent or more, it had been right three times in nine, and that it was doing it twice on this card. It went one for two. Losene Keita, at 59.6 percent, was knocked out by Aaron Pico in 74 seconds.

Where the market saw what we didn't

The mirror image: three fights where the market stood against both the fans and the model, and it went three for three as well. Lone'er Kavanagh, at 50.5 percent, knocked out Ramazan Temirov at 4:28 of the first round; the fans had Temirov at 66.3 percent, the model at 52.6. Pico was at 57.0 percent, with 61.8 percent of the crowd and 59.6 percent of the model on Keita. And Grant Dawson was at 51.0, against a model that could not separate him from Nurullo Aliev and a crowd that put 71.4 percent on Aliev. Dawson won a unanimous decision, the fans' worst number of the night.

So the fans' six misses were Kavanagh, Pico, Reyes, Dawson, Thicknesse and Santos. In the six fights where the fans and the model picked different winners, each won three: the crowd took both title fights and Abubakar Vagaev over Jacob Malkoun, the model took Reyes, Thicknesse and Santos. The five fights where all three agreed (Volkov, Yakhyaev, Magomedov, Steele, Lucindo) all went the agreed way.

What the line did in the last week

Line movement is the market changing its mind, so it is fair to ask whether it changed its mind in the right direction. On this card, it was a coin flip. Six bouts moved two points of probability or more between the first consensus and the first bell. Three moved toward the winner: Pico (from -110 on both men on October 14 to -146, the biggest move on the card), Yakhyaev (-450 to -555) and Lucindo (-365 to -422). Three moved toward the loser: Oki (+113 to -110), Chopurov (-500 to -620) and Malkoun (+157 to +126).

That last one deserves its own receipt. The prelims piece said the market was walking toward the model on Vagaev vs Malkoun. It was, and it kept walking, from 62.5 percent Vagaev at the open to 57.6 at the bell. Vagaev won a unanimous decision anyway. Scored the same way as above, the opening lines (0.236) were very slightly better forecasts than the closing ones (0.237). The title fights hardly moved at all: Volkanovski opened at -114 against -109 on August 16 and closed at -109 both ways, and Yan went from -157 to -156 in ten weeks.

The worst card of the night was mine

In Friday's preview I made a pick in all 14 fights. I went 7 of 14. Evloev, Dvalishvili, Temirov, Murzakanov, Cam Rowston, Aliev and Chopurov all lost, and of the seven winners I did name, I had the method right on three. That is one fewer winner than the crowd, one fewer than the model and three fewer than the line.

I closed that piece by saying that if both champions kept their belts, the crowd would get to say it told me so. Consider it said. The card's pick'em results are on the UFC 333 leaderboard, and the season standings on the all-time leaderboard. Three forecasters, a gap of 0.021 between the best of them and a coin, and a writer who lost to all three: whose number are you reading first next time?