Judging SCBA

Lake High is a man with a vision and a mission. When the affable, avuncular retired stockbroker teamed up with Walter Rolandi to form the South Carolina Barbecue Association (SCBA) a few years ago, they had one thing in particular in mind: To make South Carolina the acknowledged center of the barbecue world.
“We’re already the unacknowledged center of the barbecue world,” he says, standing at the podium at the SCBA judg’s seminar in Columbia, SC. He ticks off the reasons:
“Barbecue was invented here. We’re the only state that has all four basic barbecue sauces; Vinegar and pepper, mustard, light tomato and heavy tomato. North Carolina has three, as does Georgia.”
High has a big job ahead of him. South Carolina would probably be way down most people’s list of states that make the best barbecue, behind, say, Kansas, Tennessee, and North Carolina. High is out to change that. His strategy is to use his association to both advertise for South Carolina barbecue and to train up a cadre of judges who will serve as a kind of quality control for South Carolina’s “cookers” –he never calls them chefs.
SCBA judges travel to about thirty barbecue cook offs across the state each year to evaluate the product of between a dozen and 100 cookers per event. The average judge will taste eight samples of barbecue per event. He needs a lot of judges.
“We’ve found that about 12 samples is all anyone can handle without getting mouth fatigue—you blow out your taste buds,” he says.
There are about fifty people attending the seminar, as well as about ten members of the association, proudly wearing their aprons, which the get once they are “certified”, and their Senior or Master Judge boaters, which they earn by judging at least thirty events as well as serving on a cooking team up to three times, “to get an idea of what the cooks go through,” as well as to give the cooks an idea of how the judging is done.
“Cooks are paranoid”, says Lake. He tells of cooks who accused he judges of recognizing a certain cooks’ plate, or making back room deals, all of which is untrue. This paranoia probably comes from stress and fatigue, as the cookers at your average event have usually been up all night working steadily for 18-24 hours straight, preparing, cooking, and tending their pork, quite often with liberal infusions of their favorite alcoholic beverage.
“One thing I’ve learned from being on a cooking team is that a 1:30 in the morning, the jokes are a lot funnier if you’ve had a few beers under your belt.” he says.
After staying up all night and fretting over, and saucing the pork and tending the fires, the tired, bedraggled, hung over cooker has a twenty minute time window to have his pork plated (on a simple white Styrofoam plate) and taken to the judges, where it disappears inside the judging room and is not heard from again, often for several hours.
What takes place in the judging room could mean a prize of several hundred dollars to one cooking team—and a world of disappointment to many others. SCBA takes this job seriously.
“Judging is easy, scoring is difficult.” Says Rolandi, who looks like he could be former president Bill Clinton’s long lost brother.
At the seminar we go over the three main scoring systems used by other states.
High discusses the good and bad points of each.
The Kansas City system is too snobbish according to High, who reports of an event where a cooker’s sample was summarily disqualified because it wasn’t plated properly.
The Memphis system is too heavily reliant on appearance, with three out of six categories relying on sight vs. taste.
“The only way to tell if barbecue is good or not is with your mouth.” he says.
The North Carolina has no barbecue association, so event promoters end up rounding up whoever is handy to do the judging.
“They may get the local disc jocky, the Mayor’s wife and the town drunk; but some of those town drunks are pretty good judges,” he quips.
The score sheet for North Carolina makes up for the lack of expertise by giving guide lines for how the meat should look and taste, which, according to High, tends to make all the barbecue the same. High wants diversity. He talks of his “first principles”: Get rid of your biases. He asks the group; “How many of you hate mustard sauce?” A few hands go up.
“Bad, bad, bad!” he says. You can’t bring your personal bias into the judge’s tent.” When another participant avers that she thinks she would be a good judge, he asks why.
“Because I think I’d be good at judging the consistency of the sauce.”
“We don’t judge sauce. We judge meat!” He snaps. “If they bring us their meat with a container of sauce we politely hand the sauce back.”
This is a good strategy, as sauce, like memory, can hide a multitude of transgressions.
Working with Walter, who is a doctor, Lake, who has been a certified wine judge
for decades, came up with a scoring system that attempts to make what is admittedly a subjective process as scientific as possible. Samples are judged blindly. Only a number, assigned by the Marshall identifies its owner. SCBA no longer does on-site evaluations, where you go out to the cooker’s rig and sample the meat, as this could lead to bias.
SCBA is the only association that judges aroma, as well as the other standard categories of appearance, taste and tenderness. The SCBA system consists of setting a standard for each category among the samples and judging each successive sample against the standard. That way, instead of comparing eight samples against each other, you are only comparing two samples at a time; the current sample and the standard. The judging is done on a weighted twenty point system with taste and tenderness having the most weight. Judges are encouraged to carry their numbers out to two decimal places, which according to High, virtually eliminates ties; and they can change their ratings, based on their reaction to subsequent samples, unlike the Kansas City standard. In fact High mentioned the Kansas City system so often that the person sitting next to me felt the need to lean over and whisper; “I’m a Kansas City judge. We’re not that bad.”
The average score turns out to be around 14, which may seem high until understand that average cooks don’t go to cook offs.
“Most of these people cook barbecue for a living. If their barbecue wasn’t above average, they’d be out of business.” Says High.
When it’s his turn to talk, Rolandi strides to the podium and relates the three guiding principles of the judging: Fairness, consistency and objectivity. He warns about order effects; the effect on the judging of the (random) order in which the samples are received; and time effects, that occur because the samples toward the end of the line will have cooled off compared to the first samples; and the judges enthusiasm tends to wane
as they begin to get full and their taste buds become exhausted. This is why judges are limited in the number of samples they taste. Husbands and wives aren’t allowed to judge at the same table, as they can pick up subtle cues from each other. Judges are expected to keep a poker face and not talk about their samples, a rule which, according to High is punishable by a gruesome death if violated. Outliers are questioned.
“If everybody else scores the meat 3.7 on tenderness and you score it a 1.53 we’re going to ask you why.”
It may be that you just happened to get a tough piece of meat, or it may be that you need to do some soul searching.
High peppers his remarks with anecdotes from his years working cook offs. There was the guy who handed him a sample that was rotten, the guy who set his pig on fire—and ended up winning first place, the sample that was so spicy it severely hampered the judges ability to taste the subsequent samples, the time a team passed out drunk and slept through the judging, and the time they were short on judges and he and Walter had to evaluate 22 samples each.
“We survived, though,” he says.
Before judges become certified, they must attend the seminar and also work four events where they sit at a “newbies” table and are allowed to judge the meat, but their scores aren’t used.
“We don’t want you to do any damage.” Says Walter.
Newbies are shepherded by an experienced judge and are allowed to discuss among themselves how they came to their conclusions about the samples. By the time they are certified, they will have tasted upwards of 32 different samples of barbecue, and hopefully honed their palettes.
Along with putting South Carolina barbecue on the map, all of this thought and preparation goes into the SCBA’s judging system in order to fulfill the second part of Lake High’s vision.
“We want you to have fun!” he says.