You can only cup the table in front of you: the role of the comparator
by Stephen Cornford
I was taught early in my career “You can only cup the table in front of you.” This potentially confusing adage neatly summarises why we use comparators when assessing a set of coffees. This article is an exercise in thinking out loud about the tasting process and considering the problems encountered in current practice. It also suggests possible solutions to these problems for the future.
What is a comparator?
When cupping samples that you are considering buying or would like to review (e.g. stock), you will usually include a ‘comparator’ on the table. A ‘comparator’ is a coffee that you have previously assessed and that has a recently assigned ‘value’ (usually a score). This coffee is placed randomly on the cupping table amongst the other coffees.
The comparator should represent the minimum acceptable quality for a particular ‘type’ of coffee. You may be wondering why one of maximum quality is not used. Something to aim for, you would think. The main reason is that ‘standout’ attributes can be too intense and alter your perception of taste. This can cause you to overlook coffees on the table that fulfil valuable roles in our industry, for example as a component of an espresso blend.
Blind cupping
The idea behind blind cupping is to prevent a cupper’s preconceptions from impacting the assessment and scoring of a coffee. It makes you more objective.
When cupping blind, a set of coffees is examined with no information regarding the coffees’ identity, not even the comparator. After assessing each coffee, all are compared. The scores of your samples may be higher, lower or equal to the comparator, and from here you can make a purchasing decision (or a pricing amendment).
That’s the theory, anyway.
The Problems
Position
Trouble can arise when you are not aware of the comparator’s position on the table. Because the cupping is entirely blind, the cupper can only reach broad conclusions rather than using the comparator as a ‘benchmark’ of sorts’. You can end up overly focusing your attention on assessing the comparator itself, rather than comparing things to it. Additionally, if the idea of a comparator is to have something to compare against, in the process of cupping blind, you will not be able to accurately rank and detail the sweetness in intensity of everything on the table from highest to lowest in contrast to that coffee. If you knew where the comparator was, you could note if assets were higher or lower than that coffee.
Because of these factors, I would argue there is a case for indicating the comparator’s location throughout the cupping, whilst keeping all other details (such as its origin) blind until the cupping has concluded. This is not Mercanta’s current practice, but a case could be made.
Selection
The process of selecting an appropriate comparator is often difficult. Where do you find a good one? Ideally from coffees that you already have in store – or that you already know the ‘value’ of. However, in actuality, this is very often hard to do. For instance, New Crop Brazil offer samples could be put on a table with an ‘in store’ coffee with a known cup score. Depending on how that coffee has held up over the last 9 months, you could end up over- calculating the quality of all the New Crop samples. It is even more difficult to find a good comparator for coffees with distinct profiles.
Origin handicap
Related to the above: No matter what ‘they’ tell you, there is no ‘objective’ 84. Assessment depends, always, on context: that is, the table. You could easily approve a Guatemalan coffee which needed to score 85 against a Brazil of the same score. However, against an 84 Kenyan it would more likely be rejected. We call this the origin handicap. Coffee can only be compared against its own type, but that ‘type’ can be difficult to specify. That’s another article in itself.
Freshness
Pre-shipment samples cupped against their arrival sounds like common sense. However due to their small volume, even under favourable storage conditions they can age quickly and skew results. The reverse is true if you use a freshly harvested pre-shipment sample from a different origin. Ideally you would use the same comparator when approving the offer, pre-shipment and arrival; however, since coffee is constantly deteriorating, this is not practical.
Our current best practise is to regularly change the comparators we use, and work the best we can with an ever-moving goal post. It’s not optimal, but it is reality.
Lab standards
Most lab settings are easily kept consistent and have little effect on the results. However over- and under-roasting can modify a score by several points. A good cupper can pick this up; but in my opinion, even the best cannot reliably adjust by the correct factor. Therefore, strict parameters must be followed under the watchful eye of a professional roaster who, in most cases, will deliver the samples to the correct roast degree. Any sample that fails must be roasted and cupped again.
A Solution for the Future?
Freezing
It is feasible that we will soon be freezing comparators. The possibility of comparing any previous harvest to the current one without the issue of age is appealing. Mercanta is currently exploring freezing’s potential for practical application with both raw and roasted beans. Imagine the possibility of tracking whether quality is increasing or decreasing year-on-year. We might even be able to side-step the variability issue in roasting. Fresh comparators of all kinds to hand at any moment in time, regardless of the season: this would be a quality control game changer.