Improving Eurovision’s Jury System, Examining The Significant Change To Jury Scoring Written by Ellie Chalkley and Ewan Spence on April 27, 2018 | 7 Comments

2013 saw the EBU alter how the jury score was calculated. The second Malmö Contest saw individual jurors asked to rank every single song from their most favoured to least favoured (in previous years, jurors were only asked for an ordered Top Ten).

A jury as a whole can only hand points to the top ten songs. As such the full ranking method means that any song placed outside a juror’s first ten is effectively given a negative vote. In the past, if your song really turned off a single juror, that single last place rank could have dragged you out of the points of that jury, no matter what the other jurors thought of your song.

That has now changed for the better.

A System That Rewards Love

Speaking to ESC Insight, a representative of the EBU said “The calculation method which determines the points awarded by the National Juries has been changed to increase the value of a group of jurors over the opinion of a single individual. This has been recently approved by The Reference Group.” Further details can be found on eurovision.tv:

Rather than giving each rank given by a juror the same weight, the EBU will allocate predefined ‘score values’ to each ranking position, thereby increasing the value of the top-10 ranks, the top-3 in particular. These score values start with the value of 12 for the first rank and will decrease exponentially further down the ranking list. This is also called the ‘exponential weight model’. The sum of the scores for all 26 songs from the five jurors will create the national jury result where the resulting top 10 ranked countries will be awarded that jury’s 12, 10, 8 points and so on.

In practice, this new method of combining the jury rankings offers a number of advantages that increase the competitive nature of the Song Contest.

1. A single juror cannot drag a song out of the jury’s top ten ranking by placing it at the bottom of their ordered list. Previously ‘single juror drag’ meant that a good jury song was one that did not upset anyone on the jury, now it’s possible for four positive voices to drown out the single negative voice.
2. Conversely, a single juror who is incredibly positive can drag a song into the jury’s overall top ten rankings. If you can ignite the passion of a single juror you will be rewarded. Taking risks is no longer a barrier to scoring.
3. And if you have all five jurors raking your song n the top two or three positions, you can be assured of a significant lead over the competition, offering a better chance of grabbing the douze points in the final calculation.

To coin a phrase, the juries as a whole will find it much easier to #CelebrateDiversity at this year’s Eurovision Song Contest. The marmite songs that would previous have to rely almost entirely on the televote will now be in a better place to grab some jury points, while the successful jury songs are far more likely to stand out from the crowd and be justly rewarded.

Let’s look at this in a bit more detail

Here Comes The Maths

How exactly does an ‘exponential weight model’ work in practice?

The new exponential jury scoring system uses a mathematical formula to turn each jurors ranked places into points. Your first place gets 12 points, your second place gets just under 10 points, your third gets just over 8, with the points value dropping exponentially towards zero – the system will run out to many decimal points in the final calculation. Jurors favourite songs get a comparatively huge score, in line with the general Eurovision scoring method.

Eurovision 2018 scoring chart, Ellie Chalkley

Things get really interesting outside the top ten places though, because you can see that the relative difference in score between 10th rank and 18th rank is much less than the difference in score between 1st rank and 2nd rank! Being ranked low is still not great, but at least it’s not an effective score of -8 in the semis or -16 in the Grand Final!

To give you an idea of how this changes the jury ranks required to get points, I ran two simulations using my approximation of the new jury scoring method, one based on the Semi Final 1 data from 2016 and one based on totally random scores for the 2018 Semi Final 1.

Working With Real Data

Thanks to the public release of juror rankings from recent Contests, it’s possible to look at the impact of the new system using actual data. How did the ‘jury points’ under the new system translate to the announced points for the top ten songs… and the jury points scored by the eleventh placed song.

Eurovision 2016’s Semi Final 1 under the new system, Ellie Chalkley

In the real 2016 Semi 1, there was a clear group of high scoring songs. The jury scores for these songs cover a range from 55 to 60, which is 5 x 12 or the theoretical maximum. The score required to get a single point varied between countries, depending on how closely the different jurors agreed, but it seems like a score of 11 can be enough to get into 10th place and get a single jury point.

Eurovision 2016 exponential scoring chart, Ellie Chalkley

11? That surely means that you could be ranked 18th and last in the semi by 4 of the jurors and 1st by one and still be in with a chance of getting some points, especially if the other 4 jurors all agree on their 1st ranked song. The more normal way of getting to 11-ish exponential points and that final scoring spot would be to be ranked at least 10th by all jurors. This all feels fair and just.

A Random ‘Base’ Result

In my totally randomised 2018 Semi 1, the range of total jury scores given to each song was much narrower, from 3.5 to 46 compared with 2.7 to 60 for the real data. It is still possible of course to score 60 in the random model, but this is very improbable. In order to score 1 point in the totally random data, you need to get at least 17 exponential points from the Jurors. You could get that by getting one 1st place, one 5th place and 3 x last place or – more likely – having all jurors rank you at least 7th.

Eurovision 2018 random scoring chart, Ellie Chalkley

While it’s unlikely a real set of jury data would look like this randomised trial, taken together with the real 2016 data, we have a boundary condition for how well you need to perform in order to score jury points.

Eurovision 2018 random scoring chart, Ellie Chalkley

As before, you’re assured points if you manage to place in the top ten of every juror, but you’re also now possibly in line for points if your song utterly divides the jury. The ‘Marmite Song’ can score once more.

Eurovision 2018 scoring chart, Ellie Chalkley

Let’s Celebrate Competition

This change to the jury scoring system is a welcome step forwards to a more competitive Contest. It rewards positivity, it diminishes the power of a single juror to negatively impact a song, and it allows strong but divisive songs the opportunity to achieve a respectable jury score ahead of the televote the following night.

More importantly, it proves that the EBU has carefully considered the competitive angle of the Eurovision Song Contest. It remains a television show first of all, but it is a television show of a contest. ESC Insight has been critical of EBU changes that have emphasised spectacle over sporting considerations, but this subtle tweak of the rules opens up the Song Contest in a positive way.

It offers a better way to combine jury opinions, and it rewards songs that generate positive reactions in individual jurors. By allowing songs to gather solid jury votes while appealing to genre fans in the televote, it offers every song a better chance to go for the win at the Eurovision Song Contest.