Support ESC Insight on Patreon

Improving Eurovision’s Jury System, Examining The Significant Change To Jury Scoring Written by and on April 27, 2018 | 9 Comments

The EBU has announced a major change to how juries score the entries at the Eurovision Song Contest. Ellie Chalkley investigates the maths behind the changes, explains the impact of the new system, and why it makes Eurovision a more competitive contest.

2013 saw the EBU alter how the jury score was calculated. The second Malmö Contest saw individual jurors asked to rank every single song from their most favoured to least favoured (in previous years, jurors were only asked for an ordered Top Ten).

A jury as a whole can only hand points to the top ten songs. As such the full ranking method means that any song placed outside a juror’s first ten is effectively given a negative vote. In the past, if your song really turned off a single juror, that single last place rank could have dragged you out of the points of that jury, no matter what the other jurors thought of your song.

That has now changed for the better.

A System That Rewards Love

Speaking to ESC Insight, a representative of the EBU said “The calculation method which determines the points awarded by the National Juries has been changed to increase the value of a group of jurors over the opinion of a single individual. This has been recently approved by The Reference Group.” Further details can be found on eurovision.tv:

Rather than giving each rank given by a juror the same weight, the EBU will allocate predefined ‘score values’ to each ranking position, thereby increasing the value of the top-10 ranks, the top-3 in particular. These score values start with the value of 12 for the first rank and will decrease exponentially further down the ranking list. This is also called the ‘exponential weight model’. The sum of the scores for all 26 songs from the five jurors will create the national jury result where the resulting top 10 ranked countries will be awarded that jury’s 12, 10, 8 points and so on.

In practice, this new method of combining the jury rankings offers a number of advantages that increase the competitive nature of the Song Contest.

  1. A single juror cannot drag a song out of the jury’s top ten ranking by placing it at the bottom of their ordered list. Previously ‘single juror drag’ meant that a good jury song was one that did not upset anyone on the jury, now it’s possible for four positive voices to drown out the single negative voice.
  2. Conversely, a single juror who is incredibly positive can drag a song into the jury’s overall top ten rankings. If you can ignite the passion of a single juror you will be rewarded. Taking risks is no longer a barrier to scoring.
  3. And if you have all five jurors raking your song n the top two or three positions, you can be assured of a significant lead over the competition, offering a better chance of grabbing the douze points in the final calculation.

To coin a phrase, the juries as a whole will find it much easier to #CelebrateDiversity at this year’s Eurovision Song Contest. The marmite songs that would previous have to rely almost entirely on the televote will now be in a better place to grab some jury points, while the successful jury songs are far more likely to stand out from the crowd and be justly rewarded.

Let’s look at this in a bit more detail

Here Comes The Maths

How exactly does an ‘exponential weight model’ work in practice?

The new exponential jury scoring system uses a mathematical formula to turn each jurors ranked places into points. Your first place gets 12 points, your second place gets just under 10 points, your third gets just over 8, with the points value dropping exponentially towards zero – the system will run out to many decimal points in the final calculation. Jurors favourite songs get a comparatively huge score, in line with the general Eurovision scoring method.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2018 scoring chart, Ellie Chalkley

Things get really interesting outside the top ten places though, because you can see that the relative difference in score between 10th rank and 18th rank is much less than the difference in score between 1st rank and 2nd rank! Being ranked low is still not great, but at least it’s not an effective score of -8 in the semis or -16 in the Grand Final!

To give you an idea of how this changes the jury ranks required to get points, I ran two simulations using my approximation of the new jury scoring method, one based on the Semi Final 1 data from 2016 and one based on totally random scores for the 2018 Semi Final 1.

Working With Real Data

Thanks to the public release of juror rankings from recent Contests, it’s possible to look at the impact of the new system using actual data. How did the ‘jury points’ under the new system translate to the announced points for the top ten songs… and the jury points scored by the eleventh placed song.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2016’s Semi Final 1 under the new system, Ellie Chalkley

In the real 2016 Semi 1, there was a clear group of high scoring songs. The jury scores for these songs cover a range from 55 to 60, which is 5 x 12 or the theoretical maximum. The score required to get a single point varied between countries, depending on how closely the different jurors agreed, but it seems like a score of 11 can be enough to get into 10th place and get a single jury point.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2016 exponential scoring chart, Ellie Chalkley

11? That surely means that you could be ranked 18th and last in the semi by 4 of the jurors and 1st by one and still be in with a chance of getting some points, especially if the other 4 jurors all agree on their 1st ranked song. The more normal way of getting to 11-ish exponential points and that final scoring spot would be to be ranked at least 10th by all jurors. This all feels fair and just.

A Random ‘Base’ Result

In my totally randomised 2018 Semi 1, the range of total jury scores given to each song was much narrower, from 3.5 to 46 compared with 2.7 to 60 for the real data. It is still possible of course to score 60 in the random model, but this is very improbable. In order to score 1 point in the totally random data, you need to get at least 17 exponential points from the Jurors. You could get that by getting one 1st place, one 5th place and 3 x last place or – more likely – having all jurors rank you at least 7th.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2018 random scoring chart, Ellie Chalkley

While it’s unlikely a real set of jury data would look like this randomised trial, taken together with the real 2016 data, we have a boundary condition for how well you need to perform in order to score jury points.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2018 random scoring chart, Ellie Chalkley

As before, you’re assured points if you manage to place in the top ten of every juror, but you’re also now possibly in line for points if your song utterly divides the jury. The ‘Marmite Song’ can score once more.

Eurovision 2018 scoring chart, Ellie Chalkley

Eurovision 2018 scoring chart, Ellie Chalkley

 

Let’s Celebrate Competition

This change to the jury scoring system is a welcome step forwards to a more competitive Contest. It rewards positivity, it diminishes the power of a single juror to negatively impact a song, and it allows strong but divisive songs the opportunity to achieve a respectable jury score ahead of the televote the following night.

More importantly, it proves that the EBU has carefully considered the competitive angle of the Eurovision Song Contest. It remains a television show first of all, but it is a television show of a contest. ESC Insight has been critical of EBU changes that have emphasised spectacle over sporting considerations, but this subtle tweak of the rules opens up the Song Contest in a positive way.

It offers a better way to combine jury opinions, and it rewards songs that generate positive reactions in individual jurors. By allowing songs to gather solid jury votes while appealing to genre fans in the televote, it offers every song a better chance to go for the win at the Eurovision Song Contest.

About The Author: Ellie Chalkley

Ellie Chalkley is an all-round music, media and culture enthusiast and citizen of the internet. As an overly analytical pop fan and general knowledge hoarder she finds the Eurovision Song Contest bubble to be her natural home. She comments gnomically and statistically on Eurovision matters at @ellie_made.

Read more from this author...

You Can Support ESC Insight on Patreon

ESC Insight's Patreon page is now live; click here to see what it's all about, and how you can get involved and directly support our coverage of your Eurovision Song Contest.

Share This Post

If You Like This...

Have Your Say

9 responses to “Improving Eurovision’s Jury System, Examining The Significant Change To Jury Scoring”

  1. Ben says:

    “As such the full ranking method means that any song placed outside a juror’s first ten is effectively given a negative vote”

    That’s not quite true. It’s still better to be ranked 11th than 26th by a juror – and a country that got 11th from every juror would be quite likely to end up getting points!

  2. Ben says:

    This is a great idea though. It means songs that are divisive will be punished less.

  3. Shai says:

    I actually like this change.
    It makes the jury vote slightly less predictable and give every song a chance.

    On the other hand, publishing this 2 days before tet start of the rehearsals is a bit late qua timming

  4. […] you’re eager to learn more about how exactly the new system will work in practice, the ESC Insight team has created a number of simulations based on the available jury scores from previous […]

  5. O says:

    Actually this explanation with real data, is harder to understand than only the text.. I can’t understand it..

  6. Catriona Colville says:

    Divisive songs take a risk. That is their downside. I feel like rewarding pissing a lot of jurors off isn’t the best way to get a fair result. It feels designed for songs like Israel to flourish. We need both risky and safe songs at the contest. They both have upsides and downsides. If a song elicits such a negative response in a jury member that they put it last, and another juror puts it first. Both of their opinions should be equally valid. That wasn’t the case before, but it certainly won’t be now, either.

  7. […] beyond the top-10. How does it work in practice? Fansite ESC Insight has modelled some examples HERE. ESC Insight believes that this change to the jury scoring system is a “welcome step […]

  8. […] Explaining the New Jury Rules (ESC Insight) […]

  9. Kaisa Krannila says:

    Is the exact mathematical formula for calculating the jury points revealed somewhere? I think it should be!

Leave a Reply