While most of the arena was still in euphoria from Austria’s victory in the Eurovision Song Contest 2014, the European Broadcasting Union (EBU) were true to their word and posted the results from this year’s contest as quickly as possible.  

For the first time ever, the individual ranks of each jury member are available as well as the ranking of the televote from first place to last.  This means that it is now possible to run detailed models into the voting of the Contest.  In the first part of a series we’re calling ‘Voting Insight’, Ben Robertson goes through the results to assess what would have been different had the different voting systems of Eurovision over the last ten years been in place.

Why?

Having all of this data from the EBU means that we can see how Eurovision has changed through each of the evolutions, from televoting only to our current system of ranked jury and televote scores.  Firstly let’s make it clear that Austria this year was a clear runaway victory in whatever system we use, and the margin of that victory changes only slightly with each alteration.

However, our full analysis below brings up surprising twists.  Different songs would qualify for the Grand Final on Saturday night depending on the model used – in different calculations Portugal, Macedonia, and Lithuania would have made it, and San Marino, Slovenia and Malta may be on early flights home.

Also, we are able to assess how different the jury and televote are.  This is the first year where full ranked similarities and differences can be spotted between each of the results of the two seperate groups, and included here is statistically significant evidence that our jury votes are more spread out and unpredictable than our ‘unreliable’ televoters.

We do this because it helps us understand how our Eurovision Song Contest works in the way it does.

Conchita Wurst, Ready To Rise (picture: Ewan Spence)

What Is Different In The Voting Systems

Since the advent of Semi Finals in the Eurovision Song Contest we have seen various systems used for keeping score.  Classically, all the countries used televoting to announce points from the 12 points down to 1 point as has been traditional in the contest for many years.

In 2008 and 2009, the top 9 songs in each Semi-Final qualified through the televote, with one place left available for a jury wildcard, which was the highest ranked song from the juries scores overall combined score  that had not yet qualified.

In 2010 the voting system changed so that each jury and televote would create scores from 12 points down to 1 point, which would then be merged together to make a total 12 to 1 score from that country.

Our current combination system (which has been in place since 2013), has seen each jury and televote rank each song from first place to last place.  These rankings are then combined and the best combination scores 12 points, the next 10 and so forth down to 10th place which gives 1 point from that country.

We take the data provided by the EBU in order to determine how the Eurovision scoreboard would have looked changed  in the different voting systems.

Our Methodology And Our Assumptions

Using the data provided by the EBU, we have been able to extrapolate quite easily the televoting scores from each country.  A song ranked in 1st place with a televote would be given 12 points, second place would receive 10 points and so on.

The calculations of the jury scores for the old system are not quite as easy however.  EBU have calculated average jury rankings for each country, but these can not be used due to the discrepancy that exists with the new ranking system.  In previous years with juries each jury member ranked only their top 10 songs in each show.  Therefore any negative drag effects from low ratings below 10th from individual jury members in the modern day voting system are removed from our calculations, but are included in the average ranks provided by the EBU.

To complete the calculation as fairly as possible, we have made the assumption that the rating from first to last for each country in the jury scores can be used to work out what each individual jury member would give for a top 10.  For example, a first place score from a song would give it 12 points from that juror, a second place score would give 10 points.  For each of the five jury members these are then combined to form the total 1 to 12 from the jury.  This can then be, if appropriate, combined with the televoting scores in our data.  We also refer back to the official rules document from Eurovision 2012 to help in the case of tie-breaks occurring.  In particular we have had to split jury tie-breaks often using this segment of the rule book

“Should two or more songs receive exactly the same number of points from a National Jury, the order shall be ascertained by a show of hands by the members of the National Jury.”

So when two or more songs have tied for points under the old system, we have manually assessed from their ratings of all of the songs the likely voting preferences from each jury member in this scenario.

For each Semi Final and Final below we have calculated.

• The results in the case that only televoting was used
• The results in the case that only televoting was used, and countries that did not meet the threshold used their back-up jury voting from 12 points to 1 point (like methods from 2004 to 2007)
• The results in the case that only jury voting was used using the 12 points to 1 point scale rather than ranking
• The results in the case of juries and televoting split 50/50 with each giving using 12 points to 1 point (the system from 2010 to 2012)
• The average (mean) and the standard deviation (how spread out the results in the full data set are on average away from the mean) of each entry

Semi Final One

If we take the results of Semi Final One on a pure televote, there are derivations from the results announced on the Tuesday night. The Netherlands is still a comfortable winner, 27 points ahead of Hungary, but the big mover is Portugal, moving up to 6th place overall.  San Marino is a comfortable qualifier in 8th place followed by Latvia as our new qualifier in 10th position, 7 points ahead of Belgium.  Montenegro and Azerbaijan would have missed out in this case, most notable would have been Azerbaijan’s televote score of only 15 coming from three countries (8 from Russia, 6 from Ukraine and 1 from The Netherlands).

Televote Semi Final One (including back-up juries)

However, in reality a full televote would not have been possible in Semi Final One, as a total of four countries had to resort to their back up juries due to the lack of televotes cast.  These were San Marino, Albania, Montenegro and Moldova.  Including their jury results into the list of televoting results would give us the actual figures as if we were following the Eurovision voting system for semi finals from 2004 to 2007.  The Netherlands is again still a clear winner, but the success stories in this system go to Azerbaijan and to Montenegro, who each received 27 and 16 points respectively from these four jury groups.  This is enough for both to springboard over Latvia (which received 2 points from these four juries) and to tie on equal 10th place.  Montenegro received one 12 points (from Albania’s jury) compared to Azerbaijan’s zero, so would qualify under the rules of the contest at the time (and they also received points from more countries, so they would qualify under the more recent rule revisions as well).  Spare a thought here for Estonia too, which would be moved to last place in this semi final if the EBU decided to use a 100% televoting system for this year’s contest.

Jury Vote Semi Final One

The Jury Vote in Semi Final One still shows a very strong top 5 which is in agreement with the televoting top 5.  The Netherlands is a clear winner, but Hungary is closer in just 13 points behind.  The clear losers in a jury vote are Portugal, which moves to last place in the semi final ratings, and San Marino which is second last. Russia would miss out on the final in a 100% jury vote, with Estonia taking the 10th and final place five points clear.  Albania takes the second spot vacated from San Marino’s drop in points from the juries.  If the jury vote as here was used for deciding the 10th and final qualifier from the semi finals, as in 2008 and 2009 when the top 9 progressed to the final followed by the jury wildcard, Montenegro would lose out on its place to Azerbaijan, which was a comfortable 6th with jury voting.

Combined 50/50 voting method from Eurovision 2010-2012 for Semi Final One

When we combine together the jury and televote scores for each country, as was done for semi finals between 2010 and 2012, we again find that The Netherlands is a clear winner with 148 points while Azerbaijan, Iceland and Montenegro cement themselves as secure qualifiers. Qualification at the bottom end though is even tighter, with just 6 points separating songs from 10th to 14th.  With virtue to a 7 from France (as opposed to the 5 it received in the current system), Portugal takes the final spot to qualify with 39 points.  This is one point above Latvia, who are themselves two points clear of Estonia on 36.  Belgium collected 35 points and San Marino only 33 and therefore 4 places off qualifying for the Grand Final.

Semi Final One Mean and Standard Deviation Results (Televote)

Semi Final One Mean and Standard Deviation Results (Jury Vote)

There are no great contrasts when we present the average scores in Semi Final One as averages, which is how the EBU chose to present the split results last year.  We can though note that our winner The Netherlands was much more broadly supported by televoters than Emmelie De Forest was last year, her semi final score from televotes and jury were 3.33 and 3.58 respectively, whereas The Netherlands peaked with televoters at a average ranking of 2.79.

Aram MP3… he’s alone here (picture: Ewan Spence)

The standard deviations are a measure of how much variance there is either side of this average score across all of the voting countries.  This will naturally follow lower numbers for songs that are universally ranked highly and lowly.  Nevertheless the high score here of Armenia is notable, showing how Armenia got high points from some countries and low points from others.  Part of this can be seen as the drag effect due to the last place from Azerbaijan’s televote for Aram MP3, however even removing this from the data still gives a score of 3.73.  This would be the third highest spread in the semi final, with only Russia and Azerbaijan splitting opinions of viewers across the continent as much.  With the jury groups it proved to be Iceland who divided their professional opinion the most.

Semi Final Two

In Semi Final Two we are able again to perform the same analysis.  Looking at the qualifying results if they were based purely on televoting, we can see that Austria would still have been a clear winner.  Austria received a minimum of 8 points from all televotes in each semi final, except for only receiving 5 points from Lithuania.  The top 5 in the televotes in this semi final received points from all the other competing nations if we look at televotes alone.  The one change to the qualifiers in this system is that Malta drastically drops out. Malta only received 16 points from televoting, the highest being 4 points from the United Kingdom.  Georgia is still last, but the 15 points they received include an 8 from Lithuania and a 7 from Greece.

Televote Semi Final Two (including back-up juries)

Again however, this does not tell the full pictures because two countries, Georgia and Macedonia, were unable to meet the threshold required for a full televote.  Adding their back-up juries to the scores shows that Lithuania, thanks to 4th place from the Georgian jury, leapfrogs Slovenia for the final place and would have qualified under 2004 to 2007 rules therefore.  Malta received 20 points from these two juries lifting it away from Georgia in last but not enough to qualify. The country that takes Malta’s place in the Eurovision Final, under 100% televoting, would have been Ireland.  Slovenia ends up in 10th place, just one point above Lithuania. Slovenia received points from 9 countries in televoting in contrast to just 6 for Lithuania.

Jury Vote Semi Final Two

Assuming that only juries were voting in the competition sees a shock qualifier in that Macedonia would have easily made it through, with only Lithuania and the UK juries not awarding Tijana points, leaving Macedonia comfortably in 6th place.  Interestingly, last place, shared by Ireland and Israel, is given with a comparitively high 33 points each.

Should the Jury Vote have been used this year to determine the final qualifier like in 2008 and 2009, Malta would have been in the Saturday night show.  Ironically in both 2008 and 2009 Macedonia finished 10th in their semi final but did not get the jury support to go through into the Grand Final, this year they would have had jury support to lift themselves above 10th place Lithuania, but not enough to rise above Malta’s 3rd place jury score.

Combined 50/50 voting method from Eurovision 2010-2012 for Semi Final Two

When these jury scores are combined with the televoting our one change is that Macedonia takes the final spot which is vacated by Slovenia.  Austria flies away with a lowest score of 6 points from Belarus, Macedonia and Germany (in contrast to a 4 Germany gave to Austria in the new system) and qualified by over 100 points.

Semi Final Two Mean and Standard Deviation Results (Televote)

Semi Final Two Mean and Standard Deviation Results (Jury vote)

The average scores for televote suggest Israel was much closer than it actually was.  Israel appears 11th in the televote average list, however because of the low standard deviation of Israel’s points few of these scores got Israel high enough on the scoreboard to make a difference (Israel got ranked 5th from the Finnish televote, but 8 of the countries had Israel marked 11th or 12th).

A similar effect occurred for Malta, where the televotes ranged from a highest of 7th place to a lowest of 13th, and very closely grouped together.  Juries did not have to pull up the song so far to get it to qualify.  In contrast Lithuania in televotes was wildly divergent.  We note that Lithuania received 32 of those 37 points in a televoting only system from Norway, Ireland and the United Kingdom, countries that have a well-known and active Lithuanian diaspora.

TIjana is going To The Departures Lounge (picture: Ewan Spence)

On an aside, note too the average score from Conchita Wurst of 2.13 is the highest ever in a semi final televote, higher than Alexander Rybak’s 2.37 from 2009, although admittedly that was a 19 song semi final as opposed to the 15 song semi that Conchita qualified through.  Also note how Conchita, for a song that won the jury vote, has a remarkably high standard deviation from their scores.  This is further evidence of the polarising effect the Austrian entry had on some jurors, and we note the big difference in standard deviation scores between the jury and the televote.

The Grand Final

Our Grand Final has a different make up if we consider it from a jury only and a televote only perspective, as fellow Insight writer John Egan has already published.

When we consider this in the combined format from 2010 to 2012, with the individual jury rankings into consideration, unsurprisingly there is little change at the top of the leaderboard.  Austria though is able to break the 300 point barrier, as Conchita is less hurt by damagingly low jury scores from some countries.  Our top five overall remain unchanged, although Hungary is much more distant from the pack as previously and Poland takes the 6th place spot as again damaging jury scores are removed in the old combination.

Combined 50/50 method from Eurovision 2010-2012 in the Grand Final

Overall the combination of scores has little impact for most entries in the middle of the scoring.  Azerbaijan is a notable positive mover up to mid-table instead of bottom five however due to low televote positioning.  Azerbaijan was indeed last in the televoting averages this year in the 26 song final.

Grand Final Mean and Standard Deviation Results (Televote)

Grand Final Mean and Standard Deviation Results (Jury Vote)

In terms of the spread of data we can observe how Switzerland was universally well liked around the whole of Europe, no country put ‘Hunter of Stars’ less than 12th place in the final, which is a feat that even The Netherlands, Armenia, Sweden and Hungary can not claim on their entries.  The Romanian song has the highest spread of televote data, which when looking at where the high points come from we can infer is the country that has had the most benefit in the final from diaspora voting.

The juries overall show a much wider spread of results across all of the competing songs.  We can note from the standard deviations that entries from Greece and Russia divided jury opinion the most.  In Greece’s case they were in the top 10 of 11 different jury groups around Europe, but were ranked from 20th to 26th from 17 of the 37 jury groups.  Russia was in the top 10 in 10 different countries with juries, and ranked from 20th to 26th with 13 countries in the contest.

Also in case you missed it, if we use the method EBU used last year to release the jury/televote split then the negative votes Conchita received from certain juries ensured that The Netherlands is the ‘winner’ of the jury vote, their average of 7.06 just beating Conchita’s 7.10.

Overall Conclusions

Each of the methods investigated shows how different voting systems would have created differences in what was played out on stage in the Eurovision Song Contest 2014.  However it appears in all of our results that Austria would be the winner in every voting system used by the Song Contest over the last ten years, and barring some noted exceptions most songs would have seen insignificant benefit or loss depending on the system used.

What this data does note, officially for the first time, is that our juries are less predictable than the televotes.  Comparing the average standard deviations of juries and televotes across Europe in all three shows, we clearly see that the juries are significantly more spread across the continent in their voting compared to viewers at home.  With a sample size of only five jurors in each country, compared to a huge televote, this is of course no big surprise.

What is notable though is the spread of results from jurors for numerous songs is larger than even those countries we hypothesise benefitted most from a huge diaspora.  In the Grand Final only Romania and Slovenia appeared to have bigger spreads from televoters than with jury points.

Stay tuned to ESC Insight for further analysis of the voting statistics.  Episode Two will investigate the impact running order has on the Eurovision Song Contest.

Many thanks go to Alison Wren and the students studying Eurovision at Internationella Engelska Skolan i Täby for their help in data analysis.  Many calculations were made by hand and although they have been checked please forward any irregularities to Ben via ben@escinsight.com