We will evaluate by:

1) selecting an article from news

2) paste the article into an LLM along with the question: "based on this article, how confident are you that this event has happened from 0-100%? Please answer by filling in this template: {" text of disaster from list above", probability: X%}

If the average probability across leadingLLMs or the best AI agents at the time, across five tries each, is over 70%, this can resolve true.

So overall we use the LLM to evaluate the claim based on an article. I'm going to try to seek truth as much as possible and not accept a trick or false or unintuitive resolution.

Here is the list of disasters:

Here are 30 extreme hypothetical disaster scenarios for Europe:

1. A massive volcano eruption in Iceland disrupts air travel and weather across Europe

2. A huge earthquake strikes Italy, damaging historical buildings and infrastructure

3. Terrorist groups detonate dirty bombs in major European cities

4. Russia cuts off oil and gas exports to Europe, triggering an energy crisis

5. A cyberattack takes down critical infrastructure like power grids across the EU

6. A global pandemic starts in Europe and spreads rapidly across the continent

7. Coastal cities are flooded after a series of massive storms and surges

8. The Euro currency collapses, plunging Europe into a deep economic depression

9. Civil unrest and riots break out in multiple countries over austerity measures

10. A heat wave kills thousands across southern Europe, especially the elderly

11. Far-right populist parties take power in France, Germany and other nations

12. A meteor strikes Germany, leaving a massive crater and causing damage

13. England faces dire food and medicine shortages after a messy Brexit

14. Refugee crisis overwhelms resources and strains the EU's open borders

15. Scotland and Catalonia secede, destabilizing the UK and Spain

16. Widespread wildfires burn out of control in Portugal and Spain

17. A swarm of locusts infests and damages agriculture crops across southern Europe

18. Nuclear meltdown contaminates large areas in central or eastern Europe

19. Massive flooding inundates historic cities like Venice and Amsterdam

20. A military coup topples governments in Turkey or eastern European nations

21. A new strain of bird flu jumps to humans and ignites a deadly pandemic

22. Belgium splits along ethnic-linguistic lines into two separate states

23. Right-wing nationalists take over the EU parliament and government

24. Italy defaults on its debt, sparking a eurozone banking crisis

25. An armed conflict erupts in the Balkans reigniting old ethnic tensions

26. Poland and Hungary leave the EU, weakening the union

27. A huge storm surge destroys swathes of the Netherlands and low-lying areas

28. Months of extreme winter weather cripple infrastructure across northern Europe

29. Iceland's volcanoes erupt simultaneously disrupting global temperature and weather

30. Undersea volcanoes or an earthquake trigger tsunamis that hit coastal areas