LLM Evaluation Metrics Dashboard HotPotQA

Metrics Distribution

95% confidence interval for all the metrics

Detailed Explanations

correctness Details

Question	Answer	Golden Answer	Reason	Score
In what year was the university where Sergei Aleksandrovich Tokarev was a professor founded?	1755	1755	The actual output matches the expected output perfectly, with no contradictions or omissions.	1.0
Black Book starred the actress and writer of what heritage?	dutch	dutch	The actual output 'dutch' matches exactly with the expected output 'dutch', with no contradictions or omissions.	1.0
Which actor does American Beauty and American Beauty have in common?	kevin spacey	kevin spacey	The actual output matches the expected output perfectly without any contradictions or omissions.	1.0
Ken Pruitt was a Republican member of an upper house of the legislature with how many members ?	40	40 members	Actual Output provides the correct number but omits the detail of 'members' as indicated in the Expected Output.	0.6922789279063657
Between Greyia and Calibanus, which genus contains more species?	greyia	greyia	The actual output matches the expected output perfectly, with no contradictions or omissions.	1.0
Did John Updike and Tom Clancy both publish more than 15 bestselling novels?	yes	yes	The actual output matches the expected output without any contradictions or omissions.	1.0
Who was hung for assisting the attempted surrender of a defector from the American Continental Army to the British Army?	john andré	john andré	The actual output matches the expected output exactly, with no contradictions or omissions.	1.0
which Mexican and American film actress is Ethel Houbiers French voice of	salma hayek	salma hayek pinault	The actual output provides the correct name 'Salma Hayek', but omits the full name 'Salma Hayek Pinault' as required in the expected output, demonstrating a lack of detail.	0.6644437364519001
Which major international airport in south-east England ranks as the 8th busiest airport in Europe and replaced Croydon Airport?	gatwick airport	gatwick airport	The actual output matches the expected output without any contradiction, and it correctly identifies Gatwick Airport as the answer.	1.0
Isabella Kelly was born at a ruined castle characterized as one of the most isolated fortifications in Britain by who?	the changing scottish landscape	the changing scottish landscape	The actual output matches the expected output with no contradictory facts or omissions.	1.0
The facility where Robert Croft worked as a navy instructor was also known as?	us naval submarine base new london	"home of the submarine force"	The actual output names a location but does not match the specific expected output description as 'home of the submarine force' and lacks relevant detail.	0.22521734620150613
Who released the song "With or Without You" first, Jai McDowall or U2?	u2	u2	The actual output correctly states 'u2', which matches the expected output 'u2' without any contradictions.	1.0
Which "Roseanne" star is in Scream 2?	laurie metcalf	laurie metcalf	The actual output matches the expected output with no contradictions or omissions.	1.0
In what city is the company that Fastjet Tanzania was originally founded as a part of prior to rebranding based?	dar es salaam	nairobi, kenya	The actual output states 'dar es salaam', which contradicts the expected output of 'nairobi, kenya', showing a clear factual error.	0.0
What is the name of the Australian specialist electronic music magazine that features avant-rock, experimental sound art, and experimental hip hop?	cyclic defrost	cyclic defrost	The actual output perfectly matches the expected output without any contradictions or omissions.	1.0
What is the population of the city that Munsonville is in the northwest corner of?	nelson 729	729 at the 2010 census	The actual output provides the population figure but omits the context of it being the 2010 census, which is a key detail in the expected output.	0.4231998778375853
A Pair of Brown Eyes and Wild Mountain Thyme is based from what artists song?	a pair of brown eyes and wild mountain thyme are based on "wild mountain thyme" by francis mcpeake	francis mcpeake	The actual output correctly identifies the song as 'wild mountain thyme' by francis mcpeake but does not mention the specific work 'A Pair of Brown Eyes.' There is a factual basis, but it lacks clarity and omits key details.	0.4424064705467069
Armageddon in Retrospect was written by the author who was best known for what 1969 satire novel?	slaughterhouse-five	slaughterhouse-five	The actual output 'slaughterhouse-five' matches the expected output perfectly without any contradictions or omissions.	1.0
What class of instrument does Apatim Majumdar play?	sarod	strings	The actual output states 'sarod,' which is correct as a type of string instrument, but it does not explicitly categorize it under 'strings,' leading to a lack of detail and clarification.	0.4473977657931042
Which movie did Disney produce first, The Many Adventures of Winnie the Pooh or Ride a Wild Pony?	ride a wild pony 1975	ride a wild pony	The actual output correctly identifies 'Ride a Wild Pony' as the movie but includes an unnecessary date, which mildly detracts from clarity and conciseness.	0.8405181723360018
Pandikona and Berger Blanc Suisse are both what kinds of animals?	dog breeds	dogs	The actual output identifies Pandikona and Berger Blanc Suisse as dog breeds, aligning with the expected output that refers to them as dogs, but lacks precision.	0.8130113545262743
The Prussian General Carl von Clausewitz is associated with what type of realism?	classical realism	modern thinkers associated with classical realism are carl von clausewitz	The actual output correctly identifies Clausewitz with classical realism, but it fails to acknowledge the detail in the expected output regarding modern thinkers, leading to partial omission of key information.	0.4716620699911084
What where both Hawker Hurricane and No. 1455 Flight apart of?	raf tangmere	royal air force	The actual output 'raf tangmere' is related to the expected output 'royal air force', but it omits crucial details about the connection and lacks specificity.	0.405636685920513
what language did the ethnic group which Torstein Ellingsen was its drumer speaks	norwegian	norwegian language	The actual output correctly identifies the language as 'norwegian', but it omits the detail 'language' present in the expected output, which leads to a moderate penalty.	0.6647464076733561
From March 631 to April 631, Farrukhzad Khosrau V was the king of an empire that succeeded which empire?	parthian empire	the parthian empire	The actual output provides the correct answer, but it omits the definite article 'the', which is mentioned in the expected output, resulting in a minor detail loss.	0.8966914022315103
Beer Wars covers the differences between large corporate breweries, and small breweries, such as what brewery that is headquartered in Escondido, california?	stone brewing co	stone brewing	The actual output correctly identifies the brewery, but it omits 'co' which is present in the expected output, resulting in a minor detail omission.	0.775797249910536
Which head coach has led their team for a longer period of time, Tim Cluess or Steve Prohm?	tim cluess	tim cluess	The actual output matches the expected output precisely, providing a correct answer without contradictions or omissions.	0.9557469639735231
During what war were the Russia-United Kingdom relations in a state of rivalry after the abdication of Emperor Nicholas II?	world war i	the cold war (1947–91)	The actual output states 'world war i', which contradicts the expected output of 'the cold war (1947–91)' and omits key details regarding the rivalry context.	0.0
How far from Sacramento is the flight school in Atwater?	115 miles	about 115 miles (185 km)	The actual output provides the distance in miles, which matches the expected output, but lacks the conversion to kilometers, which is a significant detail.	0.7052527271083477
Baraki Barak District is situated in the western part of a province whose capital is what?	pul i alam	puli alam	The actual output has a minor spelling error ('pul i alam' instead of 'puli alam'), but it does not contradict any facts from the expected output.	0.8727920508876854
What was the 2010 population of the town where Black Crescent Mountain was located?	randolph	310	The actual output 'randolph' does not provide the correct numerical population figure expected in the output, which is '310'. This represents a significant omission of detail.	0.1784009375729015
In the NASA mission where Moon trees were taken into space, what was the nickname of the Command Module?	kitty hawk	"kitty hawk"	The actual output 'kitty hawk' matches the expected output exactly, indicating no contradictions or omissions.	1.0
Which comic series involves characters such as Nick Fury and Baron von Strucker?	nick fury vs. s.h.i.e.l.d	marvel	The actual output refers to a specific title 'nick fury vs. s.h.i.e.l.d', which does not directly indicate the broader comic series 'marvel' as expected, showing a significant omission of detail.	0.18194731215603865
College Humor is a 1933 American pre-Code musical comedy film that starred what American singer and actor who has a trademark warm bass-baritone voice?	bing crosby	harry lillis "bing" crosby jr.	While 'bing crosby' is the correct answer, it omits his full name 'harry lillis' and title 'jr.', which is a significant omission of detail.	0.41193391554262354
Who is writing a book about the Koch family who control the second-largest privately owned company in the United States?	jane mayer	jane mayer	The actual output matches the expected output exactly with no contradictions or omissions.	1.0
New York State Route 9R rejoins its parent in a hamlet located in what New York County?	albany	albany	The actual output matches the expected output perfectly without any contradiction or omission.	1.0
12 Years a Slave starred what British actor born 10 July 1977)	chiwetel ejiofor	chiwetel ejiofor	The actual output matches the expected output exactly, with no contradictions or omissions.	1.0
What was the capital of India when the Taj Mahal was commissioned?	agra	agra	The actual output 'agra' matches the expected output 'agra' exactly, with no contradictions or omissions.	1.0
In what city did the "Prince of tenors" star in a film based on an opera by Giacomo Puccini?	rome	rome	The actual output matches the expected output without contradiction and contains the necessary detail.	1.0
What river can a large African bovine be seen bathing in in the Mwabvi Wildlife Reserve?	mwabvi river	buffalo	The actual output provides the name of the river but omits the key detail about the buffalo, which is essential to the user's query.	0.18569755172804198
Jalen Jones plays basketball for an NBA team that plays their home games in what arena?	smoothie kings center	smoothie king center	The actual output contains a minor error in the naming of the arena, stating 'smoothie kings center' instead of 'smoothie king center', indicating a slight detail omission.	0.8916081051242589
The On Tour Forever album gave Blues Traveler the opportunity to display what musical trademark?	jam band sound	extensive use of segues	The actual output, 'jam band sound,' does not contradict the expected output, 'extensive use of segues,' but lacks detail and specificity, resulting in a heavy penalty for omission.	0.2707271051370691
Which Victorian poet was born in a 15th-century castle home to the Earl of Southesk?	lady charlotte elliot	charlotte carnegie	The actual output 'lady charlotte elliot' completely contradicts the expected output 'charlotte carnegie', indicating a significant factual error.	0.08795796398220011
Alexander Petrovich Nikolayev received the title Hero of the Soviet Union and the Order of Lenin for his actions during the the final major offensive of the European theatre of what war?	world war ii	world war ii	The actual output matches the expected output exactly, with no contradictions or omissions.	1.0
Where did Cale Gundy's brother play football in college?	oklahoma state	oklahoma state university	The actual output contains the correct university but omits 'university', leading to a minor detail omission.	0.6987802609720581
Who has released more solo albums, Nick Carter or Brady Seals?	nick carter	brady seals	The actual output indicates Nick Carter, while the expected output is Brady Seals, directly contradicting the facts and omitting the necessary detail about the number of solo albums.	0.01590994451606683
Which Istanbul mosque is unique for retaining a Baroque style of architecture, the Bayezid II Mosque or the Nusretiye Mosque?	nusretiye mosque	nusretiye mosque	The actual output matches the expected output without contradictions, addressing the query accurately.	1.0
What university did the last Detroit Pistons player to wear the number retired in honor of a player nicknamed "The Worm" attend?	georgetown university	georgetown university	The actual output matches the expected output exactly, with no contradictions or omissions.	1.0
The Atik Valide Mosque and Valens Aqueduct are found in what country?	turkey	turkey	The actual output correctly matches the expected output with no contradictions or omissions.	1.0
Which of the four US Presidents who have been assinated was shot on the grounds of the Pan-American Exposition at the Temple of Music in Buffalo, New York?	william mckinley	william mckinley	The actual output matches the expected output without contradictions or omissions.	1.0

EM Details

Question	Answer	Golden Answer	Reason	Score
In what year was the university where Sergei Aleksandrovich Tokarev was a professor founded?	1755	1755	Exact match	1.0
Black Book starred the actress and writer of what heritage?	dutch	dutch	Exact match	1.0
Which actor does American Beauty and American Beauty have in common?	kevin spacey	kevin spacey	Exact match	1.0
Ken Pruitt was a Republican member of an upper house of the legislature with how many members ?	40	40 members	Not an exact match	0.0
Between Greyia and Calibanus, which genus contains more species?	greyia	greyia	Exact match	1.0
Did John Updike and Tom Clancy both publish more than 15 bestselling novels?	yes	yes	Exact match	1.0
Who was hung for assisting the attempted surrender of a defector from the American Continental Army to the British Army?	john andré	john andré	Exact match	1.0
which Mexican and American film actress is Ethel Houbiers French voice of	salma hayek	salma hayek pinault	Not an exact match	0.0
Which major international airport in south-east England ranks as the 8th busiest airport in Europe and replaced Croydon Airport?	gatwick airport	gatwick airport	Exact match	1.0
Isabella Kelly was born at a ruined castle characterized as one of the most isolated fortifications in Britain by who?	the changing scottish landscape	the changing scottish landscape	Exact match	1.0
The facility where Robert Croft worked as a navy instructor was also known as?	us naval submarine base new london	"home of the submarine force"	Not an exact match	0.0
Who released the song "With or Without You" first, Jai McDowall or U2?	u2	u2	Exact match	1.0
Which "Roseanne" star is in Scream 2?	laurie metcalf	laurie metcalf	Exact match	1.0
In what city is the company that Fastjet Tanzania was originally founded as a part of prior to rebranding based?	dar es salaam	nairobi, kenya	Not an exact match	0.0
What is the name of the Australian specialist electronic music magazine that features avant-rock, experimental sound art, and experimental hip hop?	cyclic defrost	cyclic defrost	Exact match	1.0
What is the population of the city that Munsonville is in the northwest corner of?	nelson 729	729 at the 2010 census	Not an exact match	0.0
A Pair of Brown Eyes and Wild Mountain Thyme is based from what artists song?	a pair of brown eyes and wild mountain thyme are based on "wild mountain thyme" by francis mcpeake	francis mcpeake	Not an exact match	0.0
Armageddon in Retrospect was written by the author who was best known for what 1969 satire novel?	slaughterhouse-five	slaughterhouse-five	Exact match	1.0
What class of instrument does Apatim Majumdar play?	sarod	strings	Not an exact match	0.0
Which movie did Disney produce first, The Many Adventures of Winnie the Pooh or Ride a Wild Pony?	ride a wild pony 1975	ride a wild pony	Not an exact match	0.0
Pandikona and Berger Blanc Suisse are both what kinds of animals?	dog breeds	dogs	Not an exact match	0.0
The Prussian General Carl von Clausewitz is associated with what type of realism?	classical realism	modern thinkers associated with classical realism are carl von clausewitz	Not an exact match	0.0
What where both Hawker Hurricane and No. 1455 Flight apart of?	raf tangmere	royal air force	Not an exact match	0.0
what language did the ethnic group which Torstein Ellingsen was its drumer speaks	norwegian	norwegian language	Not an exact match	0.0
From March 631 to April 631, Farrukhzad Khosrau V was the king of an empire that succeeded which empire?	parthian empire	the parthian empire	Not an exact match	0.0
Beer Wars covers the differences between large corporate breweries, and small breweries, such as what brewery that is headquartered in Escondido, california?	stone brewing co	stone brewing	Not an exact match	0.0
Which head coach has led their team for a longer period of time, Tim Cluess or Steve Prohm?	tim cluess	tim cluess	Exact match	1.0
During what war were the Russia-United Kingdom relations in a state of rivalry after the abdication of Emperor Nicholas II?	world war i	the cold war (1947–91)	Not an exact match	0.0
How far from Sacramento is the flight school in Atwater?	115 miles	about 115 miles (185 km)	Not an exact match	0.0
Baraki Barak District is situated in the western part of a province whose capital is what?	pul i alam	puli alam	Not an exact match	0.0
What was the 2010 population of the town where Black Crescent Mountain was located?	randolph	310	Not an exact match	0.0
In the NASA mission where Moon trees were taken into space, what was the nickname of the Command Module?	kitty hawk	"kitty hawk"	Not an exact match	0.0
Which comic series involves characters such as Nick Fury and Baron von Strucker?	nick fury vs. s.h.i.e.l.d	marvel	Not an exact match	0.0
College Humor is a 1933 American pre-Code musical comedy film that starred what American singer and actor who has a trademark warm bass-baritone voice?	bing crosby	harry lillis "bing" crosby jr.	Not an exact match	0.0
Who is writing a book about the Koch family who control the second-largest privately owned company in the United States?	jane mayer	jane mayer	Exact match	1.0
New York State Route 9R rejoins its parent in a hamlet located in what New York County?	albany	albany	Exact match	1.0
12 Years a Slave starred what British actor born 10 July 1977)	chiwetel ejiofor	chiwetel ejiofor	Exact match	1.0
What was the capital of India when the Taj Mahal was commissioned?	agra	agra	Exact match	1.0
In what city did the "Prince of tenors" star in a film based on an opera by Giacomo Puccini?	rome	rome	Exact match	1.0
What river can a large African bovine be seen bathing in in the Mwabvi Wildlife Reserve?	mwabvi river	buffalo	Not an exact match	0.0
Jalen Jones plays basketball for an NBA team that plays their home games in what arena?	smoothie kings center	smoothie king center	Not an exact match	0.0
The On Tour Forever album gave Blues Traveler the opportunity to display what musical trademark?	jam band sound	extensive use of segues	Not an exact match	0.0
Which Victorian poet was born in a 15th-century castle home to the Earl of Southesk?	lady charlotte elliot	charlotte carnegie	Not an exact match	0.0
Alexander Petrovich Nikolayev received the title Hero of the Soviet Union and the Order of Lenin for his actions during the the final major offensive of the European theatre of what war?	world war ii	world war ii	Exact match	1.0
Where did Cale Gundy's brother play football in college?	oklahoma state	oklahoma state university	Not an exact match	0.0
Who has released more solo albums, Nick Carter or Brady Seals?	nick carter	brady seals	Not an exact match	0.0
Which Istanbul mosque is unique for retaining a Baroque style of architecture, the Bayezid II Mosque or the Nusretiye Mosque?	nusretiye mosque	nusretiye mosque	Exact match	1.0
What university did the last Detroit Pistons player to wear the number retired in honor of a player nicknamed "The Worm" attend?	georgetown university	georgetown university	Exact match	1.0
The Atik Valide Mosque and Valens Aqueduct are found in what country?	turkey	turkey	Exact match	1.0
Which of the four US Presidents who have been assinated was shot on the grounds of the Pan-American Exposition at the Temple of Music in Buffalo, New York?	william mckinley	william mckinley	Exact match	1.0

f1 Details

Question	Answer	Golden Answer	Reason	Score
In what year was the university where Sergei Aleksandrovich Tokarev was a professor founded?	1755	1755	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Black Book starred the actress and writer of what heritage?	dutch	dutch	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Which actor does American Beauty and American Beauty have in common?	kevin spacey	kevin spacey	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Ken Pruitt was a Republican member of an upper house of the legislature with how many members ?	40	40 members	F1: 0.67 (Precision: 1.00, Recall: 0.50)	0.6666666666666666
Between Greyia and Calibanus, which genus contains more species?	greyia	greyia	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Did John Updike and Tom Clancy both publish more than 15 bestselling novels?	yes	yes	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Who was hung for assisting the attempted surrender of a defector from the American Continental Army to the British Army?	john andré	john andré	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
which Mexican and American film actress is Ethel Houbiers French voice of	salma hayek	salma hayek pinault	F1: 0.80 (Precision: 1.00, Recall: 0.67)	0.8
Which major international airport in south-east England ranks as the 8th busiest airport in Europe and replaced Croydon Airport?	gatwick airport	gatwick airport	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Isabella Kelly was born at a ruined castle characterized as one of the most isolated fortifications in Britain by who?	the changing scottish landscape	the changing scottish landscape	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
The facility where Robert Croft worked as a navy instructor was also known as?	us naval submarine base new london	"home of the submarine force"	F1: 0.18 (Precision: 0.17, Recall: 0.20)	0.1818181818181818
Who released the song "With or Without You" first, Jai McDowall or U2?	u2	u2	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Which "Roseanne" star is in Scream 2?	laurie metcalf	laurie metcalf	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
In what city is the company that Fastjet Tanzania was originally founded as a part of prior to rebranding based?	dar es salaam	nairobi, kenya	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
What is the name of the Australian specialist electronic music magazine that features avant-rock, experimental sound art, and experimental hip hop?	cyclic defrost	cyclic defrost	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
What is the population of the city that Munsonville is in the northwest corner of?	nelson 729	729 at the 2010 census	F1: 0.29 (Precision: 0.50, Recall: 0.20)	0.28571428571428575
A Pair of Brown Eyes and Wild Mountain Thyme is based from what artists song?	a pair of brown eyes and wild mountain thyme are based on "wild mountain thyme" by francis mcpeake	francis mcpeake	F1: 0.20 (Precision: 0.11, Recall: 1.00)	0.19999999999999998
Armageddon in Retrospect was written by the author who was best known for what 1969 satire novel?	slaughterhouse-five	slaughterhouse-five	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
What class of instrument does Apatim Majumdar play?	sarod	strings	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
Which movie did Disney produce first, The Many Adventures of Winnie the Pooh or Ride a Wild Pony?	ride a wild pony 1975	ride a wild pony	F1: 0.89 (Precision: 0.80, Recall: 1.00)	0.888888888888889
Pandikona and Berger Blanc Suisse are both what kinds of animals?	dog breeds	dogs	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
The Prussian General Carl von Clausewitz is associated with what type of realism?	classical realism	modern thinkers associated with classical realism are carl von clausewitz	F1: 0.33 (Precision: 1.00, Recall: 0.20)	0.33333333333333337
What where both Hawker Hurricane and No. 1455 Flight apart of?	raf tangmere	royal air force	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
what language did the ethnic group which Torstein Ellingsen was its drumer speaks	norwegian	norwegian language	F1: 0.67 (Precision: 1.00, Recall: 0.50)	0.6666666666666666
From March 631 to April 631, Farrukhzad Khosrau V was the king of an empire that succeeded which empire?	parthian empire	the parthian empire	F1: 0.80 (Precision: 1.00, Recall: 0.67)	0.8
Beer Wars covers the differences between large corporate breweries, and small breweries, such as what brewery that is headquartered in Escondido, california?	stone brewing co	stone brewing	F1: 0.80 (Precision: 0.67, Recall: 1.00)	0.8
Which head coach has led their team for a longer period of time, Tim Cluess or Steve Prohm?	tim cluess	tim cluess	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
During what war were the Russia-United Kingdom relations in a state of rivalry after the abdication of Emperor Nicholas II?	world war i	the cold war (1947–91)	F1: 0.29 (Precision: 0.33, Recall: 0.25)	0.28571428571428575
How far from Sacramento is the flight school in Atwater?	115 miles	about 115 miles (185 km)	F1: 0.57 (Precision: 1.00, Recall: 0.40)	0.5714285714285715
Baraki Barak District is situated in the western part of a province whose capital is what?	pul i alam	puli alam	F1: 0.40 (Precision: 0.33, Recall: 0.50)	0.4
What was the 2010 population of the town where Black Crescent Mountain was located?	randolph	310	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
In the NASA mission where Moon trees were taken into space, what was the nickname of the Command Module?	kitty hawk	"kitty hawk"	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Which comic series involves characters such as Nick Fury and Baron von Strucker?	nick fury vs. s.h.i.e.l.d	marvel	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
College Humor is a 1933 American pre-Code musical comedy film that starred what American singer and actor who has a trademark warm bass-baritone voice?	bing crosby	harry lillis "bing" crosby jr.	F1: 0.57 (Precision: 1.00, Recall: 0.40)	0.5714285714285715
Who is writing a book about the Koch family who control the second-largest privately owned company in the United States?	jane mayer	jane mayer	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
New York State Route 9R rejoins its parent in a hamlet located in what New York County?	albany	albany	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
12 Years a Slave starred what British actor born 10 July 1977)	chiwetel ejiofor	chiwetel ejiofor	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
What was the capital of India when the Taj Mahal was commissioned?	agra	agra	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
In what city did the "Prince of tenors" star in a film based on an opera by Giacomo Puccini?	rome	rome	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
What river can a large African bovine be seen bathing in in the Mwabvi Wildlife Reserve?	mwabvi river	buffalo	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
Jalen Jones plays basketball for an NBA team that plays their home games in what arena?	smoothie kings center	smoothie king center	F1: 0.67 (Precision: 0.67, Recall: 0.67)	0.6666666666666666
The On Tour Forever album gave Blues Traveler the opportunity to display what musical trademark?	jam band sound	extensive use of segues	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
Which Victorian poet was born in a 15th-century castle home to the Earl of Southesk?	lady charlotte elliot	charlotte carnegie	F1: 0.40 (Precision: 0.33, Recall: 0.50)	0.4
Alexander Petrovich Nikolayev received the title Hero of the Soviet Union and the Order of Lenin for his actions during the the final major offensive of the European theatre of what war?	world war ii	world war ii	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Where did Cale Gundy's brother play football in college?	oklahoma state	oklahoma state university	F1: 0.80 (Precision: 1.00, Recall: 0.67)	0.8
Who has released more solo albums, Nick Carter or Brady Seals?	nick carter	brady seals	F1: 0.00 (Precision: 0.00, Recall: 0.00)	0.0
Which Istanbul mosque is unique for retaining a Baroque style of architecture, the Bayezid II Mosque or the Nusretiye Mosque?	nusretiye mosque	nusretiye mosque	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
What university did the last Detroit Pistons player to wear the number retired in honor of a player nicknamed "The Worm" attend?	georgetown university	georgetown university	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
The Atik Valide Mosque and Valens Aqueduct are found in what country?	turkey	turkey	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0
Which of the four US Presidents who have been assinated was shot on the grounds of the Pan-American Exposition at the Temple of Music in Buffalo, New York?	william mckinley	william mckinley	F1: 1.00 (Precision: 1.00, Recall: 1.00)	1.0

Human Score Details

Question	Answer	Golden Answer	Reason	Score
In what year was the university where Sergei Aleksandrovich Tokarev was a professor founded?	1755	1755	Correct answer	1.0
Black Book starred the actress and writer of what heritage?	dutch	dutch	Correct answer	1.0
Which actor does American Beauty and American Beauty have in common?	kevin spacey	kevin spacey	Correct answer	1.0
Ken Pruitt was a Republican member of an upper house of the legislature with how many members ?	40	40 members	Correct answer	1.0
Between Greyia and Calibanus, which genus contains more species?	greyia	greyia	Correct answer	1.0
Did John Updike and Tom Clancy both publish more than 15 bestselling novels?	yes	yes	Correct answer	1.0
Who was hung for assisting the attempted surrender of a defector from the American Continental Army to the British Army?	john andré	john andré	Correct answer	1.0
which Mexican and American film actress is Ethel Houbiers French voice of	salma hayek	salma hayek pinault	Correct answer (pinault is actually her husband's name since 2009)	1.0
Which major international airport in south-east England ranks as the 8th busiest airport in Europe and replaced Croydon Airport?	gatwick airport	gatwick airport	Correct answer	1.0
Isabella Kelly was born at a ruined castle characterized as one of the most isolated fortifications in Britain by who?	the changing scottish landscape	the changing scottish landscape	Correct answer	1.0
The facility where Robert Croft worked as a navy instructor was also known as?	us naval submarine base new london	"home of the submarine force"	Correct answer (actually cognee gives the exact and correct submarine base instead of the general golden answer)	1.0
Who released the song "With or Without You" first, Jai McDowall or U2?	u2	u2	Correct answer	1.0
Which "Roseanne" star is in Scream 2?	laurie metcalf	laurie metcalf	Correct answer	1.0
In what city is the company that Fastjet Tanzania was originally founded as a part of prior to rebranding based?	dar es salaam	nairobi, kenya	Correct answer (based on wikipedia Fastjet Tanzania was based in Dar es Salaam)	1.0
What is the name of the Australian specialist electronic music magazine that features avant-rock, experimental sound art, and experimental hip hop?	cyclic defrost	cyclic defrost	Correct answer	1.0
What is the population of the city that Munsonville is in the northwest corner of?	nelson 729	729 at the 2010 census	Correct answer (Cognee also gives the name os the city (I dont understand the golden answer...))	1.0
A Pair of Brown Eyes and Wild Mountain Thyme is based from what artists song?	a pair of brown eyes and wild mountain thyme are based on "wild mountain thyme" by francis mcpeake	francis mcpeake	Correct answer (with some additional information))	1.0
Armageddon in Retrospect was written by the author who was best known for what 1969 satire novel?	slaughterhouse-five	slaughterhouse-five	Correct answer	1.0
What class of instrument does Apatim Majumdar play?	sarod	strings	Correct answer (based on wikipedia and his webpage, also I dont understand the golden answer..))	1.0
Which movie did Disney produce first, The Many Adventures of Winnie the Pooh or Ride a Wild Pony?	ride a wild pony 1975	ride a wild pony	Correct answer (cognee provides the year too)	1.0
Pandikona and Berger Blanc Suisse are both what kinds of animals?	dog breeds	dogs	Correct answer	1.0
The Prussian General Carl von Clausewitz is associated with what type of realism?	classical realism	modern thinkers associated with classical realism are carl von clausewitz	Correct answer (sometimes golden answers are short sometimes overexplained?)	1.0
What where both Hawker Hurricane and No. 1455 Flight apart of?	raf tangmere	royal air force	Correct answer (RAF is for Royal Air Force)	1.0
what language did the ethnic group which Torstein Ellingsen was its drumer speaks	norwegian	norwegian language	Correct answer	1.0
From March 631 to April 631, Farrukhzad Khosrau V was the king of an empire that succeeded which empire?	parthian empire	the parthian empire	Correct answer	1.0
Beer Wars covers the differences between large corporate breweries, and small breweries, such as what brewery that is headquartered in Escondido, california?	stone brewing co	stone brewing	Correct answer	1.0
Which head coach has led their team for a longer period of time, Tim Cluess or Steve Prohm?	tim cluess	tim cluess	Correct answer	1.0
During what war were the Russia-United Kingdom relations in a state of rivalry after the abdication of Emperor Nicholas II?	world war i	the cold war (1947–91)	Incorrect (However, based on wikipedia relations were strained during world war i too so debatable)	0
How far from Sacramento is the flight school in Atwater?	115 miles	about 115 miles (185 km)	Correct answer	1.0
Baraki Barak District is situated in the western part of a province whose capital is what?	pul i alam	puli alam	Correct answer	1.0
What was the 2010 population of the town where Black Crescent Mountain was located?	randolph	310	Incorrect answer	0.0
In the NASA mission where Moon trees were taken into space, what was the nickname of the Command Module?	kitty hawk	"kitty hawk"	Correct answer	1.0
Which comic series involves characters such as Nick Fury and Baron von Strucker?	nick fury vs. s.h.i.e.l.d	marvel	I gave 0.5... basically nick fury vs shield is a six issue comic miniseries so I would say its more correct than the golden answer..	0.5
College Humor is a 1933 American pre-Code musical comedy film that starred what American singer and actor who has a trademark warm bass-baritone voice?	bing crosby	harry lillis "bing" crosby jr.	Correct answer (I think its correct, cognee didn't write the full name but the shorter actor's name)	1.0
Who is writing a book about the Koch family who control the second-largest privately owned company in the United States?	jane mayer	jane mayer	Correct answer	1.0
New York State Route 9R rejoins its parent in a hamlet located in what New York County?	albany	albany	Correct answer	1.0
12 Years a Slave starred what British actor born 10 July 1977)	chiwetel ejiofor	chiwetel ejiofor	Correct answer	1.0
What was the capital of India when the Taj Mahal was commissioned?	agra	agra	Correct answer	1.0
In what city did the "Prince of tenors" star in a film based on an opera by Giacomo Puccini?	rome	rome	Correct answer	1.0
What river can a large African bovine be seen bathing in in the Mwabvi Wildlife Reserve?	mwabvi river	buffalo	Incorrect answer, however google result: Buffalo still bathe in the Mwabvi river	0
Jalen Jones plays basketball for an NBA team that plays their home games in what arena?	smoothie kings center	smoothie king center	Correct answer	0
The On Tour Forever album gave Blues Traveler the opportunity to display what musical trademark?	jam band sound	extensive use of segues	Honestly I can't decide... the band is actually a jam band so its not completely false	0
Which Victorian poet was born in a 15th-century castle home to the Earl of Southesk?	lady charlotte elliot	charlotte carnegie	Correct Answer (Cognee actually gives the name, the golden answer is the person's childhood name)	1
Alexander Petrovich Nikolayev received the title Hero of the Soviet Union and the Order of Lenin for his actions during the the final major offensive of the European theatre of what war?	world war ii	world war ii	Correct Answer	1
Where did Cale Gundy's brother play football in college?	oklahoma state	oklahoma state university	Correct Answer	1
Who has released more solo albums, Nick Carter or Brady Seals?	nick carter	brady seals	Incorrect Answer	0
Which Istanbul mosque is unique for retaining a Baroque style of architecture, the Bayezid II Mosque or the Nusretiye Mosque?	nusretiye mosque	nusretiye mosque	Correct Answer	1
What university did the last Detroit Pistons player to wear the number retired in honor of a player nicknamed "The Worm" attend?	georgetown university	georgetown university	Correct Answer	1
The Atik Valide Mosque and Valens Aqueduct are found in what country?	turkey	turkey	Correct Answer	1
Which of the four US Presidents who have been assinated was shot on the grounds of the Pan-American Exposition at the Temple of Music in Buffalo, New York?	william mckinley	william mckinley	Correct Answer	1