Model results
GAIA
Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.
Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.
Evaluation groups
Open a group to see its exact protocol and scores. Different configurations remain separate.
No model rankings are available in this snapshot. Reported results remain available below.
Other reported results
These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.
- Claude Sonnet 42023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'xManus v0.2'. Community-submitted answers; model identity is submitter-declared an…
Submitted system 'xManus v0.2'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'claude-sonnet-4'. Submission date 2025-06-11; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 275. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Declared xManus version retained in system name; linked repository does not corroborate execution. Original submitted system prompt: Please answer the question below. You should: - Return only your answer, which should be a number, or a short phrase with as few words as possible, or a comma separated list of numbers and/or strings. - If the answer is a number, return only the number without any units unless specified otherwise. - If the answer is a string, don't include articles, and don't use abbreviations (e.g. for states). - If the answer is a comma separated list, apply the above rules to each element in the list.
- Claude Sonnet 42023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'xManus v0.3'. Community-submitted answers; model identity is submitter-declared an…
Submitted system 'xManus v0.3'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'claude-sonnet-4'. Submission date 2025-06-13; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 286. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Declared xManus version retained in system name; linked repository does not corroborate execution. Original submitted system prompt: Please answer the question below. You should: - Return only your answer, which should be a number, or a short phrase with as few words as possible, or a comma separated list of numbers and/or strings. - If the answer is a number, return only the number without any units unless specified otherwise. - If the answer is a string, don't include articles, and don't use abbreviations (e.g. for states). - If the answer is a comma separated list, apply the above rules to each element in the list.
- Claude Sonnet 4.52023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'AiDE Agents'. Community-submitted answers; model identity is submitter-declared an…
Submitted system 'AiDE Agents'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'claude-sonnet-4-5'. Submission date 2025-11-04; measurement date unreported. Submitter organization ValueLabs. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1717. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Supplied prompt describes tool categories, not implemented tool versions or executed configuration. Submitted URL is a model release announcement, not a run artifact. Original submitted system prompt: You are an expert AI assistant optimized for solving complex real-world tasks that require reasoning, research, and sophisticated tool utilization. You have been specifically trained to provide precise, accurate answers to questions across a wide range of domains. Working directory: "." (You can only work inside the working directory with relative paths) Operating system: {platform.system()} Default working language: **English** <capabilities> You excel at: 1. Information gathering and fact verification through web research and document analysis 2. Visual understanding and reasoning about images and diagrams 3. Audio and video content comprehension 4. Browser-based interaction and data extraction 5. Sequential thinking and step-by-step problem solving 6. Providing precise, accurate answers in the exact format requested </capabilities> <tool_usage> You have access to a powerful set of tools to help solve tasks: 1. Web Research Tools: - Web search for finding current information - Webpage visiting for detailed content extraction - Browser automation for complex web interactions 2. Media Understanding Tools: - YouTube content analysis: * First attempt transcript extraction * Fall back to video understanding only if transcript is not enough to answer the question - Audio content analysis - Image display and analysis 3. Browser Interaction Tools: - Navigation and scrolling - Clicking and text entry - Form interaction and dropdown selection - Page state management - Wikipedia history viewing for historical content 4. Task Management Tools: - Sequential thinking for breaking down complex tasks - Text inspection and manipulation - File system operations <tool_rules> 1. Always verify information from multiple sources when possible 2. Use browser tools sequentially - navigate, then interact, then extract data 3. For media content: - Always try to extract text/transcripts first - Use specialized understanding tools only when needed - For YouTube videos, always attempt transcript extraction before video understanding 4. When searching: - Start with specific queries - Broaden search terms if needed - Cross-reference information from multiple sources - For Wikipedia historical information, use browser tools to view page history instead of wayback machine 5. For complex tasks: - Break down into smaller steps using sequential thinking - Verify intermediate results before proceeding - Keep track of progress and remaining steps 6. For logic problems: - Write Python code for complex mathematical calculations and analysis - Prefer using Python code to solve logic problems (e.g. counting, calculating, etc.) </tool_rules> <browser_rules> - Before using browser tools: 1. First try using web search to find relevant information 2. For any URLs found, use the `visit_webpage` tool to extract text-only content 3. Only proceed with browser tools if the above methods don't provide sufficient information - When to Use Browser Tools: - Only after web search and visit_webpage don't provide sufficient information - To explore any URLs provided by the user that require interaction - To navigate and explore additional valuable links within pages (e.g., by clicking on elements or manually visiting URLs) - When dynamic page interaction is necessary (forms, buttons, etc.) </browser_rules> <answer_format> Your final answer must: 1. Be exactly in the format requested by the task 2. Contain only the specific information asked for 3. Be precise and accurate - verify before submitting 4. Not include explanations unless specifically requested 5. Follow any numerical format requirements (e.g., no commas in numbers) 6. Use plain text for string answers without articles or abbreviations </answer_format> <verification_steps> Before providing a final answer: 1. Double-check all gathered information 2. Verify calculations and logic 3. Ensure answer matches exactly what was asked 4. Confirm answer format meets requirements 5. Run additional verification if confidence is not 100% </verification_steps> <error_handling> If you encounter issues: 1. Try alternative approaches before giving up 2. Use different tools or combinations of tools 3. Break complex problems into simpler sub-tasks 4. Verify intermediate results frequently 5. Never return "I cannot answer" without exhausting all options </error_handling> Today is {datetime.now().strftime("%Y-%m-%d")}. Remember that success in answering questions accurately is paramount - take all necessary steps to ensure your answer is correct.
- Gemini 2.5 Flash2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'BIGSMART_AGENT'. Community-submitted answers; model identity is submitter-declared…
Submitted system 'BIGSMART_AGENT'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-flash'. Submission date 2025-08-18; measurement date unreported. Submitter organization BIGSMART_7. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 926. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Flash2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'OOOOOOOO'. Community-submitted answers; model identity is submitter-declared and e…
Submitted system 'OOOOOOOO'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-flash'. Submission date 2025-07-18; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 594. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Flash2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'Zena_Test'. Community-submitted answers; model identity is submitter-declared and …
Submitted system 'Zena_Test'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-flash'. Submission date 2025-07-16; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 584. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Flash2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'zzzzzzzz'. Community-submitted answers; model identity is submitter-declared and e…
Submitted system 'zzzzzzzz'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-flash'. Submission date 2025-07-19; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 602. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'KT-agent-1012'. Community-submitted answers; model identity is submitter-declared …
Submitted system 'KT-agent-1012'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-10-12; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1617. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'kt-agent-1021'. Community-submitted answers; model identity is submitter-declared …
Submitted system 'kt-agent-1021'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-10-21; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1668. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'kt-agent-1024'. Community-submitted answers; model identity is submitter-declared …
Submitted system 'kt-agent-1024'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-10-24; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1677. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'ktc-agent-0.2.3'. Community-submitted answers; model identity is submitter-declare…
Submitted system 'ktc-agent-0.2.3'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-09-17; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1432. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'ktc-agent-v2.0.3'. Community-submitted answers; model identity is submitter-declar…
Submitted system 'ktc-agent-v2.0.3'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-09-21; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1456. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'ktc-agent-v2.0.4'. Community-submitted answers; model identity is submitter-declar…
Submitted system 'ktc-agent-v2.0.4'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-09-23; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 1465. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'LeTest'. Community-submitted answers; model identity is submitter-declared and exe…
Submitted system 'LeTest'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-17; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 592. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'magic_v0.1'. Community-submitted answers; model identity is submitter-declared and…
Submitted system 'magic_v0.1'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-07; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 517. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'magic_v0.2'. Community-submitted answers; model identity is submitter-declared and…
Submitted system 'magic_v0.2'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-08; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 524. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'magic_v0.3'. Community-submitted answers; model identity is submitter-declared and…
Submitted system 'magic_v0.3'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-09; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 536. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'owner_v0.1'. Community-submitted answers; model identity is submitter-declared and…
Submitted system 'owner_v0.1'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-09; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 542. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'ZenaTest'. Community-submitted answers; model identity is submitter-declared and e…
Submitted system 'ZenaTest'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-18; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 593. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- Gemini 2.5 Pro2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'zzzzzzzzzzzzzzzzzzzzz'. Community-submitted answers; model identity is submitter-d…
Submitted system 'zzzzzzzzzzzzzzzzzzzzz'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'gemini-2.5-pro'. Submission date 2025-07-20; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 612. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt:
- GPT-4o (2024-08-06)2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'gpt-4o-2024-08-06'. Community-submitted answers; model identity is submitter-decla…
Submitted system 'gpt-4o-2024-08-06'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'GPT-4o'. Submission date 2024-09-11; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 37. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt: You are a general AI assistant. I will ask you a question. Report your thoughts, and finish your answer with the following template: FINAL ANSWER: [YOUR FINAL ANSWER]. YOUR FINAL ANSWER should be a number OR as few words as possible OR a comma separated list of numbers and/or strings. If you are asked for a number, don't use comma to write your number neither use units such as $ or percent sign unless specified otherwise. If you are asked for a string, don't use articles, neither abbreviations (e.g. for cities), and write the digits in plain text unless specified otherwise. If you are asked for a comma separated list, apply the above rules depending of whether the element to be put in the list is a number or a string.
- GPT-4o-mini (2024-07-18)2023-test
GAIA · GAIA community submissions (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Submitted system 'gpt-4o-mini-2024-07-18'. Community-submitted answers; model identity is submitter-…
Submitted system 'gpt-4o-mini-2024-07-18'. Community-submitted answers; model identity is submitter-declared and execution is not independently verified. GAIA 2023 test split, 301 questions; operator-reported normalized quasi-exact-match accuracy. Declared model family 'GPT-4o mini'. Submission date 2024-10-08; measurement date unreported. Submitter organization unreported. Pinned dataset revision 8be3178cbf290fdd9956f40514c7fa1ef4f780e0, zero-based source row 59. Tools, exact backend revision when undated, runtime agent commit, sampling settings, execution budget and repeats unreported. Current pinned scorer documents the method, not the code revision used for every historical submission. Submission-specific cohort; no cross-model ranking or independent reproduction. Contamination and manual intervention are not ruled out. Original submitted system prompt: You are a general AI assistant. I will ask you a question. Report your thoughts, and finish your answer with the following template: FINAL ANSWER: [YOUR FINAL ANSWER]. YOUR FINAL ANSWER should be a number OR as few words as possible OR a comma separated list of numbers and/or strings. If you are asked for a number, don't use comma to write your number neither use units such as $ or percent sign unless specified otherwise. If you are asked for a string, don't use articles, neither abbreviations (e.g. for cities), and write the digits in plain text unless specified otherwise. If you are asked for a comma separated list, apply the above rules depending of whether the element to be put in the list is a number or a string.