oai:www.ldc.upenn.edu:LDC2026T07
| |
OLAC Record oai:www.ldc.upenn.edu:LDC2026T07 |
| Metadata | ||
| Title: | KAIROS Phase 1 Evaluation Source Data, Annotation, and Assessment | |
| Access Rights: | Licensing Instructions for Subscription & Standard Members, and Non-Members: http://www.ldc.upenn.edu/language-resources/data/obtaining | |
| Bibliographic Citation: | Chen, Song, et al. KAIROS Phase 1 Evaluation Source Data, Annotation, and Assessment LDC2026T07. Web Download. Philadelphia: Linguistic Data Consortium, 2026 | |
| Contributor: | Chen, Song | |
| Tracey, Jennifer | ||
| Mott, Justin | ||
| Bies, Ann | ||
| Arrigo, Michael | ||
| Caruso, Christopher | ||
| Graff, David | ||
| Strassel, Stephanie | ||
| Date (W3CDTF): | 2026 | |
| Date Issued (W3CDTF): | 2026-06-15 | |
| Description: | *Introduction* KAIROS Phase 1 Evaluation Source Data, Annotation, and Assessment was developed by the Linguistic Data Consortium (LDC). It contains the English and Spanish source data (text, video and images), manual annotations, reference knowledge graphs, the system output assessed during the evaluation, and human assessment results from the Phase 1 evaluation of the DARPA KAIROS Program. The Phase 1 evaluation focused on the improvised explosive bombing scenario with nine complex events (CEs) and two surprise complex events in the mass shooting scenario: * ce1005: Sidney Aeroplane Bomb Plot, Australia, 2017 * ce1006: Stockholm Bombings, Sweden, 2010 * ce1007: Manchester Arena Bombing, England, 2017 * ce1008: Taxi Detonation, Canada, 2016 * ce1009: Spokane Bombing Attempt, Washington, 2011 * ce1010: Derry Bombing, Northern Ireland, 2019 * ce1011: Bogotá Police Academy Car Bombing, Colombia, January 2019 * ce1012: Kansas City Hospital Bombing, Missouri, 2020 * ce1013: Attempted bombing in Moses Lake, Washington, 2018 * ce1020: El Paso Walmart Shooting, Texas, 2019 * ce1021: Orlando nightclub shooting, Florida, 2016 The DARPA KAIROS (Knowledge-directed Artificial Intelligence Reasoning Over Schemas) program aimed to build technology capable of understanding and reasoning about complex real-world events in order to provide actionable insights to end users. KAIROS systems utilized formal event representations in the form of schema libraries that specified the steps, preconditions and constraints for an open set of complex events; schemas were then used in combination with event extraction to characterize and make predictions about real-world events in a large multilingual, multimedia corpus. Each KAIROS evaluation focused on a real-world scenario and several real-world complex events within that scenario, along with the possibility of surprise complex events in different but related scenarios. *Data* Source data was collected from the web by LDC. A total of 139 root web pages were collected and processed, yielding 131 text data files, 1176 image files, and 27 video files. The evaluation source data for each complex event was an input data set consisting of 10-15 documents that included multimodal English and Spanish event-relevant and off-topic distractor documents. Manual annotation and assessment of event-relevant documents for 10 complex events are included in this release. Scenario-relevant events and relations were labeled for each document to develop a structured representation of temporally-ordered events, relations and arguments that expressed the scenario-relevant events in each complex event. A reference knowledge graph (Graph G) was developed for each event; systems were expected to match the Graph G with a given schema library. Assessment data includes human assessment judgments and the system output that was manually assessed for the end-to-end evaluation task. Source data is presented in various formats: .gif, .jpg,. ltf, .mp4, .png, .psm, and .svg. Annotations are presented as tab separated files (.tab). Graph G data is presented in JSON format and in human-readable Excel (.xlsx) files. System output is presented in JSON format and as tab separated files. A software tool is also included in this release to recreate original source data from the processed XML material. *Samples* Please view these samples: * Argument Annotations (.tab) * Event Annotations (.tab) * Relations Annotations (.tab) * Temporal Annotations (.tab) * Graph G (.json) * Correctness Assessment *Sponsorship* KAIROS was sponsored by the Air Force Research Laboratory (AFRL) and the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-19-S-0014. *Updates* No updates at this time. | |
| Extent: | Corpus size: 679936 KB | |
| Identifier: | LDC2026T07 | |
| https://catalog.ldc.upenn.edu/LDC2026T07 | ||
| ISLRN: 558-102-578-740-1 | ||
| DOI: 10.35111/rfam-2766 | ||
| Language: | Spanish | |
| English | ||
| Language (ISO639): | spa | |
| eng | ||
| License: | LDC User Agreement for Non-Members: https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf | |
| Medium: | Distribution: Web Download | |
| Publisher: | Linguistic Data Consortium | |
| Publisher (URI): | https://www.ldc.upenn.edu | |
| Relation (URI): | https://catalog.ldc.upenn.edu/docs/LDC2026T07 | |
| Rights Holder: | Portions © 2019 ABC, © 2008 ACTUALIDAD MEDIA GROUP, LLC, © 2011 Autonomous Nonprofit Organization “TV-Novosti,” © 2010, 2014, 2019-2020 BBC, © 2019 BelfastTelegraph.co.uk, © 2016, 2020 Cable News Network. A Warner Media Company, © 2016 CBC/Radio-Canada, © 2010 Channel Four Television Corporation, © 2020 Chicago Tribune, © 2019 EDICIONES EL PAÍS SL, © 2019 France 24, © 2020 Google LLC, © 2010 Guardian News & Media Limited or its affiliated companies, © 2020 Heavy Inc., © 2020 Kansas City Public Radio, © 2020 KCTV5 News (A Meredith Corporation Station), © 2011, 2018 KHQ, © 2019 Morgan Murphy Media, © 2019-2020 NBC UNIVERSAL, © 2016, 2019 News Group Newspapers Limited in England, © 2019 npr, © 2019 NYP Holdings, Inc., © 2018 NYWA, © 2011, 2020 Reuters, © 2018 Sinclair Broadcast Group, Inc., © 2016 Sky UK, © 2017 Spanish Radio and Television Corporation, © 2019 Special Broadcasting Service Corporation, © 2010 Standard.co.uk, © 2020 The Associated Press, © 2019-2020 The City Paper Bogotá, © 2019 The Epoch Times, © 2019 THE IRISH TIMES, © 2017 The Lion of El Español Publications SA, © 2019 Unidad Editorial Informacion General, SLU, © 2020, 2026 Trustees of the University of Pennsylvania | |
| Subject: | English language | |
| Spanish language | ||
| Subject (ISO639): | eng | |
| spa | ||
| Type (DCMI): | Image | |
| MovingImage | ||
| Software | ||
| Sound | ||
| StillImage | ||
| Text | ||
| Type (OLAC): | primary_text | |
OLAC Info |
||
| Archive: | The LDC Corpus Catalog | |
| Description: | http://www.language-archives.org/archive/www.ldc.upenn.edu | |
| GetRecord: | OAI-PMH request for OLAC format | |
| GetRecord: | Pre-generated XML file | |
OAI Info |
||
| OaiIdentifier: | oai:www.ldc.upenn.edu:LDC2026T07 | |
| DateStamp: | 2026-06-15 | |
| GetRecord: | OAI-PMH request for simple DC format | |
Search Info | ||
| Citation: | Chen, Song; Tracey, Jennifer; Mott, Justin; Bies, Ann; Arrigo, Michael; Caruso, Christopher; Graff, David; Strassel, Stephanie. 2026. Linguistic Data Consortium. | |
| Terms: | area_Europe country_ES country_GB dcmi_Image dcmi_MovingImage dcmi_Software dcmi_Sound dcmi_StillImage dcmi_Text iso639_eng iso639_spa olac_primary_text | |
Inferred Metadata | ||
| Country: | SpainUnited Kingdom | |
| Area: | Europe | |