RuNNE-2022 Shared Task: Recognizing Nested Named Entities

Е. Л. Артемова; М. В. Змеев; Н. В. Лукашевич; И.C. Рожков; Т.В. Батура; В.В. Иванов; Е.В. Тутубалина

doi:10.28995/2075-7182-2022-21-33-41

RuNNE-2022 Shared Task: Recognizing Nested Named Entities

Переведенное название: Соревнование RuNNE-2022: извлечение вложенных именованных сущностей

Е. Л. Артемова, М. В. Змеев, Н. В. Лукашевич, И.C. Рожков, Т.В. Батура, В.В. Иванов, Е.В. Тутубалина

Кафедра систем информатики ФИТ

Результат исследования: Материалы конференций › материалы › рецензирование

Аннотация

Извлечение именованных сущностей – одна из самых востребованных на практике задач извлечения информации – предполагает поиск в тексте упоминаний имен, организаций, топонимов и других сущностей. Соревнование RuNNE посвящено задаче извлечения вложенных именованных сущностей. Разметка данных допускает следующие случаи: внутри одной именованной сущности находится другая именованная сущность. Так, например в сущность класса Organization “Московский драматический театр имени М. Н. Ермоловой” вложена сущность типа Person – “М. Н. Ермоловой”. Соревнование проводится на материале корпуса NEREL(Loukachevitch et al., 2021), собранного из новостных текстов WikiNews на русском языке. В корпусе NEREL представлено 29 классов различных сущностей, а глубина вложенности сущностей достигает 6 уровней разметки.
В рамках соревнования RuNNE мы предлагаем участникам рассмотреть few shot постановку задачи. Задача предполагает извлечение вложенных именованных сущностей, В обучающем множестве большая часть типов именованных сущностей встречается достаточно часто, а некоторое количество специально отобранных типов – встречается всего несколько раз. В тестовом множестве все типы сущностей представлены одинаково. В данной статье мы описываем соревнование RuNNE, подводим его итоги и проводим сравнение решений, полученных от участников.

Переведенное название	Соревнование RuNNE-2022: извлечение вложенных именованных сущностей
Язык оригинала	английский
Число страниц	9
DOI	https://doi.org/10.28995/2075-7182-2022-21-33-41
Состояние	Опубликовано - 18 июн. 2022
Событие	International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022" - Москва, Российская Федерация Продолжительность: 15 июл. 2022 → 18 июл. 2022 https://www.dialog-21.ru/

Конференция

Конференция	International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022"
Сокращенный заголовок	Dialogue 2022
Страна/Tерритория	Российская Федерация
Город	Москва
Период	15.07.2022 → 18.07.2022
Адрес в сети Интернет	https://www.dialog-21.ru/

Предметные области OECD FOS+WOS

1.02 КОМПЬЮТЕРНЫЕ И ИНФОРМАЦИОННЫЕ НАУКИ

Доступ к документу

10.28995/2075-7182-2022-21-33-41

Другие файлы и ссылки

Fingerprint

Подробные сведения о темах исследования «Соревнование RuNNE-2022: извлечение вложенных именованных сущностей». Вместе они формируют уникальный семантический отпечаток (fingerprint).

Цитировать

Артемова, Е. Л., Змеев, М. В., Лукашевич, Н. В., Рожков, И. C., Батура, Т. В., Иванов, В. В., & Тутубалина, Е. В. (2022). RuNNE-2022 Shared Task: Recognizing Nested Named Entities. Документ представлен на International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022", Москва, Российская Федерация. https://doi.org/10.28995/2075-7182-2022-21-33-41

@conference{df1825c3ce5047c094b8dce8f40a6861,

title = "RuNNE-2022 Shared Task: Recognizing Nested Named Entities",

abstract = "The RuNNE Shared Task approaches the problem of nested named entity recognition. The annotation schema is designed in such a way, that an entity may partially overlap or even be nested into another entity. This way, the named entity “The Yermolova Theatre” of type ORGANIZATION houses another entity “Yermolova” of type PERSON. We adopt the Russian NEREL dataset (Loukachevitch et al., 2021) for the RuNNE Shared Task. NEREL comprises news texts written in the Russian language and collected from the Wikinews portal. The annotation schema includes 29 entity types. The nestedness of named entities in NEREL reaches up to six levels. The RuNNE Shared Task explores two setups. (i) In the general setup all entities occur more or less with the same frequency.(ii) In the few-shot setup the majority of entity types occur often in the training set. However, some of the entity types are have lower frequency, being thus challenging to recognize. In the test set the frequency of all entity types is even.This paper reports on the results of the RuNNE Shared Task. Overall the shared task has received 156 submissions from nine teams. Half of the submissions outperform a straightforward BERT-based baseline in both setups.This paper overviews the shared task setup and discusses the submitted systems, discovering meaning insights for the problem of nested NER. The links to the evaluation platform and the data from the shared task are available in our github repository.",

author = "Артемова, {Е. Л.} and Змеев, {М. В.} and Лукашевич, {Н. В.} and И.C. Рожков and Т.В. Батура and В.В. Иванов and Е.В. Тутубалина",

note = "Acknowledgments: The project is supported by the Russian Science Foundation, grant # 20-11-20166. The experiments were partially carried out on computational resources of HPC facilities at HSE University (Kostenetskiy et al., 2021) and the shared research facilities of HPC computing resources at Lomonosov Moscow State University. Ekaterina Artemova was supported by the framework of the HSE University Basic Research Program.; International conference on Computational Linguistics and Intellectual Technologies {"}Dialogue 2022{"}, Dialogue 2022 ; Conference date: 15-07-2022 Through 18-07-2022",

year = "2022",

month = jun,

day = "18",

doi = "10.28995/2075-7182-2022-21-33-41",

language = "English",

url = "https://www.dialog-21.ru/",

}

Артемова, ЕЛ, Змеев, МВ, Лукашевич, НВ, Рожков, ИC, Батура, ТВ, Иванов, ВВ & Тутубалина, ЕВ 2022, 'RuNNE-2022 Shared Task: Recognizing Nested Named Entities', Документ представлен на International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022", Москва, Российская Федерация, 15.07.2022 - 18.07.2022. https://doi.org/10.28995/2075-7182-2022-21-33-41

RuNNE-2022 Shared Task: Recognizing Nested Named Entities. / Артемова, Е. Л.; Змеев, М. В.; Лукашевич, Н. В. и др.

2022. Документ представлен на International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022", Москва, Российская Федерация.

Результат исследования: Материалы конференций › материалы › рецензирование

TY - CONF

T1 - RuNNE-2022 Shared Task: Recognizing Nested Named Entities

AU - Артемова, Е. Л.

AU - Змеев, М. В.

AU - Лукашевич, Н. В.

AU - Рожков, И.C.

AU - Батура, Т.В.

AU - Иванов, В.В.

AU - Тутубалина, Е.В.

N1 - Acknowledgments: The project is supported by the Russian Science Foundation, grant # 20-11-20166. The experiments were partially carried out on computational resources of HPC facilities at HSE University (Kostenetskiy et al., 2021) and the shared research facilities of HPC computing resources at Lomonosov Moscow State University. Ekaterina Artemova was supported by the framework of the HSE University Basic Research Program.

PY - 2022/6/18

Y1 - 2022/6/18

N2 - The RuNNE Shared Task approaches the problem of nested named entity recognition. The annotation schema is designed in such a way, that an entity may partially overlap or even be nested into another entity. This way, the named entity “The Yermolova Theatre” of type ORGANIZATION houses another entity “Yermolova” of type PERSON. We adopt the Russian NEREL dataset (Loukachevitch et al., 2021) for the RuNNE Shared Task. NEREL comprises news texts written in the Russian language and collected from the Wikinews portal. The annotation schema includes 29 entity types. The nestedness of named entities in NEREL reaches up to six levels. The RuNNE Shared Task explores two setups. (i) In the general setup all entities occur more or less with the same frequency.(ii) In the few-shot setup the majority of entity types occur often in the training set. However, some of the entity types are have lower frequency, being thus challenging to recognize. In the test set the frequency of all entity types is even.This paper reports on the results of the RuNNE Shared Task. Overall the shared task has received 156 submissions from nine teams. Half of the submissions outperform a straightforward BERT-based baseline in both setups.This paper overviews the shared task setup and discusses the submitted systems, discovering meaning insights for the problem of nested NER. The links to the evaluation platform and the data from the shared task are available in our github repository.

AB - The RuNNE Shared Task approaches the problem of nested named entity recognition. The annotation schema is designed in such a way, that an entity may partially overlap or even be nested into another entity. This way, the named entity “The Yermolova Theatre” of type ORGANIZATION houses another entity “Yermolova” of type PERSON. We adopt the Russian NEREL dataset (Loukachevitch et al., 2021) for the RuNNE Shared Task. NEREL comprises news texts written in the Russian language and collected from the Wikinews portal. The annotation schema includes 29 entity types. The nestedness of named entities in NEREL reaches up to six levels. The RuNNE Shared Task explores two setups. (i) In the general setup all entities occur more or less with the same frequency.(ii) In the few-shot setup the majority of entity types occur often in the training set. However, some of the entity types are have lower frequency, being thus challenging to recognize. In the test set the frequency of all entity types is even.This paper reports on the results of the RuNNE Shared Task. Overall the shared task has received 156 submissions from nine teams. Half of the submissions outperform a straightforward BERT-based baseline in both setups.This paper overviews the shared task setup and discusses the submitted systems, discovering meaning insights for the problem of nested NER. The links to the evaluation platform and the data from the shared task are available in our github repository.

UR - https://www.scopus.com/inward/record.url?eid=2-s2.0-85140870759&partnerID=40&md5=007f3fb45603091dcc6dc2772aefacf5

UR - https://www.mendeley.com/catalogue/a44f7b5d-6ba5-360d-9062-547a2d895685/

U2 - 10.28995/2075-7182-2022-21-33-41

DO - 10.28995/2075-7182-2022-21-33-41

M3 - Paper

T2 - International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022"

Y2 - 15 July 2022 through 18 July 2022

ER -

Артемова ЕЛ, Змеев МВ, Лукашевич НВ, Рожков ИC, Батура ТВ, Иванов ВВ и др. RuNNE-2022 Shared Task: Recognizing Nested Named Entities. 2022. Документ представлен на International conference on Computational Linguistics and Intellectual Technologies "Dialogue 2022", Москва, Российская Федерация. https://doi.org/10.28995/2075-7182-2022-21-33-41