Discovering utterance fragment boundaries in small unsegmented texts

Drienkó László: Discovering utterance fragment boundaries in small unsegmented texts.

[thumbnail of msznykonf_012_273-281.pdf]
Cikk, tanulmány, mű

Download (158kB) | Preview


We propose an algorithm for inferring boundaries of utterance fragments in relatively small unsegmented texts. The algorithm looks for subsequent largest chunks that occur at least twice in the text. Then adjacent fragments below an arbitrary length bound are merged. In our pilot experiment three types of English text were segmented: mother-child language from the CHILDES database, excerpts from Gulliver's travels by Jonathan Swift, and Now We Are Six, a children’s poem by A. A. Milne. The results are interpreted in terms of four precision metrics: Inference Precision, Alignment Precision, Redundancy, and Boundary Variability. We find that i) Inference Precision grows with merge-length, whereas Alignment Precision decreases – i.e. the longer a segment is the more probable that its two boundaries are correct; ii) Redundancy and Boundary Variability also decrease with the merge-length bound – i.e. the less boundaries we insert, the closer they are to the ideal boundaries.

Item Type: Conference or Workshop Item
Journal or Publication Title: Magyar Számítógépes Nyelvészeti Konferencia
Date: 2016
Volume: 12
ISBN: 978-963-306-450-4
Page Range: pp. 273-281
Event Title: Magyar Számítógépes Nyelvészeti Konferencia (12.) (2016) (Szeged)
Related URLs:
Uncontrolled Keywords: Nyelvészet - számítógép alkalmazása
Additional Information: Bibliogr.: 281. p. ; összefoglalás angol nyelven
Date Deposited: 2019. Jul. 01. 12:21
Last Modified: 2022. Nov. 08. 11:49

Actions (login required)

View Item View Item