Vera Demberg - Teaching

Summer Semester 2011

Seminar on Incremental Processing

Incremental processing in its most strict interpretation means that information is fully and eagerly integrated with previously processed material as soon as it is available, i.e. hypotheses about syntactic structures and semantic roles are made as soon as each word is encountered. On the syntax level, strict incrementality would for example mean that each word is integrated into a syntactic structure immediately, i.e. the processor would not use any stack to keep seen words for later decision taking. The opposite of incremental processing would be a machine that first reads in the whole sentence, then decide on part-of-speech tags, then builds up a syntactic structure bottom-up, and only starts assigning semantic roles once syntactic structures are completed.

This course will look at incremental processing from two different viewpoints: 1) is there evidence (or counter-evidence) for strict incremental processing in human sentence processing? 2) Incrementality from an NLP perspective: How can incremental processing be realized, what are the advantages and disadvantages of processing incrementally, and what applications can profit from incremental processing?

The new time slot of the seminar is from 12:30s.t. to 14:00.

New: We're going to have the additional session on MAY 28th, 2-6pm.

Also, I confirmed that you can choose to do an oral exam for this course. Oral exams will take place some time during the first two weeks of August or the last week of June / first week of July.

Download a copy of the anonymous peer review form.

Introduction: The strict competence hypothesis

• Abney and Johnson (1991) Memory Requirements and Local Ambiguities of Parsing Strategies
• Steedman, M. (1992).Grammars and Processors (Tech. Rep. MS-CIS-92-52). University of Pennsylvania, Department of Computer and Information Science. To appear in H. Kamp & C. Rohrer, (Eds.),Current Theories in Natural Language Understanding. The Netherlands: Reidel.
• Shieber and Johnson (1993) Variations in incremental interpretation.
• Edward Stabler (1991) Avoid the pedestrian's paradox. In Principel-based parsing: computation and psycholinguistics, pp. 199-237

Psycholinguistic Experiments on Incrementality

• Sturt, P. and Lombardo, V. (2005). Processing coordinate structures: Incrementality and connectedness. Cognitive Science, 29:291-305.
• Aoshima, Yoshida and Phillips (2009) Incremental Processing of Coreference and Binding in Japanese
  
• Swets et al. (2008) Underspecification of syntactic ambiguities: Evidence from self-paced reading

• Altman, Kamide and Haywood (2003) The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements. Journal of Memory and Language, 49, 133-156.
  
• Van Berkum et al. (2005) Anticipating upcoming words in discourse: Evidence from ERPs and reading times.
  
• Staub and Clifton (2006) Syntactic Prediction in Language Comprehension: Evidence From Either ... or

• Tabor et al. (2004) Effects of merely local syntactic coherence on sentence processing.
• Gibson (2006) The interaction of top-down and bottom-up statistics in the resolution of syntactic category ambiguity.

• Fernanda Ferreira (2000). Syntax in Language Production: An Approach Using Tree-Adjoining Grammars

• Phillips (2002) Linear Order and Constituency
  

Incrementality in NLP

• Brian Roark (2001) Probabilistic top-down parsing and language modeling. In Computational Linguistics, 27(2), pages 249-276.

• Nivre (2004) "Incrementality in Deterministic Dependency Parsing"
  
• Menzel (2009) "Towards radically incremental parsing of natural language. "
  

3a:
• Libin Shen and Aravind Joshi (2005). Incremental LTAG parsing
• Mazzei, Lombardo, Sturt (2007). "Dynamic TAG and Lexical Dependencies" (a dynamic version of TAG that supports full connectivity in incremental processing)
  
3b:
• Demberg, Keller and Koller, (2011, under review) Incremental, Predictive Parsing with Psycholinguistically Motivated Tree-Adjoining Grammar
• Demberg and Keller, (2008) A Psycholinguistically Motivated Version of TAG

• Atterer and Schlangen (2009) RUBISC - a Robust Unification-Based Incremental Semantic Chunker
• Purver and Kempson (2004) Incremental parsing, or incremental grammar? (will require to learn about "Dynamic Syntax")

• Stephen Wu, Asaf Bachrach, Carlos Cardenas, William Schuler (2010) Complexity Metrics in an Incremental Right-corner Parser. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL'10), Uppsala, Sweden.
• Matthew Crocker and Thorsten Brants (2000) Wide Coverage Probabilistic Sentence Processing. Journal of Psycholinguistic Research: 29(6):647-669

Incremental processing in NLP Applications

• Skantze, G., & Hjalmarsson, A. (2010). Towards Incremental Speech Generation in Dialogue Systems. In Proceedings of SIGdial. Tokyo, Japan.
• Skantze, G., & Schlangen, D. (2009) Incremental dialogue processing in a micro-domain. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL-09). Athens, Greece.
• Schlangen, D., & Skantze, G. (2009) A general, abstract model of incremental dialogue processing. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL-09). Athens, Greece.

• Hassan, Sima'an, Way (2009) A Syntactified Direct Translation Model with Linear-time Decoding
• Hefny, Hassan, Bahgat (2011) Incremental Combinatory Categorial Grammar and Its Derivations

• Baumann, Atterer, Schlangen (2009) Assessing and Improving the Performance of Speech Recognition for Incremental Systems
• Kato, Y. and Matsubara, S. and Inagaki, Y.(2004) Stochastically evaluating the validity of partial parse trees in incremental parsing, Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together, pages 9--15

• Schuler, Wu, Schwarz (2009) A Framework for Fast Incremental Interpretation during Speech Decoding, Computational Linguistics, Vol 35, No. 3, pages 313-343