Natural Language Processing in Python using NLTK
Here are the materials for Natural Language Processing in Python using NLTK.
"Natural Language Processing in Python using NLTK" was presented at LinuxFest Northwest 2008 (Bellingham, April 26). It's an introductory talk aimed at Python programmers — not linguists — who work with substantial amounts of prose text and want tools beyond string processing and regular expressions. Rather than teaching Python or attempting a comprehensive treatment of NLP, the talk set out to introduce the field, introduce NLTK, walk through a few simple applications, and make the case that the toolkit is approachable enough to download and try. It draws on the presenter's background: an MA in linguistics from UCLA and 19 years working in NLP at BBN Technologies, by then applied to semantic web, text processing, and content extraction work at Logos Bible Software.
The bulk of the talk surveyed NLTK itself — at the time some 50,000 lines of code with more than 30 annotated corpora (over 300MB of data) and an accompanying 400-page book, already used for instruction at three dozen universities — and walked through concrete examples: working with corpora, tokenizing and counting words, part-of-speech tagging, word-sense disambiguation, and utilities for probability and evaluation (confusion matrices, precision/recall). It closed by pointing to capabilities there wasn't time to cover — chunking, parsing, a simple chatbot, feature-based classification — and to related NLP applications outside NLTK, like machine translation and speech recognition.