Natural Language Toolkit 0.8

Natural Language Toolkit is a suite of Python libraries and programs for symbolic and statistical natural language processing. NLTK includes graphical demonstrations and sample data.
It is accompanied by extensive documentation, including tutorials that explain the underlying concepts behind the language processing tasks supported by the toolkit.
Documentation:
A substantial amount of documentation about how to use NLTK is available from the nltk home page:
< http://nltk.sourceforge.net >
In particular, the NLTK home page contains three types of documentation:
- Tutorials teach students how to use the toolkit, in the context of performing specific tasks. They are appropriate for anyone who wishes to learn how to use the toolkit.
< http://nltk.sourceforge.net/tutorial/ >
- The toolkits reference documentation describes every module, interface, class, method, function, and variable in the toolkit. This documentation should be useful to both users and developers.
< http://nltk.sourceforge.net/ref/nltk.html >
- A number of technical reports are available. These reports explain and justify the toolkits design and implementation. They are used by the developers of the toolkit to guide and document the toolkits construction. Students can consult these reports if they would like further information about how the toolkit is designed and why it is designed that way.
< http://nltk.sourceforge.net/tech/ >
Enhancements:
Code (major):
- changed package name to nltk
- import all top-level modules into nltk, reducing need for import statements
- reorganization of sub-package structures to simplify imports
- new featstruct module, unifying old featurelite and featurestructure modules
- FreqDist now inherits from dict, fd.count(sample) becomes fd[sample]
- FreqDist initializer permits: fd = FreqDist(len(token) for token in text)
- made numpy optional
Code (minor):
- changed GrammarFile initializer to accept filename
- consistent tree display format
- fixed loading process for WordNet and TIMIT that prevented code installation if data not installed
- taken more care with unicode types
- incorporated pcfg code into cfg module
- moved cfg, tree, featstruct to top level
- new filebroker module to make handling of example grammar files more transparent
- more corpus readers (webtext, abc)
- added cfg.covers() to check that a grammar covers a sentence
- simple text-based wordnet browser
- known bug: parse/featurechart.py uses incorrect apply() function
Corpora:
- csv data file to document NLTK corpora
Contrib:
- added Glue semantics code (contrib.glue, by Dan Garrette)
- Punkt sentence segmenter port (contrib.punkt, by Willy)
- added LPath interpreter (contrib.lpath, by Haejoong Lee)
- extensive work on classifiers (contrib.classifier*, Sumukh Ghodke)
Tutorials:
- polishing on parts I, II
- more illustrations, data plots, summaries, exercises
- continuing to make prose more accessible to non-linguistic audience
- new default import that all chapters presume: from nltk.book import *
Distributions:
- updated to latest version of numpy
- removed WordNet installation instructions as WordNet is now included in corpus distribution
- added pylab (matplotlib)
Enhancements:
Code:
- changed nltk.__init__ imports to explicitly import names from top-level modules
- changed corpus.util to use the rb flag for opening files, to fix problems reading corpora under MSWindows
- updated stale examples in engineering.txt
- extended feature stucture interface to permit chained features, e.g. fs[F,G]
- further misc improvements to test code plus some bugfixes
Tutorials:
- rewritten opening section of tagging chapter
- reorganized some exercises