In this part, you will be reviewing a spell checker of your choice.
Select a spell checker and try out at least three misspellings of a word of your choice. Note the correction(s) suggested/made by the spell checker.
In terms of edit distance, how far off can your misspellings be and still have the correct spelling suggested?
Write up your findings. Be sure to properly document them: include screen shots if it helps (but remember that they are not a substitute for your written analysis). There's no need to write a lengthy paper -- about 1 page of written text (excluding screenshots and tables) will do.
PART 2: the Bible vs. Jane Austen Novels [25 points]
Are we ready to handle some seriously large texts? Yes we are! In this part, you will process two corpora: the Bible and the Jane Austen novels, using NLTK's text processing functions we reviewed in class.
The text files are included in the Project Gutenberg Selections corpus, available from the NLTK Corpora
page here: https://www.nltk.org/nltk_data/. Download the zipped archive (gutenberg.zip), unzip it, and place
the "gutenberg" directory under your script directory. In this assignment, we will be working with these two groups of text as two corpora:
The King James Version Bible (bible-kjv.txt)
Jane Austen novels (austen-emma.txt, austen-persuasion.txt, austen-sense.txt)
(Note: do NOT use NLTK's corpus loading tools such as PlaintextCorpusReader, which we will learn later. Also: do NOT access the two corpora through the pre-loaded nltk.corpus.gutenberg. The point of this homework is to treat these corpora as you would any random collection of texts you encounter on the Internet.)
Your job is to write a script that processes the two corpora for some basic stats:
opens (and later closes) the Bible text file, read in the string content,
builds a list of individual sentences,
prints out how many sentences there are,
builds a flat tokenized word list and the type list,
prints the token and the type counts of this corpus,
builds a frequency count dictionary of words,
prints the top 50 word types and their counts,
and repeats the above for the Jane Austen corpus.
There are three Austen text files, so how do you extract data from all of them? There are two ways to go:
Read the text content of each file as a string, and then concatenate the three (already pretty big!) strings into one gigantic string. Then, pass it to nltk.sent_tokenize() and nltk.word_tokenize(). To illustrate:
Alternatively, you can process each file separately for sentence/word tokens, and then merge the three token lists into one using +. In a different context (say, when you need to compare between Austen novels), this approach is the way to go.
Finally, make one observation about the two corpora. It could involve some new code of your own not included above, or it could be based off of A.--F. above. Have your script print out your observation enclosed in """...""".
These are seriously big text files, so take care not to crash your shell.
Your script file should be named bible_austen.py. I am not providing a template script file here: it is up to you to structure your script any way you see fit. But take care to make it readable and well-structured. And importantly, put comments in your script that explain what a block of code achieves. That helps my grading and your future code maintenance.
Save the printed output as a text file for submission, named bible_austen_out.txt. Recall that you can create this file by running: python bible_austen.py > bible_austen_out.txt
SUBMIT:
PART 1: Upload a word-processed file.
PART 2: Upload bible_austen.py (script) and bible_austen_out.txt (the printed output saved as a text file).
Remember to include in your scripts a comment line at the very top containing your name, Pitt email and and date, e.g.: # Na-Rae Han, naxxxhan@pitt.edu, September 10, 2026