Let's practice text processing with NLTK. You will be essentially duplicating what we did with the Gettysburg Address in class today.
- Instead of the Gettysburg address as input, use O. Henry's The Gift of the Magi, found here.
- The file should be downloaded (DO NOT copy-and-paste!! It is a bad practice to copy-paste text file contents!!) and placed in your usual Python script directory.
Instructions and tips:
- Name your new script file process_gift.py.
- Work in your Terminal environment using the command python process_gift.py or python -i process_gift.py
- Reference the input text file using its full file and directory path.
- Comment your code! (Required on all assignments going forward.)
- You should utilize shell as usual, but this text is much longer than Gettysburg, so you should take care not to flash large objects to shell. Using slicing [:50], [-50:] is one way.
- Have your script print out the following:
- How many word tokens there are,
- How many word types there are, (word types are a unique set of words)
- Top 20 most frequent words and their counts, (use for-loop on .most_common() output)
- Words that are at least 10 characters long and their counts.
- [BONUS CHALLENGE] 10+ characters-long words that occur at least twice, sorted from most frequent to least
- Format your printed output exactly like this (produced using tale.txt), down to the verbiage, spaces, blank lines, line breaks and placement of quotes.
- When you're done with your work, save the printed output as a text file by running python process_gift.py > process_gift_out.txt
- Additionally, save your shell session as a text file for submission, named process_gift_shell.txt. This is an example of a saved shell session: note that this is NOT a script. It will save your entire shell history, with errors and all, which is fine (remember: shell-side exploration is encouraged).
- Note: the story begins with the title and the author: "The Gift of the Magi by O. Henry". If you want, exclude this part when processing the text, but you are not obligated to do so.
SUBMIT:
- Upload: process_gift.py (script file), process_gift_out.txt (printed output saved as a text file), and process_gift_shell.txt (your shell session saved as a text file).
Remember to include in your scripts a comment line at the very top containing your name, Pitt email and and date, e.g.:
# Na-Rae Han, naxxxhan@pitt.edu, September 10, 2026
|
|
|