Python 2.7 TutorialWith Videos by mybringback.com
Pickling | |||||||||||
On this page: pickle module, pickle.dump(), pickle.load(), cPickle module
Pickling: the ConceptSuppose you just spent a better part of your afternoon working in Python, processing many data sources to build an elaborate, highly structured data object. Say it is a dictionary of English words with their frequency counts, translation into other languages, etc. And now it's time to close your Python program and go eat dinner. Obviously you want to save this object for future use, but how?You *could* write the data object out to a text file, but that's not optimal. Once written as a text file, it is a simple text file, meaning next time you read it in you will have parse the text and process it back to your original data structure. What you want, then, is a way to save your Python data object as itself, so that next time you need it you can simply load it up and get your original object back. Pickling and unpickling let you do that. A Python data object can be "pickled" as itself, which then can be directly loaded ("unpickled") as such at a later point; the process is also known as "object serialization". How to Pickle/UnpicklePickling functions are part of the pickle module. You will first need to import it. And, pickling/unpickling obviously involves file IO, so you will have to use the file writing/reading routines you learned in the previous tutorial.Below, grades, a small dictionary data object, is being pickled. pickle.dump() is the method for saving the data out to the designated pickle file, usually with the .p or .pkl extension.
Pickling in the BinaryThe default pickling routine shown above saves the data as an ASCII text file, albeit in a Python-specific data format. This means that your pickle file is going to be large. For improved efficiency, it is recommended to use a binary protocol instead. This is basically achieved by specifying a third, optional "protocol level" argument while dumping, e.g., pickle.dump(grades, f, -1). "-1" means the highest available binary protocol. In addition, file IO will have to be done in a binary mode: you need to use 'wb' ('b' for binary) during file writing and 'rb' during file opening.
cPickle: Pickling FasterOnce you start crunching some serious volumes of data, you will find that pickle operations move at a crawl speed. Python gives you an alternative for this: cPickle. This module is very much like the standard pickle module, only much faster (up to 1000x times!), because it was written in C.To use this module instead, all you have to do is remember to import cPickle. All other operations are the same. And pickle data files written by either module are compatible with either, meaning a pickle file written by cPickle can be unpickled by pickle and vice versa, as long as the same pickling protocol was used.
|