Pickle custom class in Django App

by: strategist_roy, 7 years ago


I recently started learning Django from this same website in order to host machine learning scripts as a web application. I had pickled a classifier in the form of a custom class (like the vote classifier of nltk video series). I found that normal objects got unpickled pretty easily whereas the custom class couldn't. A simple example of loading a pickled class in Django can be found below:-
Directory Structure:
Project
====myapp
========data
============car.pickle
========__init__.py
========urls.py
========car.py
========views.py
====project
========__init__.py
========settings.py
========urls.py
========wsgi.py
====manage.py

The contents of the various files are as:-
myapp/views.py

from django.http import HttpResponse
from .car import Car
import os
import pickle

def index(request):
modulePath = os.path.dirname(__file__)  # get current directory
filePath = os.path.join(modulePath, 'datacar.pickle')
with open(filePath, 'rb') as f:
obj=pickle.load(f)
return HttpResponse('<h1>{}</h1>'.format(obj.showName()))

myapp/car.py

class Car:

def __init__(self,name):
self.name=name

def showName(self):
return self.name

I found the car import statement working fine. Cause, in views.index,
obj=Car('Bugatti')

works flawlessly. However, whenever I try to unpickle the pickled car. It gives the following error:-

Environment:


Request Method: GET
Request URL: http://localhost:8000/myapp/

Django Version: 1.10.5
Python Version: 2.7.12
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'myapp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "C:Virtual Environmentssentiencelibsite-packagesdjangocorehandlersexception.py" in inner
  39.             response = get_response(request)

File "C:Virtual Environmentssentiencelibsite-packagesdjangocorehandlersbase.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "C:Virtual Environmentssentiencelibsite-packagesdjangocorehandlersbase.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "F:PythonwebdevdjangoMyProjectmyappviews.py" in index
  10. obj=pickle.load(f)

File "c:python27Libpickle.py" in load
  1384.     return Unpickler(file).load()

File "c:python27Libpickle.py" in load
  864.                 dispatch[key](self)

File "c:python27Libpickle.py" in load_inst
  1075.         klass = self.find_class(module, name)

File "c:python27Libpickle.py" in find_class
  1132.         klass = getattr(mod, name)

Exception Type: AttributeError at /myapp/
Exception Value: 'module' object has no attribute 'Car'

The whole code of the app can be found on:-
http://codecookies.xyz/MyProject.rar

Please, help me sort out the problem as I'm totally new to Django and couldn't find a solution after hours of googling and stackoverflow



You must be logged in to post. Please login or register an account.



To let everyone know I found the error. The error was in the module information present in the pickled file. The fix is this:-
When someone pickles from the same file as the class as shown:-

class Car:

def __init__(self,name):
self.name=name

def showName(self):
return self.name

with open('car.pickle','wb') as f:
pickle.dump(car,f)

The module in the pickle file gets set to __main__. Thus, when I tried to import in the views.py as :-

from .car import Car

There was a mismatch of module, as the Car class imported here was car.Car whereas in the pickle file it had been __main__.Car
The fix was thus to create a dummy dumping file as shown:-

from .car import Car

if __name__=="__main__":
      #Instantiate class.
      #Process whatever
      #Dump pickle file. This sets the module of the pickled file to be that of imported. i.e. 'car'

Thereafter to use it properly in views.py, do as:-

#views.py
import car
import sys
sys.modules['car']=car

#Load pickled file as usual.



-strategist_roy 7 years ago
Last edited 7 years ago

You must be logged in to post. Please login or register an account.