Custom Type Example
===================

.. warning:: The following examples document a deprecated feature. The
  :class:`~pymongo.son_manipulator.SONManipulator` API has limitations as a
  technique for transforming your data. Instead, it is more flexible and
  straightforward to transform outgoing documents in your own code before
  passing them to PyMongo, and transform incoming documents after receiving
  them from PyMongo.

  Thus the :meth:`~pymongo.database.Database.add_son_manipulator` method is
  deprecated. PyMongo 3's new CRUD API does **not** apply SON manipulators to
  documents passed to :meth:`~pymongo.collection.Collection.bulk_write`,
  :meth:`~pymongo.collection.Collection.insert_one`,
  :meth:`~pymongo.collection.Collection.insert_many`,
  :meth:`~pymongo.collection.Collection.update_one`, or
  :meth:`~pymongo.collection.Collection.update_many`. SON manipulators are
  **not** applied to documents returned by the new methods
  :meth:`~pymongo.collection.Collection.find_one_and_delete`,
  :meth:`~pymongo.collection.Collection.find_one_and_replace`, and
  :meth:`~pymongo.collection.Collection.find_one_and_update`.


This is an example of using a custom type with PyMongo. The example
here is a bit contrived, but shows how to use a
:class:`~pymongo.son_manipulator.SONManipulator` to manipulate
documents as they are saved or retrieved from MongoDB. More
specifically, it shows a couple different mechanisms for working with
custom datatypes in PyMongo.

Setup
-----

We'll start by getting a clean database to use for the example:

.. doctest::

  >>> from pymongo.mongo_client import MongoClient
  >>> client = MongoClient()
  >>> client.drop_database("custom_type_example")
  >>> db = client.custom_type_example

Since the purpose of the example is to demonstrate working with custom
types, we'll need a custom datatype to use. Here we define the aptly
named :class:`Custom` class, which has a single method, :meth:`x`:

.. doctest::

  >>> class Custom(object):
  ...   def __init__(self, x):
  ...     self.__x = x
  ...
  ...   def x(self):
  ...     return self.__x
  ...
  >>> foo = Custom(10)
  >>> foo.x()
  10

When we try to save an instance of :class:`Custom` with PyMongo, we'll
get an :class:`~bson.errors.InvalidDocument` exception:

.. doctest::

  >>> db.test.insert({"custom": Custom(5)})
  Traceback (most recent call last):
  InvalidDocument: cannot convert value of type <class 'Custom'> to bson

Manual Encoding
---------------

One way to work around this is to manipulate our data into something
we *can* save with PyMongo. To do so we define two methods,
:meth:`encode_custom` and :meth:`decode_custom`:

.. doctest::

  >>> def encode_custom(custom):
  ...   return {"_type": "custom", "x": custom.x()}
  ...
  >>> def decode_custom(document):
  ...   assert document["_type"] == "custom"
  ...   return Custom(document["x"])
  ...

We can now manually encode and decode :class:`Custom` instances and
use them with PyMongo:

.. doctest::

  >>> import pprint
  >>> db.test.insert({"custom": encode_custom(Custom(5))})
  ObjectId('...')
  >>> pprint.pprint(db.test.find_one())
  {u'_id': ObjectId('...'),
   u'custom': {u'_type': u'custom', u'x': 5}}
  >>> decode_custom(db.test.find_one()["custom"])
  <Custom object at ...>
  >>> decode_custom(db.test.find_one()["custom"]).x()
  5

Automatic Encoding and Decoding
-------------------------------

Needless to say, that was a little unwieldy. Let's make this a bit
more seamless by creating a new
:class:`~pymongo.son_manipulator.SONManipulator`.
:class:`~pymongo.son_manipulator.SONManipulator` instances allow you
to specify transformations to be applied automatically by PyMongo:

.. doctest::

  >>> from pymongo.son_manipulator import SONManipulator
  >>> class Transform(SONManipulator):
  ...   def transform_incoming(self, son, collection):
  ...     for (key, value) in son.items():
  ...       if isinstance(value, Custom):
  ...         son[key] = encode_custom(value)
  ...       elif isinstance(value, dict): # Make sure we recurse into sub-docs
  ...         son[key] = self.transform_incoming(value, collection)
  ...     return son
  ...
  ...   def transform_outgoing(self, son, collection):
  ...     for (key, value) in son.items():
  ...       if isinstance(value, dict):
  ...         if "_type" in value and value["_type"] == "custom":
  ...           son[key] = decode_custom(value)
  ...         else: # Again, make sure to recurse into sub-docs
  ...           son[key] = self.transform_outgoing(value, collection)
  ...     return son
  ...

Now we add our manipulator to the :class:`~pymongo.database.Database`:

.. doctest::

  >>> db.add_son_manipulator(Transform())

After doing so we can save and restore :class:`Custom` instances seamlessly:

.. doctest::

  >>> db.test.remove() # remove whatever has already been saved
  {...}
  >>> db.test.insert({"custom": Custom(5)})
  ObjectId('...')
  >>> pprint.pprint(db.test.find_one())
  {u'_id': ObjectId('...'),
   u'custom': <Custom object at ...>}
  >>> db.test.find_one()["custom"].x()
  5

If we get a new :class:`~pymongo.database.Database` instance we'll
clear out the :class:`~pymongo.son_manipulator.SONManipulator`
instance we added:

.. doctest::

  >>> db = client.custom_type_example

This allows us to see what was actually saved to the database:

.. doctest::

  >>> pprint.pprint(db.test.find_one())
  {u'_id': ObjectId('...'),
   u'custom': {u'_type': u'custom', u'x': 5}}

which is the same format that we encode to with our
:meth:`encode_custom` method!

Binary Encoding
---------------

We can take this one step further by encoding to binary, using a user
defined subtype. This allows us to identify what to decode without
resorting to tricks like the ``_type`` field used above.

We'll start by defining the methods :meth:`to_binary` and
:meth:`from_binary`, which convert :class:`Custom` instances to and
from :class:`~bson.binary.Binary` instances:

.. note:: You could just pickle the instance and save that. What we do
   here is a little more lightweight.

.. doctest::

  >>> from bson.binary import Binary
  >>> def to_binary(custom):
  ...   return Binary(str(custom.x()).encode(), 128)
  ...
  >>> def from_binary(binary):
  ...   return Custom(int(binary))
  ...

Next we'll create another
:class:`~pymongo.son_manipulator.SONManipulator`, this time using the
methods we just defined:

.. doctest::

  >>> class TransformToBinary(SONManipulator):
  ...   def transform_incoming(self, son, collection):
  ...     for (key, value) in son.items():
  ...       if isinstance(value, Custom):
  ...         son[key] = to_binary(value)
  ...       elif isinstance(value, dict):
  ...         son[key] = self.transform_incoming(value, collection)
  ...     return son
  ...
  ...   def transform_outgoing(self, son, collection):
  ...     for (key, value) in son.items():
  ...       if isinstance(value, Binary) and value.subtype == 128:
  ...         son[key] = from_binary(value)
  ...       elif isinstance(value, dict):
  ...         son[key] = self.transform_outgoing(value, collection)
  ...     return son
  ...

Now we'll empty the :class:`~pymongo.database.Database` and add the
new manipulator:

.. doctest::

  >>> db.test.remove()
  {...}
  >>> db.add_son_manipulator(TransformToBinary())

After doing so we can save and restore :class:`Custom` instances
seamlessly:

.. doctest::

  >>> db.test.insert({"custom": Custom(5)})
  ObjectId('...')
  >>> pprint.pprint(db.test.find_one())
  {u'_id': ObjectId('...'),
   u'custom': <Custom object at ...>}
  >>> db.test.find_one()["custom"].x()
  5

We can see what's actually being saved to the database (and verify
that it is using a :class:`~bson.binary.Binary` instance) by
clearing out the manipulators and repeating our
:meth:`~pymongo.collection.Collection.find_one`:

.. doctest::

  >>> db = client.custom_type_example
  >>> pprint.pprint(db.test.find_one())
  {u'_id': ObjectId('...'), u'custom': Binary('5', 128)}
