.. _tutorial_database_prep:

*******************************************************************************
Tutorial 2: How to prepare a database for faster load speeds
*******************************************************************************

In this tutorial, you will learn how to pre-process a conformer database
file for **FastROCS TK**, allowing for faster database load times with
:oe:meth:`OEFastROCS::OEShapeDatabase::Open`. Load times could be up to 10x
faster. See the figure below for an eMolecules dataset of 14 million
conformers.

.. _database_prep_figure_1:

.. figure:: ../../images/Tutorial_2_Figure_1.png
   :align: center
   :height: 550
   :width: 700

   **Pre-Processing Performance Impact**

To gain this extra loading performance,  you need to use the following
functions:

#. :oe:func:`OEChem::OEPreserveRotCompress` - this function works on
   the input molecule stream to ensure that rotor-offset compression
   is preserved during the preparation process. Rotor offset compression
   is a way of storing conformers as a set of torsions instead of
   storing the coordinates for every single conformer of a molecule.
   This optimization reduces the memory footprint of a multi-conformer
   molecule.

#. :oe:func:`OEChem::OEPRECompress` – this function works on the output
   molecule stream object allowing the molecules to be stored in a
   ‘pre-compressed’ format:

    - Writes rotor-offset-compressed molecules in the perfect-rotor-encoding
      format
    - There is no need to Gzip which means faster
      :oe:meth:`OEChem::OEMolDatabase::Open`.

#. :oe:func:`OEFastROCS::OEPrepareFastROCSMol` – this function works on
   each :ref:`OEMol <OEChem_OEMol>` record of the input.oeb:

    - Sets the energy of each conformer to 0.0 to avoid writing it
      to OEB.
    - Suppresses hydrogens and reorders reference conformers for compression.
    - Pre-calculates color atoms.
    - Pre-calculates self-color and self-shape terms for all conformers.

    .. note::

      The color terms cached by :oe:func:`OEFastROCS::OEPrepareFastROCSMol`
      are from the :oe:const:`OEShape::OEColorFFType::ImplicitMillsDean`
      color force field. A different color
      force field can be given as the second argument to override
      ImplicitMillsDean.

In general, calling :oe:func:`OEFastROCS::OEPrepareFastROCSMol` and 
:oe:func:`OEChem::OEPRECompress` will result in a smaller OEB file
than the default OEB.GZ output from OMEGA.

Further reduction in file-size can be achieved by using an 
:oe:const:`OEChem::OEMCMolType::HalfFloatCartesian` molecule to store
reference coordinates and torsions as 16-bit floating point. 

Here is some example code showing how to pre-process a database with
:oe:func:`OEFastROCS::OEPrepareFastROCSMol`, save to a precompressed
format, and reduce the file size by using half precision:

.. oe:language:: c++

   For added convenience, we have created a shapedatabaseprep.cpp example
   script which can be modified to meet your exact needs:

   .. only:: oehtml

      .. admonition:: Download code

         :download:`simpleprepscript.cpp <../../../../../../sublibs/tklibs/fastrocs/examples/simpleprepscript.cpp>`

   .. only:: oepdf

      .. oe:source:: simpleprepscript.cpp
         :language: c++

.. oe:language:: python

   For added convenience, we have created a SimplePrepScript.py example
   script which can be modified to meet your exact needs:

   .. only:: oehtml

      .. admonition:: Download code

         :download:`SimplePrepScript.py <../../../../../../wrappers/python/examples/fastrocs/SimplePrepScript.py>`

   .. only:: oepdf

      .. oe:source:: SimplePrepScript.py
         :language: python

.. oe:language:: java

   For added convenience, we have created a SimplePrepScript.java example
   script which can be modified to meet your exact needs:

   .. only:: oehtml

      .. admonition:: Download code

         :download:`SimplePrepScript.java <../../../../../../wrappers/java/examples/openeye/examples/oefastrocs/SimplePrepScript.java>`

   .. only:: oepdf

      .. oe:source:: SimplePrepScript.java
         :language: java
