.. _chapter_database:

********************************************************************************
Fingerprint Database
********************************************************************************

The following four examples perform the same task, detailed below:

#. reading a query structure
#. printing out the similarity score between the fingerprint
   of this query and the fingerprint generated for each molecule read
   from a database file.

In :oe:src:`Listing 9<SNIPPET-CALC-FROM-FILE-SimCalcFromFile>`, after importing the query
structure and generating its path fingerprint, the program loops over
the database file creating a path fingerprint for each structure.
Then the program calculates the Tanimoto similarity between the
fingerprint of the query and the database entry by calling the
:oe:func:`OEGraphSim::OETanimoto` function.

.. oe:source:: SimCalcFromFile.cpp
   :language: c++
   :refnum:   9
   :caption:  Similarity calculation from file
   :frag:     SNIPPET-CALC-FROM-FILE

.. oe:source:: SimCalcFromFile.py
   :refnum:   9
   :caption:  Similarity calculation from file
   :frag:     SNIPPET-CALC-FROM-FILE

..    oe:language:: python

    :ref:`Full listing. <listing-SimCalcFromFile.py>`

.. oe:source:: SimCalcFromFile.java
   :language: java
   :refnum:   9
   :caption:  Similarity calculation from file
   :frag:     SNIPPET-CALC-FROM-FILE

.. oe:source:: SimCalcFromFile.cs
   :language: csharp
   :refnum:   9
   :caption:  Similarity calculation from file
   :frag:     SNIPPET-CALC-FROM-FILE

In :oe:src:`Listing 10<SNIPPET-SIM-CALC-FROM-OEB-SimCalcFromOEB>` only the code block that is
different from :oe:src:`Listing 9<SNIPPET-CALC-FROM-FILE-SimCalcFromFile>` is shown.

In this example, it is assumed that the fingerprints are
pre-calculated and stored in an :file:`OEB` binary file as generic data
attached to the corresponding molecules.  The program loops over the
file and accesses the pre-generated fingerprints or calculates them if
they are not available.

The obvious advantage of this process is that the fingerprints one have to
be generated once when the binary file is created.  This can be
significantly faster, than generating the fingerprints on-the-fly
every time the program is executed.

.. seealso::

   The :ref:`chapter_storage` section shows an example of how to
   generate an :file:`OEB` binary file which stores molecule along
   with their corresponding fingerprints.

.. oe:source:: SimCalcFromOEB.cpp
   :language: c++
   :refnum:   10
   :caption:  Similarity calculation from OEB file
   :frag:     SNIPPET-SIM-CALC-FROM-OEB

.. oe:source:: SimCalcFromOEB.py
   :refnum:   10
   :caption:  Similarity calculation from OEB file
   :frag:     SNIPPET-SIM-CALC-FROM-OEB

..    oe:language:: python

    :ref:`Full listing. <listing-SimCalcFromOEB.py>`

.. oe:source:: SimCalcFromOEB.java
   :language: java
   :refnum:   10
   :caption:  Similarity calculation from OEB file
   :frag:     SNIPPET-SIM-CALC-FROM-OEB

.. oe:source:: SimCalcFromOEB.cs
   :language: csharp
   :refnum:   10
   :caption:  Similarity calculation from OEB file
   :frag:     SNIPPET-SIM-CALC-FROM-OEB


:oe:src:`Listing 11<SNIPPET-DB-SIM-CALC-FROM-FILE-DBSimCalcFromFile>` differs from
:oe:src:`Listing 9<SNIPPET-CALC-FROM-FILE-SimCalcFromFile>` in that
it uses an :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` object to store the
generated fingerprints.
The :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` class is designed to perform
in-memory fingerprint searches.

.. oe:source:: DBSimCalcFromFile.cpp
   :language: c++
   :refnum:   11
   :caption:  Similarity calculation  with fingerprint database from file
   :frag:     SNIPPET-DB-SIM-CALC-FROM-FILE

.. oe:source:: DBSimCalcFromFile.py
   :language: python
   :refnum:   11
   :caption:  Similarity calculation  with fingerprint database from file
   :frag:     SNIPPET-DB-SIM-CALC-FROM-FILE

..    oe:language:: python

    :ref:`Full listing. <listing-DBSimCalcFromFile.py>`

.. oe:source:: DBSimCalcFromFile.java
   :language: java
   :refnum:   11
   :caption:  Similarity calculation  with fingerprint database from file
   :frag:     SNIPPET-DB-SIM-CALC-FROM-FILE

.. oe:source:: DBSimCalcFromFile.cs
   :language: csharp
   :refnum:   11
   :caption:  Similarity calculation  with fingerprint database from file
   :frag:     SNIPPET-DB-SIM-CALC-FROM-FILE

After building the fingerprint database, the scores can be accessed by
the :oe:meth:`OEGraphSim::OEFPDatabase::GetScores` method.
This will return an iterator over the similarity scores calculated.

.. note::

   The :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` only stores
   fingerprints and not the molecules from which they are generated.
   A correspondence between a molecule and its fingerprint stored in
   the database can be established by using the index returned by the
   :oe:meth:`OEGraphSim::OEFPDatabase::AddFP` method.

.. seealso::

   :oe:src:`Listing 13<SNIPPET-MEM-SIM-SEARCH-MemSimSearch>` shows how to keep
   track of the correspondence between a fingerprint added to a
   :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` object and a molecule from which
   it is calculated.

In the last example (:oe:src:`Listing 12<SNIPPET-DB-SIM-CALC-FROM-OEB-DBSimCalcFromOEB>`),
:ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` is used again
to store the fingerprints. If the fingerprint is read from the
:file:`OEB` input binary file, then it is directly added to the
database, otherwise the fingerprint is generated on-the-fly when
passing the :ref:`OEMolBase <OEChem_OEMolBase>` molecule itself to the
:oe:meth:`OEGraphSim::OEFPDatabase::AddFP` method.

.. oe:source:: DBSimCalcFromOEB.cpp
   :language: c++
   :refnum:   12
   :caption:  Similarity calculation  with fingerprint database from OEB
   :frag:     SNIPPET-DB-SIM-CALC-FROM-OEB

.. oe:source:: DBSimCalcFromOEB.py
   :language: python
   :refnum:   12
   :caption:  Similarity calculation  with fingerprint database from OEB
   :frag:     SNIPPET-DB-SIM-CALC-FROM-OEB


..    oe:language:: python

    :ref:`Full listing. <listing-DBSimCalcFromOEB.py>`

.. oe:source:: DBSimCalcFromOEB.java
   :language: java
   :refnum:   12
   :caption:  Similarity calculation  with fingerprint database from OEB
   :frag:     SNIPPET-DB-SIM-CALC-FROM-OEB

.. oe:source:: DBSimCalcFromOEB.cs
   :language: csharp
   :refnum:   12
   :caption:  Similarity calculation  with fingerprint database from OEB
   :frag:     SNIPPET-DB-SIM-CALC-FROM-OEB


.. _section_database_search:

Sorted Search
================================================================================

Similarity searching based on a 2D representation of molecular structure
(such as fingerprints) is one of the most common approaches for
virtual screening.
A molecule that is structurally similar to an active molecule is more
likely to be active.

A virtual screening strategy involves going through a molecule database
and calculating the similarity between a reference structure and each
of the molecules, followed by ranking the similarity scores in
descending order to identify molecules that are the most similar to
the reference structure.

:oe:src:`Listing 13<SNIPPET-MEM-SIM-SEARCH-MemSimSearch>` shows how to search a molecule
database using the :oe:meth:`OEGraphSim::OEFPDatabase::GetSortedScores`
method to identify analogs based on their fingerprint similarity.
First the molecules are imported using an :ref:`OEMolDatabase <OEChem_OEMolDatabase>` object.
Then a :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` object is created that will
store the corresponding fingerprints.
Iterating over the molecules a fingerprints are added to the database by calling
the :oe:meth:`OEGraphSim::OEFPDatabase::AddFP` method.
In case when a molecule can not be accessed from the :ref:`OEMolDatabase <OEChem_OEMolDatabase>`
object, an empty fingerprint is added to the :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>`
object. This ensures that the indices of the two databases are synchronized.
After generating the fingerprints, the program reads
reference molecules (in the :term:`SMILES` format) from standard input.
Then the input :term:`SMILES` string is parsed and the fingerprint database is
searched to identify structures with the highest similarity scores.
Finally, the :term:`SMILES` string of the best hits are written to
standard output.

.. oe:source:: MemSimSearch.cpp
   :language: c++
   :refnum:   13
   :caption:  Similarity search in memory
   :frag:     SNIPPET-MEM-SIM-SEARCH

.. oe:source:: MemSimSearch.py
   :language: python
   :refnum:   13
   :caption:  Similarity search in memory
   :frag:     SNIPPET-MEM-SIM-SEARCH

..    oe:language:: python

    :ref:`Full listing. <listing-MemSimSearch.py>`

.. oe:source:: MemSimSearch.java
   :language: java
   :refnum:   13
   :caption:  Similarity search in memory
   :frag:     SNIPPET-MEM-SIM-SEARCH

.. oe:source:: MemSimSearch.cs
   :language: csharp
   :refnum:   13
   :caption:  Similarity search in memory
   :frag:     SNIPPET-MEM-SIM-SEARCH

As mentioned before, :ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` is a
fingerprint container and does not store the corresponding
molecule. Therefore the molecules have to be stored in a separate container
in this case in an :ref:`OEMolDatabase <OEChem_OEMolDatabase>` object.
When the :oe:meth:`OEGraphSim::OEFPDatabase::GetSortedScores` returns
the iterator over the best similarity scores, the associated index can
be utilized to access the corresponding structure in the
:ref:`OEMolDatabase <OEChem_OEMolDatabase>` object.

In the above example, the entire database was searched to identify
structurally similar molecules.
However, the user can also specify a segment of the database to be
searched by providing a begin and end index.

.. seealso::

  * :ref:`OEMolDatabase <OEChem_OEMolDatabase>` class in the **OEChem TK** manual

  * Examples of fingerprint searches in the API section:

    * :oe:meth:`OEGraphSim::OEFPDatabase::GetScores` method
    * :oe:meth:`OEGraphSim::OEFPDatabase::GetSortedScores` method

.. _section_database_simfunc:

Searching with User-defined Similarity Measures
================================================================================

By default, the :oe:const:`Tanimoto<OEGraphSim::OESimMeasure::Tanimoto>`
similarity is used  when calling either the
:oe:meth:`OEGraphSim::OEFPDatabase::GetScores` method or the
:oe:meth:`OEGraphSim::OEFPDatabase::GetSortedScores` method.
The user can set other types of similarity measures to be applied by
calling the :oe:meth:`OEGraphSim::OEFPDatabase::SetSimFunc` method with
a value from the :oe:constns:`OEGraphSim::OESimMeasure` namespace.
Each of the constants from this namespace corresponds to one of the
built-in similarity calculation methods.

There is also a facility to use user-defined similarity
measures when searching a fingerprint database.
The following example shows how a similarity calculation can
be implemented by deriving from the :ref:`OESimFuncBase <OEGraphSim_OESimFuncBase>`
class.

Formula:  :math:`Sim_{Simpson}(A,B) = \sqrt{\frac{bothAB}{min(onlyA+ bothAB),(onlyB+ bothAB))}}`


.. oe:source:: SimFunc.cpp
   :language: c++
   :frag:     SNIPPET-SIM-FUNC-1

.. oe:source:: SimFunc.py
   :language: python
   :frag:     SNIPPET-SIM-FUNC-1

..    oe:language:: python

    :ref:`Full listing. <listing-SimFunc.py>`

.. oe:source:: SimFunc.java
   :language: java
   :frag:     SNIPPET-SIM-FUNC-1

.. oe:source:: SimFunc.cs
   :language: csharp
   :frag:     SNIPPET-SIM-FUNC-1


After implementing the similarity calculation, it can be added to an
:ref:`OEFPDatabase <OEGraphSim_OEFPDatabase>` object, henceforth this new
similarity calculation will be used.

.. oe:source:: SimFunc.cpp
   :language: c++
   :noindent:
   :frag:     SNIPPET-SIM-FUNC-2

.. oe:source:: SimFunc.py
   :language: python
   :noindent:
   :frag:     SNIPPET-SIM-FUNC-2

..    oe:language:: python

    :ref:`Full listing. <listing-SimFunc.py>`

.. oe:source:: SimFunc.java
   :language: java
   :noindent:
   :frag:     SNIPPET-SIM-FUNC-2

.. oe:source:: SimFunc.cs
   :language: csharp
   :noindent:
   :frag:     SNIPPET-SIM-FUNC-2


.. seealso::

   * The :ref:`section_measure_user_defined` section
