.. include:: ../toolkit-substitutions

.. _chapter_mdlqueries:

********************************************************************************
Substructure Search with MDL Queries
********************************************************************************

.. index :: MDL query

Section :ref:`section_patternmatch_sss` describes how to perform substructure
search initialized with a SMILES or SMARTS string.
**OEChem TK** also provides the ability to interpret and utilize query structures
expressed in the :term:`MDL` query  file format
(see MDL query example in :ref:`Figure: Example of MDL query substructure<figure_mdlq_query>`).
:oe:src:`Listing 1<SNIPPET-MDLQuerySearchSingleMatch>` shows how to initialize an
:oe:cls:`OESubSearch<OEChem::OESubSearch>` object from a MDL query file and
perform a substructure search.

.. _figure_mdlq_query:

.. figure:: images/MDLQuery.png
   :align: center

   **Example of MDL query substructure**


.. oe:source:: MDLQuerySearchSingleMatch.cpp
   :language: c++
   :caption:  Example of substructure search using MDL query file
   :refnum:   1
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchSingleMatch.py
   :language: python
   :caption:  Example of substructure search using MDL query file
   :refnum:   1
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchSingleMatch.java
   :language: java
   :caption:  Example of substructure search using MDL query file
   :refnum:   1
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchSingleMatch.cs
   :language: csharp
   :caption:  Example of substructure search using MDL query file
   :refnum:   1
   :frag:     SNIPPET

After opening the MDL query and the target files, the model used to assign
aromaticity to the imported structures can be adjusted.

.. oe:source:: MDLQuerySearchSingleMatch.cpp
   :language: c++
   :noindent:
   :frag:     SNIPPET-SETAROMODEL

.. oe:source:: MDLQuerySearchSingleMatch.py
   :language: python
   :noindent:
   :frag:     SNIPPET-SETAROMODEL

.. oe:source:: MDLQuerySearchSingleMatch.java
   :language: java
   :noindent:
   :frag:     SNIPPET-SETAROMODEL

.. oe:source:: MDLQuerySearchSingleMatch.cs
   :language: csharp
   :noindent:
   :frag:     SNIPPET-SETAROMODEL

If the aromaticity model is not specified for the input files, then the **OpenEye**'s
aromaticity model is used by default.
For more information about the various aromaticity models of **OEChem TK**
see :ref:`chapter_aromaticity`.

.. oe:source:: MDLQuerySearchSingleMatch.cpp
   :language: c++
   :noindent:
   :frag:     SNIPPET-READQMOL

.. oe:source:: MDLQuerySearchSingleMatch.py
   :language: python
   :noindent:
   :frag:     SNIPPET-READQMOL

.. oe:source:: MDLQuerySearchSingleMatch.java
   :language: java
   :noindent:
   :frag:     SNIPPET-READQMOL

.. oe:source:: MDLQuerySearchSingleMatch.cs
   :language: csharp
   :noindent:
   :frag:     SNIPPET-READQMOL

In general, the aromaticity model chosen should be consistent between the
query and target molecules to be searched.
Using different aromaticity models may produce false negatives as aromatic
systems may be treated differently.
Section :ref:`section_mdlqueries_aromaticity` further explains the effects of
using various aromaticity models when performing a substructure search.

.. index:: OEReadMDLQueryFile

:oe:func:`OEReadMDLQueryFile<OEChem::OEReadMDLQueryFile>` function reads the
MDL query directly into a :oe:cls:`OEQMolBase<OEChem::OEQMolBase>` object ,
which then can be used to initialize an :oe:cls:`OESubSearch<OEChem::OESubSearch>`
instance.

.. index :: OEBuildMDLQueryExpression

The MDL query structure can also be read into a :oe:cls:`OEMolBase<OEChem::OEMolBase>`
object (see code snippet below).
In this case, the :oe:func:`OEReadMDLQueryFile<OEChem::OEReadMDLQueryFile>`
function attaches the query features present in the input MDL file to the related
atoms and bonds of the :oe:cls:`OEMolBase<OEChem::OEMolBase>` object.
The :oe:cls:`OEQMolBase<OEChem::OEQMolBase>` object can be subsequently created by
calling the :oe:func:`OEBuildMDLQueryExpressions<OEChem::OEBuildMDLQueryExpressions>`
function.


.. oe:language:: c++

   .. code-block:: c++

      OEGraphMol mol;
      OEReadMDLQueryFile(qfile,mol);
      // mol can be manipulated here
      OEQMol qmol;
      // build OEQMol with OEMDLQueryOpts::Default option
      OEBuildMDLQueryExpressions(qmol,mol);
      OESubSearch ss(qmol);


.. oe:language:: python

   .. code-block:: python

      mol = OEGraphMol()
      OEReadMDLQueryFile(qfile,mol)
      # mol can be manipulated here
      qmol = OEQMol()
      # build OEQMol with OEMDLQueryOpts_Default option
      OEBuildMDLQueryExpressions(qmol,mol)
      ss = OESubSearch(qmol)


.. oe:language:: java

   .. code-block:: java

      OEGraphMol mol = new OEGraphMol();
      oechem.OEReadMDLQueryFile(qfile,mol);
      // mol can be manipulated here
      OEQMol qmol = new OEQMol();
      // build OEQMol with OEMDLQueryOpts.Default option
      oechem.OEBuildMDLQueryExpressions(qmol,mol);
      OESubSearch ss = new OESubSearch(qmol);

.. oe:language:: csharp

   .. code-block:: csharp

      OEGraphMol mol = new OEGraphMol();
      OEChem.OEReadMDLQueryFile(qfile,mol);
      // mol can be manipulated here
      OEQMol qmol = new OEQMol();
      // build OEQMol with OEMDLQueryOpts.Default option
      OEChem.OEBuildMDLQueryExpressions(qmol,mol);
      OESubSearch ss = new OESubSearch(qmol);


The declaration of these functions are:

.. oe:language:: c++

   .. code-block:: c++

      bool OEReadMDLQueryFile(oemolistream &ifs, OEMolBase& mol)
      bool OEReadMDLQueryFile(oemolistream &ifs, OEQMolBase& qmol,
                              unsigned int opts = OEMDLQueryOpts::Default)
      bool OEBuildMDLQueryExpressions(OEQMolBase &qmol, const OEMolBase &mol,
                                      unsigned int opts = OEMDLQueryOpts::Default)


.. oe:language:: python

   .. code-block:: python

      OEReadMDLQueryFile(ifs,mol)
      # ifs-oemolistream, mol-OEMolBase
      # returns true or false
      OEReadMDLQueryFile(ifs,qmol,opts)
      # ifs-oemolistream, mol-OEQMolBase, opts-integer with OEMDLQueryOpts_Default
      # returns true or false
      OEBuildMDLQueryExpressions(qmol,mol,opts)
      # ifs-oemolistream, mol-OEQMolBase, opts-integer with OEMDLQueryOpts_Default
      # returns true or false


.. oe:language:: java

   .. code-block:: java

      boolean OEReadMDLQueryFile(oemolistream ifs, OEMolBase mol);
      boolean OEReadMDLQueryFile(oemolistream ifs, OEQMolBase qmol, int opts);
      // opts (default value = OEMDLQueryOpts.Default)
      boolean OEBuildMDLQueryExpressions(OEQMolBase qmol, OEMolBase mol, int opts);
      // opts (default value = OEMDLQueryOpts.Default)

.. oe:language:: csharp

   .. code-block:: csharp

      bool OEReadMDLQueryFile(oemolistream ifs, OEMolBase mol);
      bool OEReadMDLQueryFile(oemolistream ifs, OEQMolBase qmol, int opts);
      // opts (default value = OEMDLQueryOpts.Default)
      bool OEBuildMDLQueryExpressions(OEQMolBase qmol, OEMolBase mol, uint opts);
      // opts (default value = OEMDLQueryOpts.Default)


.. _section_mdlqueries_interpret:

MDL Query Interpretation
================================================================================

The `opts` parameter defines how the MDL query is interpreted when
an :oe:cls:`OEQMolBase<OEChem::OEQMolBase>` object is constructed.
The following options are present in the :oe:constns:`OEChem::OEMDLQueryOpts`
namespace:


#. :oe:const:`OEChem::OEMDLQueryOpts::Default`

   Only constraints explicitly specified in the MDL file are added to the
   :ref:`OEQMolBase <OEChem_OEQMolBase>` query structure.
   See section :ref:`section_mdlqueries_features` for the supported MDL
   query features.

#. :oe:const:`OEChem::OEMDLQueryOpts::SuppressExplicitH`

   This option controls how the explicit hydrogens of the query are matched
   to the explicit/implicit hydrogens of the target structures.
   For more information see :ref:`section_mdlqueries_hydrogens`.

#. :oe:const:`OEChem::OEMDLQueryOpts::AddBondAliphaticConstraint`

   If this option is specified, then an aliphatic query bond can only be mapped
   to the aliphatic bonds in the target structure.
   :ref:`Figure: Interpretation A<figure_mdlq_aliphatic>` shows how the MDL query
   structure is interpreted when the
   :oe:const:`OEChem::OEMDLQueryOpts::AddBondAliphaticConstraint`  option is used.

   Query 'A' will match all three of the target compounds displayed in
   :ref:`Figure: Interpretation A<figure_mdlq_aliphatic>`.
   If the *OpenEye* model is used to perceive aromatic rings, then query 'B'
   substructure is present only in target '2'.
   If the MDL aromaticity model is used, then target '3' is also a hit, since
   five-membered heterocycles are not considered aromatic in this model.
   For more information about different aromaticity models and their effects
   on substructure searching see section :ref:`section_mdlqueries_aromaticity`.

   .. _figure_mdlq_aliphatic:

   .. figure:: images/mdlq-ConstAliphatic.png
      :scale: 100
      :align: center

      **Interpretation A**

      *Interpretation of the MDL query with 'AddBondAliphaticConstraint' option.*

#. :oe:const:`OEChem::OEMDLQueryOpts::AddBondTopologyConstraint`

   By default, ring bonds in the query structure can only be mapped to ring bonds in the target,
   and chain bonds in the query can be mapped to both ring and chain bonds.
   The :oe:const:`OEChem::OEMDLQueryOpts::AddBondTopologyConstraint` option enables enforcement
   of bond topology constraint during substructure search i.e. ring bonds in the query can
   only be mapped to ring bonds, and chain bonds in the query can
   only be mapped to chain bonds.

   :ref:`Figure: Interpretation B<figure_mdlq_topology>` shows how the MDL query structure
   is interpreted when the :oe:const:`OEChem::OEMDLQueryOpts::AddBondTopologyConstraint`
   option is used.
   Query 'A' will match all three of the target compounds displayed in
   :ref:`Figure: Interpretation B<figure_mdlq_topology>`, while query 'B' is present
   only in target '3'.

   If the 'ring' property is specified in the MDL query file for a particular chain
   bond in the query structure, then its topology constraint is overridden.

   .. _figure_mdlq_topology:

   .. figure:: images/mdlq-ConstTopology.png
      :scale: 100
      :align: center

      **Interpretation B**

      *Interpretation of the MDL query with `AddBondTopologyConstrain' option.*

#. :oe:const:`OEChem::OEMDLQueryOpts::MatchAtomStereo`

   When this option is enabled, an S/R atom stereo configuration in the query will only match any S/R configuration in the target molecule,
   but not to a R/S (opposite) or an unspecified one. If an atom stereo configuration is undefined in the query, it is allowed to match to
   any target atom regardless of its stereo configuration.

#. :oe:const:`OEChem::OEMDLQueryOpts::MatchIsotope`

   When this option is enabled, query atoms with an isotope can match to a target atom only if they have the same atomic mass.
   If the query atom has no specified isotope number, it will match to any target atom regardless of its atomic mass.

.. _section_mdlqueries_features:

Supported MDL Query Features
================================================================================

The table below summarizes the MDL query features currently supported by the **OEChem TK**.
These query feature can only be read with the low level :oe:func:`OEChem::OEReadMDLQueryFile`
function and not the high-level :oe:func:`OEChem::OEReadMolecule` function.

.. _table_mdlqueries_summary:

.. list-table:: **Summary of supported MDL query features in V2000 and V3000**
   :widths: 30 30 100
   :header-rows: 1

   * - CTFile V2K
     - CTFile V3K
     - Notes
   * - ``M  CHG``
     - ``CHG=``
     - formal charge
   * - ``<atomblock>``
     - ``HCOUNT=``
     - number of allowed hydrogens V2K:0=off, 1=H0, 2-5 for H1-H4; V3K:0=off, -1=H0, 1-5 for H1-H5
   * - ``M  ISO``
     - ``MASS=``
     - mass difference
   * - ``M  RBC``
     - ``RBCNT=``
     - limit of number of allowed ring bonds attached to an atom
   * - ``M  SUB``
     - ``SUBST=``
     - number of allowed substitutions of an atom
   * - ``M  ALS``
     - ``[aaa,bbb,..]``
     - list alternative atom types for an atom
   * - ``<atomblock>``
     - ``STBOX=``
     - stereo care (e.g. for c/t doubles)
   * - ``L,A,Q``
     - ``L,A,Q``
     - L=atom list, A=Any atom except hydrogen, Q=Any atom except hydrogen and carbon
   * - ``M  UNS``
     - ``UNSAT=``
     - specifies whether or not an atom is unsaturated, *i.e.*, having at least one multiple bond
   * - ``<bondblock>``
     - ``STBOX=``
     - bond stereochemistry that is considered, if both ends of the bond are marked with
       *stereo care* flags in the atom block
   * - ``<bondblock>``
     - ``TOPO=``
     - bond topology (0=off, 1=ring, 2=chain)
   * - ``<bondblock>``
     - ``<bondblock>``
     - 4-aromatic, 5=single or double, 6=single or aromatic, 7=double or aromatic, 8=any

.. _figure_mdlq_example_query:

.. figure:: images/mdlq-Example-Query.png
   :scale: 100
   :align: center

   **Query**

   **Example of MDL query structure.**

.. seealso::

   * :ref:`chapter_mdlquery` chapter for the depiction of the supported query features
   * Accelrys `CTfile Formats  document <http://accelrys.com/products/collaborative-science/biovia-draw/ctfile-no-fee.html>`_
     for more information about the atom and bond query features

.. _section_mdlqueries_aromaticity:

Aromaticity
================================================================================

Perceiving aromaticity in the query and the target structures is important in
order to ensure that the result of a substructure search is independent of the
different |Kekule| representations of the participating structures.
A query bond which is part of an aromatic ring system can be mapped to any
aromatic bonds in the target.
:ref:`Figure: Aromaticity Match<figure_mdlq_aromaticity_Kekule>`  shows an example where both
|Kekule| representations of the benzene-1,2-diol substructure are present
in the two target structures.

.. _figure_mdlq_aromaticity_Kekule:

.. figure:: images/mdlq-Aromaticity-Kekule.png
   :scale: 100
   :align: center

   **Aromaticity Match**

   *The result of substructure search is independent of the Kekulé representations of the participating structures.*


Altering the aromaticity model will affect the results of a  substructure search.
:ref:`Figure: Aromaticity A<figure_mdlq_aromaticity_A>` --
:ref:`Figure: Aromaticity C<figure_mdlq_aromaticity_C>`
show several examples where different results were obtained by applying
the MDL and the OpenEye aromaticity models.
This is a consequence of the fact that in the MDL aromaticity
model, five-membered heterocycles are not considered aromatic.

.. note::

   It is highly recommended to apply the same aromaticity model to the
   query and the target structures.

:oe:src:`Listing 1<SNIPPET-MDLQuerySearchSingleMatch>` shows an example of how to
change the aromaticity flavor of input files.


.. _figure_mdlq_aromaticity_A:

.. figure:: images/mdlq-Aromaticity-A.png
   :scale: 100
   :align: center

   **Aromaticity A**

   *Example of a substructure search applying different aromaticity models.*


.. _figure_mdlq_aromaticity_B:

.. figure:: images/mdlq-Aromaticity-B.png
   :scale: 100
   :align: center

   **Aromaticity B**

   *Example of a substructure search applying different aromaticity models.*


.. _figure_mdlq_aromaticity_C:

.. figure:: images/mdlq-Aromaticity-C.png
   :scale: 100
   :align: center

   **Aromaticity C**

   *Example of a substructure search applying different aromaticity models.*


.. _section_mdlqueries_arom_generic:

Aromaticity with Generic Atoms
================================================================================

If a ring in the query structure contains generic atom(s) (see example in
:ref:`Figure: Generic atom example A<figure_mdlq_aromaticity_ring6_GA>`), then the aromaticity of the ring can
not be perceived.
In order to maintain the independence from the |Kekule| representation,
6-membered rings with alternating single/double bonds are assumed to be
aromatic.

.. _figure_mdlq_aromaticity_ring6_GA:

.. figure:: images/mdlq-Aromaticity-Mem6-GenericAtom.png
   :scale: 100
   :align: center

   **Generic atom example A**

   *Example of aromaticity assumption in a 6-membered ring with generic atoms.*


Similarly, a 5-membered ring with generic atom(s) is considered aromatic if it is
composed of two single and two double bonds.
See Example in :ref:`Figure: Generic atom example B<figure_mdlq_aromaticity_ring5_GA>`.


.. _figure_mdlq_aromaticity_ring5_GA:

.. figure:: images/mdlq-Aromaticity-Mem5-GenericAtom.png
   :scale: 100
   :align: center

   **Generic atom example B**

   *Example of aromaticity assumption in a 5-membered ring with generic atoms.*

.. _section_mdlqueries_hydrogens:

Explicit Hydrogens
================================================================================
During the substructure search, each query atom has to be mapped to a target atom
in order to detect subgraph isomorphism.
Therefore, a problem can arise if the query structure contains explicit hydrogens
or an atom list with hydrogens (see example in :ref:`Figure: Query<figure_mdlq_explicitH_query>`),
but the target structure has implicit hydrogens :ref:`Figure: Targets<figure_mdlq_explicitH_targets>`)

.. _figure_mdlq_explicitH_query:

.. figure:: images/MDLQueryExplicitH.png
   :align: center

   **Example of MDL query structure with explicit hydrogens**

.. oe:source:: MDLQuerySearchMapping.cpp
   :language: c++
   :caption:  Example of substructure search with accessing atom mapping
   :refnum:   2
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchMapping.py
   :language: python
   :caption:  Example of substructure search with accessing atom mapping
   :refnum:   2
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchMapping.java
   :language: java
   :caption:  Example of substructure search with accessing atom mapping
   :refnum:   2
   :frag:     SNIPPET

.. oe:source:: MDLQuerySearchMapping.cs
   :language: csharp
   :caption:  Example of substructure search with accessing atom mapping
   :refnum:   2
   :frag:     SNIPPET

This problem can be solved in two ways:

1. The explicit hydrogens in the query molecule can be suppressed during
   :oe:cls:`OEQMolBase<OEChem::OEQMolBase>` construction by using the
   :oe:const:`OEChem::OEMDLQueryOpts::SuppressExplicitH` option
   (see example in :oe:src:`Listing 1<SNIPPET-MDLQuerySearchSingleMatch>`).
   A query atom can only be mapped to a target atom if it has at least as many
   implicit hydrogens as explicit hydrogens were removed from the query atom.
   This solution is recommended only if the presence or absence of the query
   substructure is of interest, but not the complete mapping between the query and
   the target.
   The :oe:meth:`SingleMatch<OEChem::OESubSearch::SingleMatch>` function returns
   whether or not the query is present in the target, but the mapping is not
   accessible.

.. index:: OEAddExplicitHydrogens

2. If the complete mapping between the query and the target is of interest,
   the :oe:func:`OEAddExplicitHydrogens<OEChem::OEAddExplicitHydrogens>` function
   has to be called for each target structure before performing the substructure
   search.
   (See example in  :oe:src:`Listing 2<SNIPPET-MDLQuerySearchMapping>`).


In both cases, the query presented in :ref:`Figure: Query<figure_mdlq_explicitH_query>` will match
only target 'C' and 'D' shown in :ref:`Figure: Targets<figure_mdlq_explicitH_targets>`.
:ref:`Figure: Atom mapping<figure_mdlq_explicitH_target_mapping>` shows the three
detected substructures in target 'C' when adding explicit hydrogens to the target structures.

The execution of the substructure search is significantly faster if the hydrogens
are suppressed in the target structures, since the search space can be an
order of magnitude smaller.

.. _figure_mdlq_explicitH_targets:

.. figure:: images/MDLQueryExplicitHMatch.*
   :align: center

   **Targets -- Example of target structures with implicit hydrogens**

.. _figure_mdlq_explicitH_target_mapping:

.. figure:: images/MDLQueryExplicitHMapping.*
   :align: center

   **Atom mapping -- Example of substructure search with atom mapping**
