Synergistic Fusion of Graph and Transformer Features for Enhanced Molecular Property Prediction. (arXiv:2310.03027v1 [physics.chem-ph])

Molecular property prediction is a critical task in computational drug
discovery. While recent advances in Graph Neural Networks (GNNs) and
Transformers have shown to be effective and promising, they face the following
limitations: Transformer self-attention does not explicitly consider the
underlying molecule structure while GNN feature representation alone is not
sufficient to capture granular and hidden interactions and characteristics that
distinguish similar molecules. To address these limitations, we propose SYN-
FUSION, a novel approach that synergistically combines pre-trained features
from GNNs and Transformers. This approach provides a comprehensive molecular
representation, capturing both the global molecule structure and the individual
atom characteristics. Experimental results on MoleculeNet benchmarks
demonstrate superior performance, surpassing previous models in 5 out of 7
classification datasets and 4 out of 6 regression datasets. The performance of
SYN-FUSION has been compared with other Graph-Transformer models that have been
jointly trained using a combination of transformer and graph features, and it
is found that our approach is on par with those models in terms of performance.
Extensive analysis of the learned fusion model across aspects such as loss,
latent space, and weight distribution further validates the effectiveness of
SYN-FUSION. Finally, an ablation study unequivocally demonstrates that the
synergy achieved by SYN-FUSION surpasses the performance of its individual
model components and their ensemble, offering a substantial improvement in
predicting molecular properties.