Synthesis of Mathematical programs from Natural Language Specifications. (arXiv:2304.03287v1 [cs.AI])

Several decision problems that are encountered in various business domains
can be modeled as mathematical programs, i.e. optimization problems. The
process of conducting such modeling often requires the involvement of experts
trained in operations research and advanced algorithms. Surprisingly, despite
the significant advances in the methods for program and code synthesis, AutoML,
learning to optimize etc., there has been little or no attention paid to
automating the task of synthesizing mathematical programs. We imagine a
scenario where the specifications for modeling, i.e. the objective and
constraints are expressed in an unstructured form in natural language (NL) and
the mathematical program has to be synthesized from such an NL specification.
In this work we evaluate the efficacy of employing CodeT5 with data
augmentation and post-processing of beams. We utilize GPT-3 with back
translation for generation of synthetic examples. Further we apply rules of
linear programming to score beams and correct beams based on common error
patterns. We observe that with these enhancements CodeT5 base gives an
execution accuracy of 0.73 which is significantly better than zero-shot
execution accuracy of 0.41 by ChatGPT and 0.36 by Codex.