Reproducibility in Evolutionary Computation. (arXiv:2102.03380v1 [cs.AI])

Experimental studies are prevalent in Evolutionary Computation (EC), and
concerns about the reproducibility and replicability of such studies have
increased in recent times, reflecting similar concerns in other scientific
fields. In this article, we suggest a classification of different types of
reproducibility that refines the badge system of the Association of Computing
Machinery (ACM) adopted by TELO. We discuss, within the context of EC, the
different types of reproducibility as well as the concepts of artifact and
measurement, which are crucial for claiming reproducibility. We identify
cultural and technical obstacles to reproducibility in the EC field. Finally,
we provide guidelines and suggest tools that may help to overcome some of these
reproducibility obstacles.