Geodesics, non-linearities and the archive of novelty search

• DOI: 10.1145/3520304.3528944
• Publication Date: 2022-07-19T15:29:44Z
• Authors: Achkan Salehi; Alexandre Coninx; Stephane Doncieux
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; 0202 electrical engineering, electronic engineering, information engineering; Computer Science - Neural and Evolutionary Computing; [INFO]Computer Science [cs]; Neural and Evolutionary Computing (cs.NE); 02 engineering and technology; Machine Learning (cs.LG)

The Novelty Search (NS) algorithm was proposed more than a decade ago. However, the mechanisms behind its empirical success are still not well formalized/understood. This short note focuses on the effects of the archive on exploration. Experimental evidence from a few application domains suggests that archive-based NS performs in general better than when Novelty is solely computed with respect to the population. An argument that is often encountered in the literature is that the archive prevents exploration from backtracking or cycling, i.e. from revisiting previously encountered areas in the behavior space. We argue that this is not a complete or accurate explanation as backtracking - beside often being desirable - can actually be enabled by the archive. Through low-dimensional/analytical examples, we show that a key effect of the archive is that it counterbalances the exploration biases that result, among other factors, from the use of inadequate behavior metrics and the non-linearities of the behavior mapping. Our observations seem to hint that attributing a more active role to the archive in sampling can be beneficial.<br/>4 pages, 3 figures<br/>