Title: Difficulties in Performance Comparison of Evolutionary Multi-Objective and Many-Objective Optimization Algorithms
Abstract: A large number of non-dominated solutions (instead of a single final solution) are usually obtained by applying an evolutionary multi-objective optimization (EMO) algorithm to a multi-objective problem. Thus the comparison of different EMO algorithms needs the comparison of different solution sets. The quality of a solution set is usually measured by performance indicators such as the hypervolume (HV) and the inverted generational distance (IGD). In this talk, we discuss difficulties in the comparison of different EMO algorithms through such a performance indicator. First, we briefly explain evolutionary multi-objective and many-objective optimization. Next we discuss the specification of the population size for comparing different EMO algorithms. In general, an appropriate specification of the population size is different for each EMO algorithm. Moreover, some algorithms have an external archive population in addition to the main population whereas others have only the main population. As a result, the number of obtained non-dominated solutions by a different EMO algorithm is totally different. This makes fair comparison very difficult. We explain how fair comparison can be performed when the number of obtained non-dominated solutions by each EMO algorithm is totally different.. Then we discuss difficulties related to the use of the HV indicator. For example, it is explained that HV-based comparison results strongly depends on the specification of a reference point for HV calculation. It is also explained that a set of uniformly distributed solutions over the entire Pareto front is not always the best distribution of solutions for HV maximization. Those difficulties become more serious when EMO algorithms are compared for many-objective optimization. Dependency of IGD-based comparison results on the specification of reference points and the optimal distribution of solutions for IGD minimization are also explained.
Short Bio: Dr. Ishibuchi received the BS and MS degrees from Kyoto University in 1985 and 1987, respectively. In 1992, he received the Ph. D. degree from Osaka Prefecture University where he was a professor since 1999. From April 2017, he is with Department of Computer Science and Engineering, Southern University of Science Technology (SUSTech), Shenzhen, China as a Chair Professor. He received a Best Paper Award from GECCO 2004, HIS-NCEI 2006, FUZZ-IEEE 2009, WAC 2010, SCIS & ISIS 2010, FUZZ-IEEE 2011, ACIIDS 2015 and GECCO 2017. He also received a 2007 JSPS (Japan Society for the Promotion of Science) Prize. He was the IEEE CIS Vice-President for Technical Activities (2010-2013), an IEEE CIS Distinguished Lecturer (2015-2017), and the President of the Japan EC Society (2016-2018). Currently, he is the Editor-in-Chief of IEEE CI Magazine (2014-2019) and an IEEE CIS AdCom member (2014-2019). He is also an Associate Editor of IEEE TEVC, IEEE Access and IEEE T-Cyb. He is an IEEE Fellow. In 2018, he was selected in the “Recruitment Program of Global Experts for Foreign Experts” known as the “Thousand Talents Program” in China. At present, he stands as the Editor-in-Chief of IEEE CI Magazine (2014-2019) furthermore he is an IEEE CIS AdCom member (2014-2019).