Robust Swarm Robotics System Using CMA-NeuroES with Incremental Evolution

Swarm robotics (SR) [1-3] is a novel approach inspired by the observation of social insects, such as ants and wasps. These examples of social insects show that simple individuals can successfully accomplish difficult tasks when they coordinate as a group. This kind of systemlevel behaviour, which appears to be robust, scalable and flexible, is impressive to researchers working on robotics. Similarly to these social insects, SR systems (SRSs) are expected to accomplish tasks beyond the capabilities of a single robot. By definition, an SRS comprises a number of relatively simple and typically homogeneous robots that a desired collective behaviour emerges from the local interactions among the agents and between the agents and the environment, similar to the social insects.


Introduction
Swarm robotics (SR) [1][2][3] is a novel approach inspired by the observation of social insects, such as ants and wasps. These examples of social insects show that simple individuals can successfully accomplish difficult tasks when they coordinate as a group. This kind of systemlevel behaviour, which appears to be robust, scalable and flexible, is impressive to researchers working on robotics. Similarly to these social insects, SR systems (SRSs) are expected to accomplish tasks beyond the capabilities of a single robot. By definition, an SRS comprises a number of relatively simple and typically homogeneous robots that a desired collective behaviour emerges from the local interactions among the agents and between the agents and the environment, similar to the social insects.
The expression swarm intelligence was first conceived by Beni to denote a class of cellular robotic systems in 1980s. These works used many simple agents occupy one-or two-dimensional environment to generate patterns and self-organize their nearest neighbour interactions. At that time, the definition swarm intelligence only marginally covers works on cellular robotic systems, which does not take the inspiration from social insect behaviour. Recently, the expression "swarm intelligence" moved on to cover a wide range of researches from optimization to social insect studies, losing its robotics context in the meantime. Nowadays, the term SR has started to be used as the application of swarm intelligence to multi-robot systems. This concern was first explicitly started by Sahin in 2005 [2].
As previously mentioned before, an SRS must have three functional properties at the system level that are observed in natural swarms: Robustness is the ability to operate despite disturbances resulting from the malfunctioning of its individuals. For instance, lost individuals can be immediately replaced by others, with the operation will continuing smoothly. This is seen as the key advantage of the SRS approach [4,5].
Flexibility is the ability of an SRS to generate modularized solutions to various tasks, meaning that an SRS must be able to adapt their behaviours to different environments.
Scalability is the ability of an SRS to operate with a wide range of group sizes and support a large number of individuals.
The concept of swarm engineering was introduced by Kazadi [6] in 2000 and the first formal introduction of swarm engineering was released by Winfield et al. in 2004 [7]. Researchers indicated that finding a predictable and controllable design methodology for swarms is the core research direction of swarm engineering [8,9]. Today, although swarm engineering is still in a very early stage, core topics of swarm intelligence, the design, and analysis have already received attention from SR researchers. The notable swarm-bots project [10] was begun in 2001 and terminated in 2005, and was followed by the swarmanoid project in 2006 [11]. New approaches to the design and implementation of self-organizing and the self-assembling problem of autonomous robots were studied in the project [12].
In this study, a cooperative food foraging problem with obstacles in the environment is investigated. We have augmented the covariance matrix adaptation evolution strategy (CMA-ES) with an artificial neural network to create an efficient approach, CMA-NeuroES, for an SRS to solve simple food foraging problems [13]. However, when an evolutionary approach meets a complex task, it is typical that a simple ER strategy will face a situation that all individuals of the first generation are scored with the same null value; moreover, the selection process cannot operate as expected. This bootstrap problem occurs very often with difficult tasks. To avoid this kind of failure, an incremental evolution approach with staged evolution and environmental complexification for a cooperative food foraging task is adopted.
In addition, an SRS can work dynamically as the individual robots are deployed respectively. This kind of decentralized control ensures that SRS has no common failure point. The failure of individual robots will hardly affect SRS performance. The resulting high-level robustness contrasts with the high engineering cost of fault tolerance in conventional robotics systems, and this is free as a basic property of an SRS. Consequently, we compare the best controllers evolved by CMA-NeuroES with two other evolutionary algorithms, fast evolution strategies (FESs) [14,15] and real-coded genetic algorithms (GAs), [16] through random breakdown tests in computer simulations. As an intrinsic property of SRS, we want to find out whether the loss of some individuals can be compensated for by others or not and also whether the destruction of a particular part of the swarm is will stop its operation or not. This paper is organized as follows: in Section 2, the related work to SRS and the benchmarks of SRS are presented. Section 3 introduces the cooperative food foraging problem we use in our experiments. Section 4 explains neuro evolution based on a CMAES, and a pre-experiment on a CMA-NeuroES controller for cooperative food foraging problem is described. In Section 5, we explain why and how we apply incremental evolution to a cooperative food foraging problem. Section 6 discusses the computer simulation setup and the results of our proposed method. Section 7 draws conclusions and describes our future studies.

Related Works
SRS's design methods can be divided into two categories [17]. (i) Behaviour based design, where in the individual behaviours of robots are designed by hand, is the most commonly used design method. Researchers in this field have proposed various behaviour-based design approaches for controlling an SRS. A typical example can be found in Kube and Zhang [18]. They used a design method, i.e., task modelling, where in a robot controller with a finite state machine is carefully designed by a human programmer without a global controller. In this case, individual behaviours were iteratively adjusted until the desired collective behaviour was obtained. They demonstrated its effectiveness in box-pushing problems, wherein the collective behaviour was obtained after individual behaviours were iteratively adjusted and tuned [19]. However, this approach might have a limitation in problem complexity since no other applications have been published since then. (ii) Automatic design methods are famous for applying evolutionary robotics (ER) that add artificial evolution to robotic systems with a sensory-motor interface to the environment, i.e., evolving a robot controller represented as an artificial neural network [20][21][22].
Although the evolutionary robotics approach had been successfully applied to the single robot domain, it recently has been used for evolving group collective behaviours and the performance of an evolutionary computation approach is strongly dependent on the performance of the artificial neural network optimization [23]. Reynolds was among the first to apply evolutionary robotics techniques to collective behaviour making in 1993. He improved on his early work on the simulation of the flocking behaviour of birds, i.e., the boids. Visual apparatus and the control system was evolved to avoid collisions and to escape from the predator. Based on the experiment of the boids, Ward et al. evolved e-boids that groups of artificial fish capable of displaying schooling behaviour in 2001. In these studies above, the author reported that the creatures were not explicitly rewarded for coordinated motion. On the other hand, Quinn explored two ways of evolving controllers for coordinated motion behaviour by using two simulated Keeper robots. The first approach called clonal emphasize the member of the group are homogeneous and share the same geno type. The second approach called a clonal requires each member of the group with different genotypes, which means a heterogeneous group. The results indicate that alcohol evolution got better performance than clonal evolution. However, the authors report the heterogeneous approach may not be suitable when the group size becomes larger and the role allocation in the group may be not clear. Perez-Uribe et al. successfully evolved small groups of artificial ants for a simple foraging task by simulation to prove homogeneous groups achieve a better performance in 2003.
The collective behaviours of an SRS can be divided into four main categories: spatially-organizing, navigation, collective decision − making, and other collective behaviours. SRS researchers could use these basic collective behaviours to work on complex problems, for example, cooperative food foraging problems [24]. Spatially-organizing behaviours focus on how to organize and distribute robots and objects in space, which an SRS could organize and distribute in several ways: for instance, aggregation is the simplest spatial organization of robots in an SRS that are spatially close to each other in an environment. Navigation behaviours focus on how to organize and coordinate the movements of an SRS. These behaviours include collective exploration, coordinated motion, and collective transport, which allow an SRS to explore an environment, coordinate similarly to a flock of birds, or cooperate to transport an object that is too heavy for a single robot. Collective decision making behaviours focus on how a group of robots influence each other when making choices. For example, to maximize an SRS's performance, task allocation can be specialized by the robots themselves over different tasks. The behaviours that cannot be categorized are called other collective behaviours [25,26].
These collective behaviours are basic behaviours that can be combined to take over complex real world applications [27]. In cooperative food foraging problems, an SRS requires the most basic behaviours among benchmark problem in SRS. Self-organization of swarm behaviour is needed to cooperate and to move heavy food sources, cooperatively. It also requires navigation behaviour to search for food sources and to find a way back to the nest. Last but not least, decision-making behaviours allow robots in a cooperative food foraging problem to change their roles in seconds [28].

Problem formulation
The cooperative food foraging problem was inspired by the behaviour of ants searching for food sources and bringing the food to the nest. The task is to find better search strategies that maximize the ratio of bringing food to the nest in a specified environment [29,30]. Figure 1 shows the food foraging problem we investigate in this paper. The field is a 5,000 × 5,000 length square unit. The nest, a 1,000 × 1,000 square unit goal area, is located at the centre of the field. One hundred autonomous mobile robots are randomly placed in the nest as the initial condition. Three food sources, F, are randomly placed in the field. Every robot is set to be able to move a food source up to a five-unit weight. However, all of the food sources are 24-unit weight, which means that one food source requires at least five robots to move it cooperatively in a specific direction. A new food source appears soon after one food source is collected during 5,000 time steps. Three obstacles are fixed in the field at a given point that we set in the field. The large static friction we set for each obstacle are impossible for robots to move it, which means the SRS must avoid these obstacles and maximize the ratio of bringing food sources to the nest. The goal of cooperative food foraging task is that SRS should collect as many food sources as possible.

Robot setup
The SRS in this paper is assumed to be homogenous, i.e., all the robots in the system are assumed to have the same specifications, as shown in Figure 2. Each robot is 50 length units in diameter and has two types of sensors: eight infrared (IR) sensors and an Omni Vision camera. The eight IR sensors are arranged around a robot. 4 IR sensors equally distributed in the front of the robot, and 2 IR sensors are equally distributed in the back of robot. The other 2 IR sensors are set at two sides of the robot, separately. Each IR sensor provides a value that is inversely proportional to the distance to an object, which might be a food source, an obstacle, a wall, or other robots within the sensor range of 64 length units. The values are normalized between zero and one. The Omni-Vision camera is located at the centre of each robot. The robot's sensor abilities are summarized as follows: • Distance from an IR sensor to objects: O i (i=0, 1, · · ·, 7).
• Distance and direction to the nearest robot: r R1 , sin θ R1 and cos θ R1 .
• Distance and direction to the second nearest robot: r R2 , sin θ R2 and cos θ R2 .
• Distance and direction to the nearest food source: r F1 , sin θ F1 and cos θ F1 .
• Distance and direction to the second nearest food source: r F2 , sin θ F2 and cos θ F2 .
• Direction to the nest: sin θ N and cos θ N .
• Global direction of the robot: sin θ D and cos θ D .
Information obtained by the two types of sensor forms an input layer of a robot controller comprising 24 inputs connected to a motor on the right and another motor on the left that controls two differential driven wheels, enabling robot to move forward or to turn left or right using the rotational difference between wheels. In addition, each input neuron receives Gaussian noise, whose mean and standard deviation (SD) are 0 and 0.03, respectively. Four fully inter-connected hidden layers are adopted from our preliminary experiments for computer simulation. Because the two motor wheels are controlled by EANN output, the output layer consists of two neurons. The neurons of recurrent artificial neural networks (RANN) are connected as shown in Figure 3, as in our previous study [30]. Therefore, the number of synaptic connections is 162. All robots are assumed to have the same RANN controller.

Neuroevolution based on CMA-ES for cooperative food foraging problem
CMA-NeuroES: CMA-ES is a stochastic, iterative method for difficult nonlinear and non-convex optimization problems. It has been proven to be a powerful evolutionary optimization algorithm for a variety of test functions, and benchmark problems, and it performs especially well in searching landscapes with discontinuities, noise, and local optima [31,32]. CMA-ES was introduced by Hansen Ostermeier in 1996 [33], and its use of covariance matrix adaptation made this evolution strategy a highly elaborate optimization algorithm. After weighted recombination was introduced to CMA-ES in 2001 [34], the so-called rank-μ-update greatly reduced time complexity [35] in 2003. The performance of CMAES was improved after researchers found that increasing the population size can enhance global search characteristics [36]. In 2008, Raymond and Hansen presented a new approach that reduces evaluation time and space complexity for CMA-ES [37].   In CMA-ES, the offspring for the next generation (g+1) are generated by sampling a multivariate normal distribution with mean m ∈ n R and covariance C ∈ n n R × [38]. Each solution point ( ) at generation (g+1) in this algorithm presents an n-dimensional real-valued decision variable vector. These variables are altered by recombination and mutation, which correspond to the calculation of the mean value of μ solution points selected from offspring λ. In this algorithm, mutation is used to add a normally distributed random vector with zero mean, and the covariance matrix is updated during evolution to improve searching. Formally, solution points This is realized by adding a zero-mean random vector drawn from a multivariate normal distribution specified with step size σ and covariance matrix ( ) is the mean value of the population in generation g, and N(g) k (0, σ ( ) g 2 C ( ) g ) is a multivariate normal distribution with zero mean and covariance matrix C in the g th generation.
CMA-ES efficiency is provided by self-adaptation of C and σ. This allows CMA-ES to search efficiently in a highly correlated search space. For details, see reference [33,39,40].
CMA-NeuroES is a weight-evolving artificial neural network that applies CMA-ES to weight optimization. Since the adaptation of C allows efficient searching in the existence of correlation between parameters, we expect that CMA-NeuroES will show good performance on the optimization of the synaptic weights for our robot controller [31,41,42]. Every robot in our SRS has the same type of CMA-NeuroES controller. Each robot receives 16-input information from the environment. Swarm behaviour fitness is calculated from a fitness table to evaluate a robotic swarm and update the mean value m, covariance matrix C and global step-size σ. CMA-NeuroES operates in five steps: Step 1: Set all synaptic neural network weights randomly at the initial generation. If it is not the first generation, create offspring from (1).
Step 2: Start the simulation, then evaluate the fitness of λ offspring by using the fitness table.
Step 3: Send fitness and μ parents to CMA NeuroES to create new offspring, and update all synaptic weights.
Step 4: Choose synaptic weights with higher fitness as parents for the next generation.
Step 5: Repeat Step 1 to start a new generation until the terminal condition is met.

Incremental evolution
In evolutionary robotics approaches, the situation where no initial search pressures exist can occur when solving highly complex tasks. The result of our experiment on CMA-NeuroES with conventional evolution for the cooperative food foraging problem shows that there are three runs wherein the SRS collected nothing. This situation, the bootstrap problem in ER, occurs when all of the individuals in the initial generation are scored with null fitness prohibiting the progress of evolution. Overcoming the bootstrap problem is one of the difficulties in the ER approach. Incremental evolution is an approach for solving bootstrap problems in highly complex tasks with evolutionary approaches. Mouret and Doncieux [43] categorized incremental evolution into four main approaches: staged evolution, environmental complexification, behavioural decomposition, and fitness shaping. Staged evolution is an approach in which an objective task is divided into ordered sub-tasks, with every sub-task having a corresponding fitness function. A navigation task performed with staged evolution was presented by Bajaj and Ang Jr. [44]. A mobile robot was placed in a simple environment wherein only one obstacle existed. The fitness value was calculated using a straight navigation component and an avoiding obstacles component. At a later stage, the robot was placed in a more complex environment, in which closer walls and sharp turns had been added to the environment. The fitness value was calculated in the same manner as that in the first stage. In the third and final stages, the fitness value was calculated as the product of the value calculated in the previous stage and the wall-following factor. The final result was that the robot acquired the wall-following behaviour.
Environmental complexification works on a fitness value calculation in which the task complexity can be continuously modified by operating on certain parameters. A typical example was presented by Gomez and Miikkulainen [45] in 1997. The task was for a predator whose behaviour was controlled by an evolving artificial neural network to capture a prey within a fixed number of time steps. Behavioural decomposition is an approach in which the robot controller is divided into sub controllers. Every robot controller is evolved separately to solve a sub-task. Nardi et al. [46] evolved a position controller for an autonomous helicopter with three phases of incremental evolution. In the first phase, a simple yaw controller was evolved. In the next phase, the rest of the controller, comprising three modules, specifically, guidance, pitch, and role modules, was evolved independently. In the final phase, these modular controllers were simultaneously evolved to enable them to adapt to each other.
Fitness shaping uses a weighted sum of multiple evaluation criteria to create a fitness gradient for artificial evolution to follow. Nolfi and Parisi [47] evolved an autonomous robot that picks up objects. They used a fitness formula with five components, which correspond to the following scenarios: the robot is approaching the target object, the target object is in front of the robot, the robot tries to pick up the object, the robot has the object in its grasp, and the robot releases the object outside the area.
To improve the performance of the SRS in solving a complex cooperative food foraging problem, we proposed staged evolution with environmental complexification using the CMA-NeuroES approach. In our cooperative food foraging task, we assume that three basic behaviours, (1) food-exploration, (2) food-transportation, and (3) obstacle avoidance, are required to solve our problem. Therefore, three stage incremental evolutions were provided, as shown in Figure 4.
Sub-Task 1 is a very simple problem, in which all three food sources are placed in the field without any obstacles in the environment. Every food source in Sub-task 1 requires at least three robots to move it (The dynamical friction for every food sources is 14 power units). The expectation is that SRS will acquire the basic behaviour of food exploration and food-transportation to the nest, i.e., collects three food sources. When the SRS solved Sub-Task 1, Sub-Task 2, in which two obstacles are added to the field and the positions of food are changed is given to the SRS. The third basic behaviour of obstacle avoidance will be acquired after Sub-Task 2. At that time, every food sources needs at least four robots to move it (We increased the dynamical friction to 19 power units). Our simulation will then randomly add a source after the first food source is collected. When the SRS has solved Sub-Task 2, a final task, Goal Task, in which two new obstacles are placed into the field with a narrow path between them, is posed. In that case, food sources are too large to be moved through the narrow path. This trap makes our cooperative food foraging task much more difficult. The SRS learns more advanced food-transportation through obstacle avoidance behaviour. In Goal Task, new food sources are randomly created after each food source has been collected. Every food source in Goal Task requires at least five robots to move it. In our cooperative food foraging task, task transitions to the next sub-task occur only when the SRS has solved the current sub-task continuously for ten generations. The number of generations for task-transition has been optimized in our preliminary experiment.
The performance of SRS was also evaluated using the four components shown in Table 1. In the case of Sub-Task 1, the fitness value f of the SRS is calculated as f 1 +f 2 +f3+f4. In the case of Sub-Task 2, the fitness value f is calculated as f 2 +f3+f4. The f 1 is omitted; because the SRS has already learned the aggregation behaviour for food sources through Sub-Task 1. In the case of Goal Task, the simulation will run 5,000 time steps to see how many food sources the SRS can collect. The fitness value of Goal Task f is calculated as f 2 +f 3 , because the SRS has already learned to touch food sources, and food sources will be added to the field continually during the 5,000-time step simulation.

CMA-NeuroES controller for cooperative food foraging problem
The performance of SRS is depicted by four components shown in Table 1. The SRS collects 0.0015 at each robot and each time step when a robot touches one of the food sources. The sum of the points is set at the f1 component. The SRS collects a bonus point each time the swarm successfully returns to the nest with a food source. The sum of the points is set at the f 2 component. However, since there can be cases wherein they cannot finish bringing the food source to the nest within the time limit, partial evaluation for moving a food source is considered. For each food source, the points awarded are calculated as 1,500 × (1 -drem/dinit) at the end of the run, where drem and dinit, the remaining distance to the nest and the initial distance from the nest, respectively, are produced as points. The sum of these points is set as the f 3 component. When all the food sources have been moved to the nest, the f 4 component is calculated as 1.0 × [remaining time steps] when the task is achieved. Otherwise, f 2 +f3+f4 are evaluated as zero. The CMA-NeuroES parameter setting is as follows. The offspring λ are set at 100, and the initial SD is set at 0.2, with the initial covariance matrix C= I. The computer simulations' last generation is set at 500 and 10 independent experimental runs are conducted.

Applying incremental evolution to CMA-NeuroES for the cooperative food foraging problem
In our compearation computer simulation, (μ, λ)-FastES (FES) [48,49] and a real-coded GA [50][51][52][53] were also used to evolve the synaptic connection weights of the artificial neural network that generated the robots' actions. To make the experiment comparable, four approaches are proposed to solve the cooperative food foraging problem: CMA-NeuroES with conventional evolution, CMA-NeuroES, FES, and realcoded GA with incremental evolution. The parameter settings of the other evolutionary algorithms are as follows. The real-coded GA's population size is also set at 100. Tournament selection with size two and elite preservation with size one are adopted. The mutation rate is set at 1.0. This means that all the synaptic connections are mutated for each generation by adding Gaussian noise, whose mean and SDs are 0 and 0.05, respectively. No crossover was used. These parameter tunings had been performed in our preliminary experiments. All the last generation of the artificial evolution are set at 500, and ten independent experimental runs were conducted.

Robustness test
The robustness of the best robot controllers with each approach was measured by conducting a breakdown test with the Goal Task ( Figure  4c). In this test, the robots with the best controllers of each approach were selected. The fact that an SRS can work dynamically as individual robots are deployed has the advantage that the failure of indi560 visual robots will hardly affect the performance of an evolved SRS. In our test, the SRS continued its collective behaviour for searching food sources and returning food sources to the nest, even after a few robots had stopped working. Every robot is tested to determine whether it is broken at every time step. The breakdown coefficient (Bc) is calculated as follows: r S (1) In this equation, r S is the stop rate, the s R are the random steps from 0 to 5,000, and N is the number of robots. The test system will decide if any robot is broken by comparing c B with a uniformly distributed double value between 0.0 and 1.0 from a random number generator's through at every time step. If c B is larger than the random number, then the robot will stop. All the broken robots remain in the field and can be detected by robot sensors. Stop counter c S will count one after a robot is stopped to control the number of broken robots. Our breakdown simulation runs 5,000 time steps for ten iterations. Moreover, we consider the limitation of the stop counter by 10,20,30,50, meaning that 10, 20, 30, 50 robots in the SRS, respectively, will be stopped randomly during the simulation. Figure 5 shows the result of the CMA-NeuroES controller for the cooperative food foraging problem. Our SRS successfully collected food sources in seven of ten runs and the maximum number of collected food sources was five. However, three runs performed very poorly; in them the SRS collected nothing at all. Figure 6 shows the fitness transitions of the best individuals and the averages of each individual in f1 f2

Results
Touching a food source A food source reaches the nest A food source is moved toward the nest +1500 All foods reach the nest +1.0 × [remaining time steps]   the best run. Figure 7 shows that the incremental evolution approaches with evolutionary algorithms collected at least two food sources, indicating that they were successful in solving the Sub-Tasks for all the runs. The best run of CMA NeuroES collected eight food sources in Goal Task, whereas conventional evolution had three runs that collected nothing. It is clear that not only the maximum fitness but also the average fitness of CMA NeuroES with incremental learning is higher than those of others.  Table 3 shows the results of the average returned food source numbers, and the SD of four approaches for ten iterations. As a result, we see that conventional evolution performs poorly. When 50 robots stopped during our simulation, only one food source could be returned to the nest. In the incremental approach, the number of returned food sources of FES decreased to two, when the stopped robots number increased from 10 to 50. Real-coded GA shows its robustness, because the returned food source numbers decreased from four to three and the SD shows its stability. However, CMA-NeuroES with incremental evolution performed best overall. When ten robots in the SRS stopped, the SRS could still return at least five food sources to the nest. The robust of CMA-NeuroES enables it to return four food sources even when half of the robots in the SRS are stopped.
Typical behaviour observed in robustness tests for an SRS with a CMA-NeuroES controller is shown in Figure 8, wherein 50 robots are stopped during the simulation. Some robots immediately find a food source (Marked 2) after leaving the nest (Figure 8a and 8b). At the same time, some robots are stopped at the beginning of our robustness tests and become obstacles in the field. In Figure 8c, another food source (Marked 1) is found by another group of robots, when the first group of robots is trying to return the food sources to the nest. The food source (Marked 1) is collected after our SRS successfully bypassed the narrow pass in the field (Figure 8c-8e) while two food sources (Marked 2, 3) are already near the nest. An additional food source (Marked 4) is added to the field as soon as the first food source is collected (Figure 8f). Nearly half of the robots are stopped at this moment, after the food sources (Marked 4 and 5) are collected (Figure 8h and 8i). At the end of the simulation, 50 robots in our SRS have been stopped. Two groups of robots are still trying to collect the food sources (Marked 7 and 8) as the simulation 660 ends (Figure 8j).

Conclusion
In this paper, we point out that the definition of swarm intelligence was extended in 2004 and there is almost no relationship between the cellular robotic systems with SRS. After that we successfully applied the ER approach to a specific complex cooperative food foraging problem with a large SRS including one hundred homogenous robots. Swarm behaviour emerged as a result of CMA-NeuroES with incremental evolution. The incremental evolution approach of staged evolution and environmental complexification helps ER avoid the bootstrap problem. The result of incremental evolution outperforms the conventional evolution approach for cooperative food foraging. In addition, a robustness test confirmed that the incremental evolution approach with CMA-NeuroES is robust, because it can solve the same cooperative food foraging problem, even when half of the robots are stopped. We expect that our proposed method and robustness test will also hold for other SRS benchmarks. In the future, we will focus primarily on the analysis of SRSs [54][55][56][57]. As an SRS can be considered as a large network with interactions among robots, we investigated finding subgroups in a robotic swarm [58] using the technology of complex networks. The next step will be to describe how subgroups in an SRS develop and change in a large robotic swarm using a duration table. That will enable researchers to understand the detail of task allocation in an SRS from a macroscopic viewpoint.