International Journal of Swarm Intelligence and Evolutionary Computation

We propose a novel hybrsid algorithm named, FOA-FA to solve the nonlinear programming problems (NLPPs). The main feature of the hybrid algorithm is to integrate the strength of fruit fly optimization algorithm (FOA) in handling continuous optimization and the merit of firefly algorithm (FA) in achieving robust exploration. The methodology of the proposed algorithm consists of two phases. The first one employs a variation on original FOA employing a new adaptive radius mechanism (ARM) for exploring the whole scope around the fruit flies locations to overcome the drawbacks of original FOA which has been continues for the nonnegative orthant problems. The second one incorporates FA to update the previous best locations of fruit flies to avoid the premature convergence. The hybrid algorithm speeds up the convergence and improves the algorithm’s performance. The proposed FOA-FA algorithm is tested on several benchmark problems and two engineering applications. The numerical comparisons have demonstrated its effectiveness and efficiency.


Introduction
Traditional optimization methods can be classified into two distinct groups; direct and gradient-based methods. In direct search methods, only objective function and constraint value are used to guide the search, whereas gradient-based methods use the first and/or second-order derivatives of the objective function and/or constraints to guide the search process. Since derivative information is not used, the direct search methods are usually slow, requiring many function evaluations for convergence. For the same reason, they can be applied to many problems without a major change of the algorithm. On the other hand, gradient based methods quickly converge to an optimal solution, but are not efficient in non-differentiable or discontinuous problems. In addition, if there is more than one local optimum in the problem, the result may depend on the selection of an initial point, and the obtained optimal solution may not necessarily be the global optimum. Furthermore, the gradient search may become difficult and unstable when the objective function and constraints have multiple or sharp peaks. Therefore, to overcome these shortcomings, a lot of research has focused on meta-heuristic methods. The meta-heuristic methods have many advantages compared to the traditional nonlinear programming techniques, among which the following three are the most important: (i) They can be applied to problems that consist of discontinuous, nondifferentiable and non-convex objective functions and/or constraints. (ii) They do not require the computation of the gradients of the cost function and the constraints. (iii) They can easily escape from local optima. Many meta-heuristic algorithms such as, genetic algorithm (GA) [1], particle swarm optimization (PSO) [2], differential evolution [3] and ant colony optimization [4] have shown their efficacy in solving computationally intensive problems.
Among the existing meta-heuristic algorithms, a well-known branch is the FA which is a meta-heuristic search algorithm, has been developed by Yang [5,6]. FA mimics some characteristics of tropic firefly swarms and their flashing behavior. A firefly tends to be attracted towards other fireflies with higher flash intensity. The main advantages of FA are namely intensification and diversification or exploitation and exploration: As light intensity decreases with distance, the attraction among fireflies can be local or global, depending on the absorbing coefficient, and thus all local modes as well as global modes will be visited. Therefore, it has captured much attention and has been successfully applied to solve several optimization problems including [7][8][9].
Fruit fly optimization algorithm (FOA) is one of the latest metaheuristic methods which proposed by Pan [10] for solving optimization problems. The main inspiration of FOA is that the fruit fly itself is superior to other species in sensing and perception, especially in osphresis and vision. They feed chiefly on rotten fruits. In the process of finding food, their osphresis organs smell all kinds of scents in the air. They then fly towards the corresponding locations. When they get close to the food locations, they find foods using their visions and then fly towards that direction. FOA has many advantages such as a simple structure, ease of implementation and speed to acquire solutions. Therefore, FOA has been successfully applied to solve several optimization problems [10][11][12][13][14].
Recently, hybridization is recognized to be essential aspect of high performing algorithms. Pure algorithms cannot reach to an optimal solution in a reasonable time. In addition, the premature convergence in pure algorithms is inevitable. Thus pure algorithms are almost always inferior to hybridizations. To our best knowledge, this is the first time that the hybridization between FOA and FA has been proposed for solving NLPPs. This hybridization aims to improve the performance of the original FOA and eliminate its drawbacks.
In this paper, we propose a novel hybrid algorithm named, FOA-FA for solving the nonlinear programming problems and two engineering applications. The proposed FOA-FA algorithm employed the merits of both fruit fly optimization algorithm (FOA) and firefly algorithm (FA) and its methodology consists of two phases. The first one employs a variation on original FOA employing a new adaptive radius mechanism for exploring the whole scope around the fruit flies locations to overcome the drawbacks of original FOA which has been continues for the nonnegative orthant problems. In addition, several other improvements for original FOA are considered, such as: Initialization of the candidate solutions through the search space uniformly and a new modulation coefficient. Moreover, the premature convergence of original FOA degrades its performance by reducing its search capability, leading to a higher probability of being trapped to a local optimum. Therefore, the second phase incorporates the FA to update the previous best locations of fruit flies to force FOA jump out of premature convergence, because of its strong searching ability. Consequently, the hybrid algorithm speeds up the convergence and improves the algorithm's performance.
The rest of the paper is organized as follows. In Section 2 we describe some preliminaries of the nonlinear programming problem. In Section 3, we introduce briefly the basics of both FOA and FA algorithms. The hybrid algorithm named, FOA-FA algorithm is proposed and explained in details in Section 4. The numerical experiments are given in Section 5 to substantiate the superiority of the proposed algorithm. Finally we summarize the paper with some comments.

Nonlinear Programming Problem (NLPP)
The nonlinear programming problem can be defined as follow [15]: Where F(x) is the objective function defined on the search space S, n S ℜ ⊆ , X is called a decision variable and g k (x), h j (x) are defined the inequality and equality constraints, respectively. The sets S ⊆ Ω and Ω − = S φ define the feasible and infeasible search spaces, respectively. Usually, the search space S is defined as an n-dimensional rectangle in n ℜ as in (2) (domains of variables defined by their lower and upper bounds): Any point Ω ∈ x is called a feasible solution, otherwise, X is an infeasible solution.

Fruit fly optimization algorithm (FOA)
In this section we describe the basic steps of the original Fruit fly optimization algorithm (FOA) in details as follows [10].
Step 1: Set the main parameters of FOA and give initial location for the fruit fly swarm randomly P

x-axis, y-axis
Step 2: Endow personal fruit fly with random direction for finding food by using: where m is the swarm size of fruit flies Step 3: Since the food location cannot be known, calculate the distance (Dist i ) of the fruit fly from the origin and find out the smell concentration judgment value (S i ). Suppose that S i is the reciprocal (Dist i ) of as follows: Step 4: Calculate the smell concentration (Smell i ) of the individual fruit fly location by substituting the smell concentration judgment value (S i ) into the smell concentration judgment function (also called Fitness function).
Step 5: Find out the individual fruit fly with the maximal smell concentration (the maximal value of Smell) among the fruit fly swarm: Step 6: Keep the maximal smell concentration value and location (x, y) of the best fruit fly. Then, the swarm flies towards that location.

y-axis = y(best index)
Enter iterative optimization to repeat the implementation of step 2-6. When the smell concentration is not superior to the previous iterative smell concentration any more, or the iterative number reaches the maximal iterative number, the circulation stops.

Firefly algorithm (FA)
The Firefly algorithm was developed by Xin-She [6] and it is based on idealized behavior of the flashing characteristics of fireflies. For simplicity, we can summarize these flashing characteristics as the following three rules: a) All fireflies are unisex, so that one firefly is attracted to other fireflies regardless of their sex. b) Attractiveness is proportional to their brightness, thus for any two flashing fireflies, the less bright one will move towards the brighter one. The attractiveness is proportional to the brightness and they both decrease as their distance increases. If no one is brighter than a particular firefly, it will move randomly.
c) The brightness of a firefly is affected or determined by the landscape of the objective function to be optimized [6].
For simplicity we can assume that the attractiveness of a firefly is determined by its brightness or light intensity which in turn is associated with the encoded objective function. In the simplest case for an optimization problem, the brightness I of a firefly at a particular position X can be chosen as . However the attractiveness is relative, it should vary with the distance r ij between firefly i and firefly j. As light intensity decreases with the distance from its source and light is also absorbed in the media, so we should allow the attractiveness to vary with degree of absorption.

Step 1: Attractiveness and light intensity
In the firefly algorithm, there are two important issues: the variation of the light intensity and the formulation of the attractiveness. We know, the light intensity I(r) varies with distance r monotonically and exponentially, that is: Where I 0 the original light intensity and γ is the light absorption coefficient. As firefly attractiveness is proportional to the light intensity seen by adjacent fireflies, we can now define the attractiveness β of a firefly by Equation (10) ) ( ) ( 1 0 Where r is the distance between each two fireflies and β 0 is their attractiveness at r = 0 i.e., when two fireflies are found at the same point of search space. The value of γ determines the variation of attractiveness with increasing distance from communicated firefly.

Step 2: Distance
The distance between any two fireflies i and j at x i and x j respectively, the Cartesian distance is determined by equation (11)

Step 3: Movement
The firefly i movement is attracted to another more attractive (brighter) firefly j is determined by: Where the second term is due to the attraction while the third term is randomization with α being the randomization parameter and r and is a random number generator uniformly distributed in [0, 1].

The Hybrid FOA-FA Algorithm
In this section, the proposed algorithm will be presented in detail. The main reason for developing the FOA-FA is to overcome the drawbacks of the original FOA which cannot handle the negative domain that is the candidate solution cannot be uniformly generated in the problem domain. Also original FOA is easy to fall into premature convergence because the random term of the Equation (3) picks small values within a radius equal to one around the best location.
The methodology of the proposed FOA-FA algorithm consists of two phases. The first one employs the FOA, where a swarm of fruit flies that moves in various directions using the ARM. Therefore, these moves will follow a uniform distribution. The other phase incorporates the FA to update the previous best locations of fruit flies to force FOA jump out of premature convergence, because of its exploitation and exploration ability. Consequently, the hybrid algorithm speeds up the convergence and improves the algorithm's performance. The main steps of the proposed algorithm as follows:

Step 1: Initialize the parameters
The main parameters of the proposed algorithm are the maximum iterationnumber (T),thewarmsize(m) and distance zone of fruit fly (R).
The main parameters of the proposed algorithm are the maximum iteration number (T),thewarmsize(m) and distance zone of fruit fly (R).

Step 2: Initialize the swarm location
Initial fruit fly swarm location is accompanied within the search space, that is, the algorithm assigns a random vector for i-th fruit fly as in Equation (13): Where rand is a random value from the uniform distribution on the interval [0, 1].

Step 3: Solution construction using osphresis
The osphresis that introduced in the original FOA (see Equation 3) was picked small values within a radius that equals one. So, we introduce the ARM for enhancing the diversity of solutions and achieving an effective exploration to the whole scope around the fruit flies locations, where the search radius decreases dynamically with iteration number as shown in Figure 1 (i.e., R=100, T=100). Therefore, the process of the solution construction using osphresis is implemented as follows: Where R is the search radius that determines the extent of the search subspace which we want to cover during the run and ε is a

Step 4: Evaluation
The smell concentration ( i Smell ) of the individual fruit fly is related to objective function (fitness function). In this step, the fitness  function is evaluated based on the decision variable X as in Equation (15) instead of using the smell concentration judgment value (S i ). Therefore, the best food source with the lowest fitness x best is also found as in Equation (16) .
Step 5: Update the swarm location In this step, the swarm flies toward the best obtained location using Equation (16). If X best is better than the current fruit fly swarm location X i -axis, it will replace the swarm location and become a new one in the next iteration, i.e.,

Step 6: Crossover operator for updating the osphresis
In this step, the crossover operator is adopted for enhancing the osphresis between fruit flies as follows: is random numberand isthe extending-modulation operator is specified by the user. The idea of this step is to make the new generation of fruit flies around the best fruit fly, where the new solution not within x i (t) and x best (t) but the segment may be extended equally on both sides (i.e., 3γ-3 as in Figure 2. Therefore, the best food source with the lowest fitness x best is also found as in Equation (16), i.e., . If x best is better than the current fruit fly swarm location X i -axis , it will replace the swarm location and become a new one in the next iteration, i.e., Step 7: FA phase for updating the diversity The random term of the Equation (3) makes the fruit flies swarm fluctuates around the so far best solution; consequently, the diversity loss occurs. Under this condition, the best solution might be easily trapped in a local optimum or premature convergence. FA is incorporated to update the previous best locations of fruit flies to force FOA jump out of premature convergence, because of its exploitation and exploration ability. Consequently, the hybrid algorithm speeds up the convergence and improves the algorithm's performance. In the rest of this paper, we will discuss the implementation FA in three steps as follows ( Figure 3).
Step 7.1: Initialization: Initialize a swarm of fireflies with the obtained best ant position, where each firefly contains n variables (i.e., the position of the i th firefly in the n dimensional search space can be represented as . Furthermore every member of the swarm is characterized by its light intensity (i.e., initialize each firefly with distinctive light intensity. I i (x i ), i = 1, 2, …m

Step 7.2: Light intensity
Calculate the light intensity I i (x i ), i = 1, 2, …m for each firefly which in turn is associated with the encoded objective function, where for

Step 7.3: Attractiveness and movement
In the firefly algorithm, the attractiveness of a firefly is determined by its light intensity which in turn is associated with the encoded objective function and the distance r ij between firefly i and firefly j. In addition, light intensity decreases with the distance from its source, and light is also absorbed in the media, so we should allow the attractiveness to vary with the degree of absorption. As a firefly's attractiveness is proportional to the light intensity seen by adjacent fireflies, we can now define the attractiveness β of a firefly by Equation (10). On the other hand, the movement of a firefly i is attracted to another more attractive (brighter) firefly J is determined by Equation (13).
Since each member of the swarm explores the problem space taking into account results obtained by others, therefore the randomization term may moves the fire fly to lose its best location, so we introduce a modification on the randomization term that makes the fireflies approached from the optimum. A further improvement on the convergence of the algorithm is to vary the randomization parameter 1 α so that it decreases gradually as the optima are approaching. For example, we can use Where T c is the maximum number of generations for FA, and θ є [0, 1] is the randomness reduction constant.
The basic steps of the proposed FOA-FA algorithm can be summarized as the pseudo code shown in Figure 4.

Results and Discussions
In this section, performance of the proposed FOA-FA algorithm is tested on eleven benchmark problems, which are known from the literature and reported in Young [16] and two engineering applications [17,18]. The solutions of the proposed FOA-FA are compared with the prominent algorithms that reported in Young [16][17][18]. The test problems are listed in Table 1. The algorithm is coded in MATLAB 7, which has an Intel Core I 5 (1.8 GHz) processor and 4 GB RAM memory.       On the other hand, extensive experimental tests were conducted to see the effect of different values on the performance of the proposed algorithm. Based upon these observations, the following parameters have been set as in Table 2.

The performance evaluation metrics
Our hybrid algorithm was tested on set of the test problems in engineering domain. Each of the test problems was run thirty times independently, with different seeds. In order to evaluate the closeness between the obtained optimal solution and the theoretical optimal solution, as well as measuring the number of evaluations for the proposed algorithm and the existing algorithms, we used two different metrics.
The speed: The speed with which the optimum is obtained is a criterion for examining the performance. For the proposed FOA-FA to be competitive, it is important to be able to compare its speed with the other existing algorithms. As time is a relative notion depending on the computer used, we preferred to measure the speed by counting the number of evaluations of the tested problem. On the one hand, this criterion is independent from the type of computer used, and on the other hand the evaluation of the function is the operation taking the longest time in the program.
The error: So the test for closeness, we used the evaluation of the error between the result given by the algorithm and the exact value of the optimum, for each test problem. The relative error was used each time it was possible: When the optimum to reach was 0, it was no longer possible to use this expression so we calculated the absolute error:  The swarm size (m) 100 Initial attractiveness (β0) 1 The light absorption coefficient (γ) 1 The randomness reduction constant (Ѳ) 0.96 Randomization parameter (α) 0.95  from the literature. The test problems have been solved by FOA-FA for 30 times. The starting values of the variables for each problem were selected randomly for all runs from the solution space. The results found by FOA-FA such as the best and average function value, numbers offunction evaluation (NFE) and solution time in seconds have been recorded in Table 3, whereas for the other algorithms only the function value and numbers of function evaluation are given because the solution times, the best and average function value for some algorithms not given. First, we start by comparing the optimum solution of the given the test problems for the proposed FOA-FA algorithm and the existing algorithms. Based on the obtained results that demonstrated in Table 3 we notes that, the proposed FOA-FA is successful finding the optimum solution for the test problems F2 and F4 better than both the theoretical optimum solution and the obtained by other existing algorithms. Also the proposed FOA-FA algorithm outperforms the existing algorithms for functions F 1 , F 6 , F 7 , F 8 , F 9 , F 10 and F 11 but equals the SZGA for the test problems F 6 , F 7 and F 8 . Second, as shown from Table 4, the proposed algorithm speeds up the convergence through inclusion of the lowest evaluations, where the proposed algorithm elapsed number of evaluations less than the other existing algorithms for finding the global optimum as in Figure 4, that is the proposed algorithm is the faster than the existing algorithms.
Finally, the last performance metric is the relative error or the absolute error, where as depicted from Table 4, the absolute error by using the proposed algorithm is better than the other exists algorithms for all test problems except SZGA for the test problem F 4 and also the relative error is superior to the other exists algorithms for all test problems except SZGA for the test problem F 4 The SZGA seems to better than the proposed algorithm for the test problem F 4 this because the error is calculated related to the theoretical optimum. In fact, the proposed algorithm outperforms the theoretical optimum for the test problem F 4 , where if the relative error and the absolute error are calculated related to the obtained optimum by our algorithm we find it 0.12567 E-5 and 0.5 E-6 respectively. Therefore, the simulation results prove superiority of the proposed algorithm to those reported in the literature, where it is completely better than the other algorithm

Design optimization problems
The design optimization problems are a very important research area in engineering studies because real world design problems require the optimization of relevant engineering applications. In terms of robustness and efficiency of the available methods, these methods are still in need of improvements. Therefore, to meet the ever increasing demands in the design of the electrical devices, the proposed FOA-FA has been substantiated its capability for solving these applications. Thus, the validity of the proposed algorithm has been proved superiority by using two engineering applications, the linear synchronous motor (LSM) and air-cored solenoid [17,18].

Design of LSM:
The linear synchronous motor (LSM) operates on the same working principle as that of a permanent magnet rotary D.C. motor [17]. As in a rotary motor there are two parts in a LSM, one is the set of permanent magnets and the other is the armature that has conductors carrying current. The permanent magnets produce a magnetic flux perpendicular to the direction of motion. The flow of current is in the direction perpendicular to both the direction of the motion and the direction of the magnetic flux.
To validate the proposed algorithm, it is employed to optimize the geometrical design of the linear electric actuator problem as is shown in Figure 5(a) and 5(b). The objective function is to maximum force subject to some of constraints on heat, radius, saturation, demagnetization and maximum force constraint. There are four design variables: the current in each slot, x 1 , the dimensions of the slot, and x 2 x 3 , and the height of the magnet, x 4 . The mathematical formulation of the objective function is described as follows: . . .
The obtained result for design of LSM is demonstrated in Table 5, by giving the best, worst and mean values for the objective function. Also Figure 6 depicts the relation between the maximum force and number of iterations for FOA before applying FA and after Applying FA (i.e., the hybrid algorithm FOA-FA). The results of the proposed FOA-FA is superior to those obtained using the Deshpande [17].

Design of an air-cored solenoid:
The optimization problem of a coreless solenoid with rectangular cross section x 1 ×x i and a mean radius x 3 Figure 5(b) is tackled from Barba [18]. This problem can be formulated as: maximize the inductance L(x 1 , x 2 , x 3 ) and satisfy some of constraints for the given length k 1 = 10 m and k 2 = 10 -6 of the current carrying wire. In order to simplify the analysis, two variables, x 1 and x 2 , are considered. Correspondingly, the optimization problem is simplified as:  Table 6 demonstrates the result for design of air-cored solenoid, by giving the best, worst and mean values for the objective function. On the other hand, Figure 7 depicts the relation between the maximum inductance against the number of iterations for FOA before applying FA and after Applying FA (i.e., the hybrid FOA-FA). The result of the proposed FOA-FA is superior to those obtained by [18]. Figure 8 shows the graphical representation of the optimum solution for the LSM and air cored solenoid. It is obvious that the algorithm outperforms the other algorithms for the both problems.
We concluded that the integrated algorithm of FOA with FA has improved the quality of the founded solutions and also it guarantees the faster converge to the optimal solution. The FOA is presented in the    first stage to provide the initial solution to optimization problem (close to the optimal solution as possible) followed by FA to enhance the quality of the solution. However, because of its random behavior, FOA may suffer from slow convergence. So we introduced the neighborhood search via adaptive radius mechanism Δ (t,R) that returns a value in the range [0, R]. On the other hand, to reach a quick and closer   result to optimal solution, and to improve the efficiency of the FOA, hybridization between FOA and FA was implemented.
In this subsection, a comparative study has been carried out to assess the performance of the proposed FOA-FA algorithm concerning the hybridization, closeness to optimal solution and computational time. On the first hand, pure algorithms suffer from reach to an optimal solution in a reasonable time. Also, the sinking into premature convergence is inevitable in pure algorithms. Consequently, our hybridization algorithm has twofold feature, solving NLPPs in a reasonable time and avoiding the sinking into the premature convergence. On the other hand, the proposed FOA-FA algorithm is highly competitive when compared with the other existing methods in term of calculating the relative, absolute error and number of evolutions. Accordingly, it provides the facility to search in the negative and the nonnegative orthant through adaptive radius mechanism. Finally, the feasibility of using the proposed algorithm to handle NLPPs has been empirically approved.

Conclusions
A novel hybrid algorithm named, FOA-FA for solving the nonlinear programming problems and engineering applications is proposed. The proposed FOA-FA algorithm employed the merits of both fruit fly optimization algorithm (FOA) and firefly algorithm (FA) and consists of two phases. The first one employs a variation on original FOA employing a new adaptive radius mechanism for exploring the whole scope around the fruit flies locations to overcome the drawbacks of original FOA, which has been an continues to be a fruitful paradigm for designing effective the nonnegative orthant algorithms. Moreover, the premature convergence of original FOA degrades its performance by reducing its search capability, leading to a higher probability of being trapped to a local optimum. Therefore, the second phase incorporates the FA to update the previous best locations of fruit flies to force FOA jump out of premature convergence, because of its strong searching ability. The hybrid algorithm speeds up the convergence and improves the algorithm's performance.
A careful observation will reveal the following benefits of the proposed optimization algorithm.
1) It can efficiently safeguard the solution from sinking into the premature convergence through incorporating the FA.
2) It can overcome the lack of the exploration of the FOA which roaming in the nonnegative orthant by introducing the adaptive radius mechanism.
3) It emphasizes the diversity of solutions by incorporating the FA that enriches the exploratory capabilities of the proposed algorithm.
4) It competitive when compared with the other existing algorithm.
5) It can find can find the global minimum for the problems very efficiently.
6) It can accelerate the convergence and improves the algorithm's performance through elapsed low computational time.
The future work will be focused on three directions: (i) the application of FOA-FA to real world problems; (ii) the extension of the method to solve the multi-objective problems; and (iii) another applications in engineering.