Design Approach for Evolutionary Techniques Using Genetic Algorithms: Application for a Biped Robot to Learn to Walk and Rise after a Fall

Abstract

In this research, a biped robot is programmed to learn to walk using genetic algorithms. The biped robot has the ability to rise after a fall. The system is made according to human genetics, where a child tries to learn to walk, takes a few steps, falls, and then stands up somehow after a fall. After this process, the system restarts and tries to walk again. The biped robot evolves after each generation, and after each generation it will have a more natural walking posture. The system is unpredictable, because the biped robot sometimes evolves faster and sometimes evolves slower, like in human genetics where some children are faster and other children are slower to walk. The biped robot is considered to have evolved enough after having a walking posture similar to that of a human and does not fall.

1. Introduction

Biped robots have been the subject of broad exploration. [1] noticed that huge headway has been made toward the advancement of robots that can walk, run, and perform other trained assignments. According to [2], the construction of a stable walking stride for biped robots remains a significant challenge. The complexity and nonlinearity of the system make it difficult to find a solution using conventional approaches [3]. Numerous systems, including robots, have seen improvements brought about by genetic algorithms [4]. In this paper, a method to improve the gait of a biped robot is described using evolutionary algorithms [5].

According to [6], robotic bipedal movement is a challenging endeavor that requires precise control and adaptation to maintain mobility and stability. According to [7], durable and nimble biped robots must learn to stand up after falling. Genetic algorithms (GAs) provide a potential solution to these issues by optimizing control systems according to ideas of natural selection [8]. According to [9], the purpose of this research is to investigate the application of genetic algorithms to the locomotion of biped robots with a focus on the learning process of walking and recovering from falls.

According to [10], for robots to interact with objects, navigate environments dominated by humans, and perform a variety of tasks, they must be able to move on two legs. By mimicking human stride, biped robots can achieve greater mobility and adaptability, making them suitable for a variety of applications, such as humanoid companionship, search and rescue, and prosthetics [11]. However, establishing stable and robust bipedal walking remains challenging due to the complexity of the underlying dynamics and control processes [12].

For biped robots to operate and be durable in the long term, they must be able to recover independently from falls [13]. Falls may be caused by unexpected difficulties, uneven ground, or mechanical failures [14]. Biped robots can regain stability and resume their locomotion activities without the need for human intervention by creating efficient recovery mechanisms [15]. Robotic adaptation and effective responses to unexpected events are made possible by genetic algorithms, which offer a promising approach [16,17].

The concepts of natural selection and evolution served as the basis for the development of genetic algorithms, which are optimization approaches [18]. These are carried out by iteratively improving a population of potential mates through the processes of reproduction, crossover, and mutation [19,20,21,22]. Genetic algorithms can be used to evolve control parameters that regulate walking patterns in the context of biped robots learning to walk [23,24,25,26]. Genetic algorithms can efficiently explore the search space and converge to the best control strategies for bipedal locomotion by representing potential solutions as individuals in a population, evaluating their performance according to predetermined fitness criteria, and using genetic algorithms to create new generations [27].

A crucial component of bipedal locomotion, in addition to learning to walk, is the ability to recover from falls [28]. By developing control strategies that enable the robot to detect falls, analyze the fall scenario, and perform the appropriate actions to regain stability and stand up, genetic algorithms can be extended to improve recovery behaviors [29]. The genetic algorithm can iteratively refine recovery methods and the robot’s ability to adjust to various falls and execute the recovery cycle for greater results [30].

The purpose of this scientific investigation is to investigate how genetic algorithms are used in the learning process of biped robot locomotion and recovery from falls. The particular goals of this study include:

(a)

The development of a system that precisely models the elements of bipedal movement and fall situations;

(b)

The establishment of appropriate fitness criteria for evaluating the walking and recovery performance of biped robots;

(c)

The utilization of a framework that is based on genetic algorithms to optimize control parameters or policies for stable walking and effective fall recovery;

(d)

The evaluation of control strategies developed through extensive simulations and experimental validation in terms of their robustness and effectiveness.

By utilizing the power of genetic algorithms to improve the learning process of walking and the recovery capabilities of biped robots, the findings of this study have the potential to advance the field of bipedal robotics. The resulting biped robots can exhibit improved stability, adaptability, and resilience in complex and dynamic environments by optimizing control strategies with genetic algorithms.

2. Literature Review

L. Yang et al. used an optimized Fourier series and a genetic algorithm to create a two-legged robot walking pattern [31].

A biped robot’s walking pattern was created using a genetic algorithm and optimized Fourier functions.

Nonetheless, it is essential to note that the paper focuses exclusively on simulations and tests on a real robot are not included. Furthermore, the article does not address important considerations when developing a comprehensive walking strategy for a biped robot, such as falling robots and the recovery process after a fall.

Miao Kehua et al. developed gennet [32], a flexible control framework for a biped robot made up of genetic algorithms and neural networks.

A noteworthy strategy that incorporates genetic algorithms and neural networks is presented.

However, it is essential to note that the paper focuses primarily on theoretical aspects and does not include any substantial simulation studies or evidence of actual robot testing. In addition, crucial considerations for designing and improving algorithms for biped robot walking, such as fall management and the robot’s ability to recover from falls, are not addressed in the paper.

Runming Zhang et al. presented a paper focusing on energy proficiency and speed in bipedal walking using a genetic algorithm [33].

The use of a genetic algorithm is emphasized to improve the speed and energy efficiency of bipedal locomotion.

Nevertheless, it is important to note that the paper only presents simulations and does not provide experimental validation on a real robot. Additionally, the article does not address pivotal angles, for example, fall management and recuperation, which are imperative areas of consideration to improve and enhance the walking system of bipedal robots.

Sang-ho Choi et al. presented a paper investigating the generation of ideal trajectories for a biped robot using genetic algorithms [34].

This paper evaluates the generated trajectory using both simulations and actual robot tests.

However, it is essential to note that the paper does not address the robot’s specific walking technique; instead, its sole focus is on creating the walking trajectory. Furthermore, the article does not discuss important aspects of the overall walking process of a bipedal robot, such as the management of falls and the recovery process after falls.

Table 1. Comparison of the parameters of the genetic algorithm of the proposed approach with other similar methods.

GA Parameter	Proposed Approach	L. Yang et al. [31]	Miao Kehua et al. [32]	Runming Zhang et al. [33]	Sang-Ho Choi et al. [34]
DOF	6	6	21	8	8
Height [cm]	24	51	-	54.6	0.685
Weight [kg]	0.833	24	-	2.3	47
Speed [m/s]	0.015	0.37 or 0.45	3.73 or 5.1	0.23 (top speed)	-
Real robot	yes	no	no	no	yes
Fall management	yes	no	no	no	no
Cost	low	low	high	low	high
Complexity	low	low	high	low	low
Resource utilization	low	low	high	high	high

presents a comparison between the biped robot utilized in this paper and other biped robots discussed in scientific journals or conference papers.

It is evident that the biped robot employed in this study is similar to that used by other researchers. However, a notable distinction lies in the absence of an upper body in the robot used in this paper, since it relies solely on its legs for walking. This absence is justified by the notion that an upper body is unnecessary for a biped robot’s process of learning to walk.

It can be seen that, in the scientific literature, there are not too many approaches for biped robots learning to walk that involve fall management. For the present paper, this is the main focus. Due to the fact that a real system tries to copy the human genetics, fall management is very important because, without this, the system will only be partially automated. A child learning to walk will rise somehow after a fall. This is the kind of system that is intended to be implemented for this biped, which learns to walk using genetic algorithms.

3. Problem Solving

3.1. Theoretical Background

This is a complex system that includes the robot’s mechanical construction, control system, and actuation system and controls the walking motion of a biped robot. The motors and other parts of the actuation system produce the power and force expected to move the robot legs. The control system controls the robot’s mobility and ensures that it stays stable while moving. The actual construction of the robot moves its center of gravity, weight distribution, and other essential components that affect how it walks.

Biped robot walking gaits have been created using different methods. Rule-based optimization and machine learning are the main types. Rule-based approaches include the creation of a set of rules to direct the robot’s movement. Using numerical improvement methods, advancement strategies include the quest for the most ideal response. To improve robot learning and performance, machine learning techniques are used to train it with data and algorithms.

Finding the best answers to challenging problems using genetic algorithms has been shown to be quite promising. A class of optimization algorithms known as “genetic algorithms” is based on the concepts of genetics and natural selection. The algorithm generates a population of potential solutions, assesses their fitness, and chooses the best for the next generation. Up until the ideal answer is discovered, the process is repeated.

To teach the biped robot to walk, various parameters and formulas are used within the genetic algorithm framework.

Table 2. Parameters of the genetic algorithm.

Total population	150	Generation number	50
Crossover rate	0.75	Mutation rate	0.01
String length	8bits

presents the specific parameters used in the genetic algorithm, which were determined based on the experimental results. A population size of 150 was established from the experimental results and each individual represents a command string that controls the servomotors of the biped robot. The objective is to enable the robot’s movements to resemble natural sun tracking as closely as possible.

Typically, after 50 generations, the command string obtained from the biped robot has the ability to effectively control the servomotors, ensuring natural movement. However, it is not always necessary to reach the 50th generation for optimal performance.

The crossover rate, indicating the likelihood of the exchange of genetic material between individuals, can be calculated by dividing the number of generations by the total number of generations used in the tests. In this case, dividing 50 by 67 results in a crossover rate of 0.75.

The mutation rate refers to the probability that the gene (or character) mutates or changes within the command string of the biped robot. In this example, a mutation rate of 0.01 implies a 1% chance that a gene will mutate or a character in the command string will change.

Given that the length of the command string is eight bits, each gene (represented by a character) in the string consists of eight bits. The characters in the command string have ASCII values (American Standard Code for Information Interchange) represented by two hexadecimal digits, each hexadecimal digit being equivalent to four bits or four binary digits.

After completing 50 generations, the genetic algorithm demonstrated a significant improvement in the precision of the biped robot, enabling it to walk more naturally. However, before reaching generation 50, the system exhibited lower precision. This conclusion was drawn on the basis of multiple test runs.

To calculate the crossover rate or parameter $C_p$, Equation (1) describes the method used. We divided the generation number $G_n$ by the total generation number $G_T$, which resulted in a value of 0.75 for our particular case.

$$C_p=\frac{G_n}{G_T}=\frac{50}{67}=0.75$$

(1)

The process of generating genes (characters) and assembling them into gene arrays or command strings for the robot can be observed in Equation (2). This can be achieved by assigning ASCII values to randomly selected numbers within the specified MIN and MAX ranges. The range encompasses numbers from 0 to 9 as well as the colon (“:”) and semicolon (“;”) characters, which have ASCII values ranging from 48 to 59. An example of a generated gene array would appear as “7;20;20”, representing the angle of movement of each motor joint of one leg of the biped.

$$g_i=ASCII\left(ran{d}_{MIN,MAX}\right),MIN=48,MAX=59$$

(2)

Equation (3) illustrates the process of determining a random midpoint, denoted as $P_m$, for crossing two arrays of parent genes. Using this midpoint, a portion of the gene array is inherited from one parent, while another portion is inherited from the other parent, resulting in the creation of a child array. The distribution of genes from each parent can vary, as the midpoint is not always symmetric, but instead is a random point. When both arrays of parent genes have the same lengths, a random number is generated in the range of 0 to the length n of the arrays.

$$P_m=ran{d}_{0,n}$$

(3)

Similar to the principles of biological genetics, computational genetics also involves the process of mutation, as it is necessary to ensure the generation of diverse offspring. Without mutation, there would be the possibility that a child could be identical to its parent (when the random midpoint $P_m$ is at zero), which would hinder evolution. Consequently, the biped robot would never achieve the most natural movement. In computational genetics, as in biological genetics, a child can have both mutated genes and genes inherited from its parents. This uniqueness is what makes each individual on Earth distinct, even if they share some genes. Mutation plays a crucial role in the process of evolution.

Equation (4) describes the procedure of generating a random mutation. A random number is generated between 0 and 1, and if this number is less than the mutation rate or the parameter $M_p$ (which in our case is 0.01), a mutation occurs. This involves creating a new gene, known as $g_i$, with an ASCII value that falls within the limits of MIN and MAX (48 and 59, respectively). The gene or character in the current loop index is replaced only if the randomly generated number, within the range of 0 and 1, is less than the mutation rate or parameter $M_p$, which is set to 0.01.

$$\left\{\begin{array}{c}\hfill ran{d}_{0,1}<M_p,M_p=0.01\hfill \\ \hfill g_i=ASCII\left(ran{d}_{MIN,MAX}\right),MIN=48,MAX=59\hfill \end{array}\right.$$

(4)

The fitness function used in the genetic algorithm is presented in Equation (5):

$$F\left(w\right)=\frac{1}{T}{\int }_0^T(K_p\Delta x^2+K_d\Delta {\dot{x}}^2)dt$$

(5)

where $K_p$ and $K_d$ are proportional and derivative gains, respectively; $\Delta x$ is the deviation from the desired position; $\Delta \dot{x}$ is the deviation from the desired velocity; and T is the duration of a single gait cycle.

The mutation operator used in the genetic algorithm is presented in Equation (6):

$$w_{i,j}=w_{i,j}+\delta w_{i,j}$$

(6)

where $w_{i,j}$ is the j-th element of the i-th weight matrix, and $\delta w_{i,j}$ is a random perturbation.

The velocity update equation used in particle swarm optimization is presented in Equation (7):

$$v_{i,j}=w{v}_{i,j}+c_1{r}_1(p_{i,j}-x_{i,j})+c_2{r}_2(g_j-x_{i,j})$$

(7)

where $v_{i,j}$ is the velocity of the i-th particle in the j-th dimension, w is the inertia weight, $c_1$ and $c_2$ are the cognitive and social parameters, respectively, $r_1$ and $r_2$ are random numbers between 0 and 1, $p_{i,j}$ is the best position achieved by the i-th particle in the j-th dimension, $g_j$ is the best position achieved by any particle in the j-th dimension, and $x_{i,j}$ is the current position of the i-th particle in the j-th dimension.

To generate the gait of the biped robot, a Fourier series can be used in the genetic algorithm, which is presented on Equation (8):

$$f\left(t\right)=\frac{1}{2}{a}_0+\sum _{i=1}^{\infty }{a}_i+sin\left(\frac{2\pi i}{T}t\right)+\sum _{i=1}^{\infty }{b}_i+cos\left(\frac{2\pi i}{T}t\right)$$

(8)

The Truncated Fourier Series (TFS) is shown in Equation (9):

$$f\left(t\right)=\sum _{i=1}^n{a}_i+sini\omega t+c_f,\omega =\frac{\pi }{T_S}$$

(9)

where $a_i$, n, and $c_f$ are constants, and $\omega$ is the fundamental frequency. $T_S$ is the period, which is half of T.

The TFS for the hip joints is computed as shown in Equation (10):

$${\theta }_{rh},{\theta }_{lh}=\left\{\begin{array}{c}\hfill {\theta }_h^{+}={\displaystyle \sum _{i=1}^n}R·A_{i}sini{\omega }_h(t-t_h^{+})+c_h\\ \hfill {\theta }_h^{-}={\displaystyle \sum _{i=1}^n}R·B_{i}sini{\omega }_h(t-t_h^{-})+c_h\end{array}\right.$$

(10)

where ${\theta }_{rh}$ and ${\theta }_{lh}$ are the angles of the right and left hips, ${\omega }_h=\pi /(t_3-t_0)=\pi /(t_6-t_3)$, is the frequency of the walk stride, $A_i$ and $B_i$ are constants, ${\theta }_h^{+}$ and ${\theta }_h^{-}$ are the upper and lower parts of the hip joint trajectory, $t_h^{+}$ and $t_h^{-}$ are time-shift values, and R is an amplitude scaling parameter used to change the length of the step, which was initially set at 1.

Similarly the knee joints can be expressed as shown in Equation (11):

$${\theta }_{rk},{\theta }_{lk}=\left\{\begin{array}{c}\hfill {\theta }_{k1}={\displaystyle \sum _{i=1}^n}R·C_{i}sini{\omega }_k(t-t_k)+c_k\hfill \\ \hfill {\theta }_{k2}=c_k\ge 0\hfill \end{array}\right.$$

(11)

where ${\theta }_{rk}$ and ${\theta }_{lk}$ are the angles of the right and left knees, ${\omega }_k=\pi /(t_6-t_2)=\pi /(t_1-t_0)$, $C_i$ is a time constant, and $t_k$ is the time-shift.

The force vector can be expressed as shown in Equation (12):

$$Q=M\left(\theta \right)\ddot{\theta }+C(\theta ,\dot{\theta })\dot{\theta }+F\left(\dot{\theta }\right)+G\left(\theta \right)$$

(12)

where $\theta$ is the coordinate vector of the joints, $\dot{\theta }$ is the speed vector of the joints, $\ddot{\theta }$ is the acceleration vector of the joints, M is the inertia matrix, C is the Coriolis force, F is the friction, G is the gravity, and Q is the force vector.

A new fitness function was computed, where we produced 100 random chromosomes, as shown in Equation (13):

$$f=\prod _1^{40}A\left({\varphi }_{t+1}\right)\left(\sum _1^{40}\left(D({\varphi }_t,{\varphi }_t^{\prime })·\left(1-\frac{N_p}{50}\right)·Z\left(S({\varphi }_{t+6},M)\right)\right)\right)$$

(13)

where $S({\varphi }_{t+6},M)$ is the center of gravity.

$Z\left(S\right(*\left)\right)$ shows if S(*) is still inside the ZMP (zero moment point) after six cycles. $N_p$ represents the number of neurons of chromosome p, as shown in Equation (14):

$$Z\left(S(*)\right)=\left\{\begin{array}{c}\hfill 0S({\varphi }_{t+i},M)\in ZMP\\ \hfill 1S({\varphi }_{t+i},M)\in ZMP\end{array}\right.(i=6)$$

(14)

where $A\left({\varphi }_t^{\prime }\right)$ is the output angle, where 1 is positive and 0 is negative.

The method of selecting individuals with small changes to the angles of joints is presented on Equation (15)

$$D=({\varphi }_t,{\varphi }_t^{\prime })=1/\sqrt{{\displaystyle \sum _1^8}{({\varphi }_t-{\varphi }_t^{\prime })}^2}$$

(15)

The formulas for the pitch angle of the COM (center of mass) and the lateral movement of the waist during the SSP (single supporting phase) are presented in Equations (16) and (17):

$$\ddot{\theta }{l}_{xz}-gsin\theta =0$$

(16)

$$\ddot{y}{l}_{xz}cos\theta =(g-\ddot{\theta }{l}_{xz}sin\theta -l_{xz}{\dot{\theta }}^{2}cos\theta )(y-y_p)$$

(17)

The pattern generation extended from the DSP (double supporting phase) trajectory is presented next.

The derivative of the time function in the $x,y,z$ directions is 0, and the minimum order of therms is 2, as shown in Equations (18)–(20):

$$x=c_{x1}{t}^5+c_{x2}{t}^4+c_{x3}{t}^3+x_{x4}{t}^2$$

(18)

$$y=c_{y1}{t}^5+c_{y2}{t}^4+c_{y3}{t}^3+x_{y4}{t}^2$$

(19)

$$z=c_{z1}{t}^5+c_{z2}{t}^4+c_{z3}{t}^3+x_{z4}{t}^2$$

(20)

The displacement of the COM can be expressed as shown in Equations (21)–(23):

$$x_n=\sum _i^{}{c}_x,6-i{\left(ndt\right)}^i,n<N_{dou}$$

(21)

$$y_n=\sum _i^{}{c}_y,6-i{\left(ndt\right)}^i,n<N_{dou}$$

(22)

$$z_n=\sum _i^{}{c}_z,6-i{\left(ndt\right)}^i,n<N_{dou}$$

(23)

where $dou$ represents the length of the DSP, and N is the length of one cycle.

Equations (24)–(28) show how to continue the calculation:

$$x_n=l_{xz}sin{\theta }_n+x_p$$

(24)

$$z_n=l_{xz}cos{\theta }_n$$

(25)

$$\ddot{y}=\frac{(g-\ddot{\theta }{l}_{xz}sin{\theta }_n-l_{xz}{\dot{\theta }}_n^2)(y-y_p)}{cos{\theta }_n},N_{dou}<n<N-N_{dou}$$

(26)

$${\dot{y}}_n={\dot{y}}_l+dt\sum _i^n{\ddot{y}}_i$$

(27)

$$y_n=y_l+dt\sum _i^n{\dot{y}}_i$$

(28)

$\theta$ is computed as shown in Equations (29)–(31):

$${\ddot{\theta }}_n=gsin{\theta }_n/l_{xz}$$

(29)

$${\dot{\theta }}_n={\dot{\theta }}_l+dt\sum _i^n{\ddot{\theta }}_i$$

(30)

$${\theta }_n={\theta }_l+dt\sum _i^n{\dot{\theta }}_i$$

(31)

For a robot with six AC servos, the balancing equations can be written using the D’Alembert principle as shown in Equation (32):

$$\begin{array}{c}\hfill {\ddot{x}}_0-\frac{{\ddot{z}}_0+g_z}{z_0}{x}_0=-\frac{({\ddot{z}}_0+g_z)x_{zmpD}}{z_0}-g_x+\hfill \\ \hfill +\left({\displaystyle \sum _{i=1}^6}{m}_i({\ddot{x}}_i+g_x)z_i-{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)(x_i-x_{zmpD})\right)/m_0{z}_0\hfill \\ \hfill {\ddot{y}}_0-\frac{{\ddot{z}}_0+g_z}{z_0}{y}_0=-\frac{({\ddot{z}}_0+g_z)y_{zmpD}}{z_0}-g_y+\hfill \\ \hfill +\left({\displaystyle \sum _{i=1}^6}{m}_i({\ddot{y}}_i+g_y)z_i-{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)(y_i-y_{zmpD})\right)/m_0{z}_0\hfill \end{array}$$

(32)

where $m_0$ is the balancing mass, $x_0$ and $y_0$ are the sagittal and lateral motions of the balancing mass, $x_{zmpD}$ and $y_{zmpD}$ are the desired ZMPs (zero movement points), and $x_i$, $y_i$, and $z_i$ are the positions of the mass $m_i$ in world coordinates.

Balancing motion compensates the movements of the leg and body, but also causes spin movement, as presented by Equation (33):

$$M_{Tz}=\sum _{i=0}^6{m}_i({\ddot{x}}_i+g_x)(y_i-y_{zmp})-\sum _{i=0}^6{m}_i({\ddot{y}}_i+g_y)(x_i-x_{zmp})+m_g{R}^2{\ddot{\theta }}_z$$

(33)

where R is the distance from the center to the mass center of a balancing roll, and ${\ddot{\theta }}_z$ is angular acceleration in the z direction.

ZMP verification is presented in Equation (34):

$$\begin{array}{c}\hfill x_{zmp}={\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)x_i-{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{x}}_i+g_x)z_i/{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)\\ \hfill y_{zmp}={\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)y_i-{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{y}}_i+g_y)z_i/{\displaystyle \sum _{i=1}^6}{m}_i({\ddot{z}}_i+g_z)\end{array}$$

(34)

3.2. The Proposed Genetic Algorithm

We developed a bipedal walking robot with two legs and six degrees of freedom on each leg. The robot was modeled using the ROS (Robot Operating System) framework. A genetic algorithm was used to optimize the robot’s gait. The genetic algorithm was implemented using the DEAP (Distributed Evolutionary Algorithms in Python) framework.

The genetic algorithm consisted of the following steps:

Initialization:

The population of candidate solutions was generated randomly.

Evaluation:

The suitability of each candidate solution was evaluated using a simulated environment. The simulation environment included a physics engine that simulated robot movements and fitness features that evaluated robot performance.

Selection:

The best solution was selected based on fitness.

Reproduction:

Selected solutions were used to create candidates for the next generation of solutions through crossover and mutation.

Termination:

The process was repeated until the exit criteria were met.

A simplified diagram of the genetic algorithm can be seen in Figure 1. This diagram can be used by a humanoid robot to improve its walking abilities and regain balance after a fall.

The genetic algorithm employs certain terminologies borrowed from biological genetics, which can be interpreted slightly differently when translated into software, but still retain their original meaning.

In the genetic algorithm, DNA or gene arrays refer to command strings or character arrays that are sent to each motor joint of the biped robot to produce movement. For example, a gene may appear as “7;20;20”, which represents the angle of movement for each motor joint of one leg of the biped, for example, 7${}^{\circ }$ at the ankle, 20${}^{\circ }$ at the knee, and 20${}^{\circ }$ at the hip, and the values for the other leg are symmetric to maintain balance.

Gene arrays are constructed as character arrays, where a gene represents a character, for example, “7” denotes the movement angle for a specific joint. The phenotype in this case is 7, while the genotype is the ASCII value of “7”, which is 55. During gene generation, random ASCII numbers are generated and then converted to the corresponding characters to obtain the phenotype or command string from the biped robot motor.

All DNAs or gene arrays are treated as parents and placed in a mating pool to produce an offspring or a child by crossing. The crossing process involves concatenating one half of the command string of a parent with the second half of the other parent’s command string.

Mutation, which involves changing a random gene, is necessary for genetic diversity, since there are no organisms with identical genetic information. In programming, mutation involves changing a character in a random location within the gene array by generating a random ASCII value and converting it to the corresponding character according to the ASCII table. The mutated character is then inserted into a random location in the parent gene array.

The essential variables in the genetic algorithm include the mutation rate (e.g., 0.01, the probability that genes change randomly), total population (e.g., 150, the number of parent command strings to cross), target (the desired command string of the biped robot for natural walking), population (generated biped robot command character arrays) and match (computed fitness score arrays).

The first step is population initialization, which involves generating biped command character arrays (genes).

The second step is selection, where a fitness score is calculated to determine the closest match to the desired command string of the biped robot. Then, a matching pool consisting of robot command strings with higher fitness scores is created. The higher the fitness score, the more times the robot command string is added to the mating pool to increase the chances of crossing and producing an offspring.

The third step involves reproduction, which is achieved by crossing two randomly selected robot command strings from the mating pool. To cross, half of one string is combined with the other half of the other string. If the resulting string is too similar to the parent strings and does not produce the desired result, a mutation occurs, randomly changing one character in the string. This process is similar to genetic inheritance in biological organisms. The new child becomes part of the mating pool and can be crossed with another parent to create a new offspring. This process continues until the target robot command string is achieved. Only one motor is used to create the target string, and the other joints are calculated on the basis of this value. The robot’s position is constantly measured and, if it falls, it attempts to stand up and continue walking. The rise algorithm is based on the values of $0Z$ and $0X$ of the accelerometer, which indicate whether the robot has fallen forward, backward, or to the side.

If the $0Z$ value is less than a specific limit multiplied by −1, the biped robot has fallen on its back and will perform the algorithm to stand up from that position.

If the value of $0Z$ is greater than another specific limit, the biped robot has fallen on its front and will execute the algorithm to stand up from that position.

If the value of $0X$ is less than a certain limit multiplied by −1, the biped robot has fallen to the left and needs to use a combination algorithm to rise. This algorithm involves kicking with the left leg to turn the robot to the front and then using the “getting up from front position” algorithm.

If the value of $0X$ is greater than a certain limit, the biped robot has fallen to the right and will use the same combination algorithm but with the right leg instead.

These combination algorithms were created because the biped robot is unable to make complex side movements, as a human would when falling to the side.

Even when the robot does not fall, it still continuously generates new generations of the biped robot command string to improve its walking movement, stopping only when it reaches the target biped robot command string for the most natural walking movement.

3.3. Implementation

A genetic algorithm was able to find the optimal gait for a bipedal robot. The optimized gait was faster and more stable than the initial gait. The fitness function used in the simulation environment included several parameters, such as walking speed, robot stability, and robot energy consumption.

Figure 2 shows the hardware configuration of the biped robot used to implement a genetic algorithm to learn or improve robot walking skills and stand up after falling.

The biped robot used was called BRAT from Lynxmotion, which only has legs without an upper body, arms, or head, which is sufficient for the biped to learn to walk. It has six degrees of freedom (DOFs) and six motors, two at the hips, two at the knees, and two at the ankles.

The control system for the robot runs on a Raspberry Pi 3 Model B, not the B+ version. This particular version was chosen because it has Wi-Fi onboard and consumes less power than the B+ version. The B+ version has a power consumption level of 350 mA (1.9 W) in idle mode, while Model B has a power consumption level of 260 mA (1.4 W). When under high CPU load (400%), the B+ version consumes 980 mA (5.1 W), and Model B consumes 730 mA (3.7 W). Since the robot needs to be autonomous, it has to carry onboard batteries which must be as small and lightweight as possible while still being able to power all of the robot’s electronics.

The Raspberry Pi 4 Model B was out of the question, because it consumes 540 mA (2.7 W) in idle mode and 1280 mA (6.4 W) under 400% CPU load.

The Lynxmotion SSC-32 servo controller board is powered by a classic 9 V battery, as 5 V is not sufficient to power its electronics.

The final setup consists of three circuit boards, the Raspberry Pi 3 Model B, the Lynxmotion SSC-32 servo controller board and the MPU 6050 accelerometer, powered by two batteries, a 5 V, 2600 mAh power bank and a classic 9 V battery.

The Raspberry Pi 3 Model B is the main control unit that runs the genetic algorithm to generate numbers that are converted into motor command strings and sent to the Lynxmotion SSC-32 servo control board via the RS-232 serial interface. The SSC-32 serial control interface decodes motor commands and sends PWM signals to the six connected Hitec HS-645 servo motors. The biped has the ability to rise after a fall if the genetic algorithm does not generate the best motor command.

The Hitec HS-645 servomotors have gears and electronic circuits mounted on them and are powered by a 5 V power supply with three connection wires: one red wire $V_{CC}$ for power, one black wire for grounding, and one yellow wire for PWM input. The servomotors can only move 180${}^{\circ }$ due to mechanical restrictions.

The biped has the MPU 6050 accelerometer mounted on top to detect if it has fallen in one of four positions: front, back, left, or right. To simplify the process, only the accelerometer values on the $0Z$ and $0X$ axes need to be checked. The accelerometer data are sent to the Raspberry Pi 3 Model B through the I2C serial port, which triggers the “getting up from fallen position” algorithm. The algorithm sends motor position data to the Lynxmotion SSC-32 servo control board via the RS-232 serial interface. The servo board controls the Hitec HS-645 servo motors through the PWM. To rise from a front or back fall, the accelerometer data on the $0Z$ axis are checked, and an algorithm is used to raise the biped from these positions. To stand up from a left or right fall, the accelerometer data are checked on the $0X$ axis. It was found that if the biped kicks with the same leg on the side that it has fallen (that is, the left leg if it falls to the left), it will switch as if it had fallen to the front and then it can get up normally.

4. Results

4.1. Experimental Results

Figure 3 illustrates how the biped robot attempts to walk, falls sometimes, and then manages to get back up again. If the robot falls forward, it tries to stand up by moving forward. If it falls backward, it tries to stand up by moving backwards. If it falls to one side, it attempts to use the leg on that side to kick and rotate towards the front, so that it falls forward and can then try to stand up in the same way as when it falls forward.

Ultimately, the biped robot learns to walk naturally.

4.2. Discussion

The idea of a biped learning to walk with genetic algorithms is not new. The novelty of this paper is the implementation method and the possibility of the biped standing up after a fall. The idea is that a robot should learn to walk in the same way as a child who, after a fall, will stand up and try to walk again. With genetic algorithms, software engineers try to copy biological genetics as much as possible, because it is known that evolution created living beings to be in this actual state that they are. Evolution will generate more and more advanced living beings and, by creating software with genetic algorithms, more advanced machines will be produced.

The big advantage of this work is that the robot can learn to walk and stand up after a fall, and this was tested on a real Lynxmotion BRAT biped robot. Other implementations in this field have presented robots that learn to walk using genetic algorithms but do not stand up after a fall due to the fact that a fall can destroy a real robot. Many implementations have involved simulations only when testing the falling feature of biped robots to protect the robotic assembly. Other implementations have evolved the genetic algorithm by simulations only. This prevents a real biped robot from falling. Other scientists prevent biped robots from falling using harnesses, in this way tying the biped robots to the ceiling or to especially created frames. Many simple implementations do not manage falls. When a robot falls, it is put back on its feet by the scientist, and the process is continued after the biped robot is back on its feet.

The hardest part of choosing components for an autonomous biped robot is the choice of battery, which must be powerful enough to power the entire system, electronics, and servos, but should be light enough to be carried by the biped robot.

The most significant achievement highlighted in this article is the inclusion of fall management in the biped robot featured in this thesis, allowing it to recover from a fall. This capability is essential for a robot to learn to walk, as even young children experience falls during their own walking development. The success of children in learning to walk is highly dependent on their ability to stand up after falling. Therefore, this natural approach should be incorporated into biped robots that are attempting to learn to walk using genetic algorithms.

The significance of the paper is that there are not too many works in the scientific literature on biped robots that can learn to walk and can stand up after a fall.

The optimization method is shown in Figure 1, and this presents the genetic algorithm. Basically, after each generation, the walking of the biped robot is optimized in order to move to a more advanced level.

Without the implementation of the ability to recover from falls, the automation of walking learning cannot be fully achieved. In such cases, the biped robot will require a support structure or human intervention to stand up, resulting in partial automation.

5. Conclusions

We presented a biped robot who can learn to walk and also stand up after a fall. The main accomplishment of this work is that the biped robot can learn to walk using genetic algorithms and can stand up after a fall. Everything was tested on a real Lynxmotion BRAT biped robot.

The main innovation part of this work is that not too many articles in the scientific literature present biped robots that can learn to walk and stand up after a fall. Because the system uses genetic algorithms, it behaves like a real child learning to walk and getting up in some way after a fall.

To determine the position of the biped robot and to determine whether the biped robot is on its feet or if it has fallen, an MPU 6050 accelerometer was used. To control the whole system and run the genetic algorithm, a Raspberry Pi Model B was used. Robotic motors are controlled by the Lynxmotion SSC-32 servo controller. The genetic algorithm is implemented in Python scripting language.

Further enhancements could be made by implementing the system in a more advanced biped robot or by using a larger biped robot. It might be interesting to implement the algorithm on a full humanoid robot, such as Lynxmotion Pete, or even on a bigger humanoid robot.