Design and Control Strategy of Soft Robot Based on Gas–Liquid Phase Transition Actuator

Abstract

In this paper, a soft robot driven by a gas–liquid phase transition actuator with a new structure is designed; The soft robot is driven by the pressure generated by electrically induced ethanol phase transition. The gas–liquid phase transition drive was found to be able to generate a larger driving force by using only low voltage. Compared with the gas drive of a traditional soft robot, gas–liquid phase transition-driven soft robot does not require a complex circuit system and a huge external energy supply air pump, making its overall structure more compact. At the same time, because of the new structure of the actuator on the soft robot, the soft robot has good gas tightness and less recovery time. A reinforcement depth learning control strategy is also added so that the soft robot with this actuator could better grip objects of different sizes and weights.

1. Introduction

When gripping fragile and rotten objects or those objects with complex and irregular shapes, a traditional rigid gripper often has difficulty achieving a good, compliant grip unless precision sensors are added to improve its interaction ability. However, using precision sensors will inevitably increase the total cost. Soft grippers are usually made of hyperelastic materials, which have good flexibility and environmental adaptability; therefore, compliance grasping can be better realized [1,2]. Soft grippers with good flexibility and environmental adaptability have broad application prospects in the field of industrial automation and automatic robot equipment [3].

In the field of soft robots, driving mode has always been a research hotspot. So far, many kinds of soft grippers with different driving modes have been studied. Seyed mehdirakhtala and co-workers proposed a new soft gripper [4], which was equipped with two sensors and used a pneumatic drive. The soft grippers introduced in [5,6] also use a pneumatic drive. The use of a pneumatic drive means that the supply system needs to be provided from the outside. If you want to achieve rapid response, you must provide a huge supply system. This greatly reduces the portability of the pneumatic-driven soft gripper. Long Li and colleagues presented a compact soft gripper with a polylactic acid-based variable stiffness [7]. Kiju Lee and colleagues introduced a new cable underdrive actuated soft gripper called TWISTER Hand in 2020 [8]. These two soft grippers are driven by external motors with passive structures. The disadvantage of an external motor drive is that the motor components are rigid and bulky, which limits the flexibility and flexibility of the soft gripper. Some soft grippers were driven by electroactive polymers [9,10,11,12]. Electroactive polymer is a kind of polymer that can be reversibly deformed by electrical stimulation. An electroactive polymer drive has the advantages of being able to handle a large strain (>100%) and having high efficiency, high energy density, and fast response (millisecond order). However, an electroactive polymer drive requires a very high voltage, up to several thousand volts. The driving voltage is too high, which will increase the risk of soft robots. Moreover, a complex manufacturing process and stacking configuration with a large electrode area is required, which increases the complexity of its manufacturing [13]. Mingfang Liu and colleagues used shape memory alloy (SMA) to design a soft robot with variable stiffness [14]. Haibin Yin and co-workers introduced a new soft gripper with variable stiffness [15] and deduced its mechanical model. The same point of these two soft grippers is the use of SMA drive. SMA will show the shape memory effect due to the crystallization change between the martensite phase and austenite phase caused by temperature. However, the biggest disadvantages of this driving mode are a slow response time and low efficiency [16].

In recent years, gas–liquid phase transition has increasingly been applied. In soft robots, a thermal flow control system based on a gas phase transition driving mode has been shown to have a low cost and the ability to establish fluid pressure in the device, which can be used to eliminate the need for large motors and compressors in the traditional fluid elastic driving system. By using low-voltage equipment to directly trigger the phase transition, the driving system can become very compact; this driving mode also produces a large force and deformation. Currently, the application of gas–liquid phase transition in the soft robotics field can be roughly divided into two types: (1) placing the liquid in silica gel for phase transition [17,18,19] or (2) having the liquid encapsulated in a gas bag for the phase transition [20,21,22,23]. First, because of the limitations of the material properties of silica gel, perfect sealing cannot be achieved. The phase transition in the gas will flow out of the tiny gaps of silica gel as the service time increases, which greatly reduces the service life. The second method is limited by the gas bag material, which is easily damaged. To simultaneously solve these two shortcomings of current gas–liquid phase transition soft robots, a new gas–liquid phase transition actuator is designed in the present study, and a gas–liquid phase transition soft gripper is developed with this actuator, as shown in Figure 1.

Table 1. The characteristics comparison of various gas-liquid phase transition soft robots.

Citation of Soft Robot	Gas Tightness	Degree of Vulnerability	Liftable Weight
Ragesh Chellattoan [17]	Poor	Difficult to be destroyed	110 g
Tomoki Noguchi [18]	Poor	Difficult to be destroyed	-
Seyed M. Mirvakili [19]	Poor	Difficult to be destroyed	59 g
Takefumi Hiraki [20]	Good	Easy to be destroyed	11 g
Tatsuya Usui [21]	Good	Easy to be destroyed	-
Koya Narumi [22]	Good	Easy to be destroyed	-
Nicholas Kellaris [23]	Good	Easy to be destroyed	500 g
This paper	Good	Difficult to be destroyed	410 g

shows the characteristics comparison of various gas-liquid phase transition soft robots.

The main contributions of the current paper are as follows:

• In this study, a new type of gas–liquid phase transition actuator is designed, and a gas–liquid phase transition soft gripper is developed with this actuator. This actuator solves the two shortcomings of easy gas leakage and easy destruction in current gas–liquid phase transition soft robots. This actuator is found to have several outstanding advantages: (1) Compared with fluid drive and motor drive, this kind of gas–liquid phase transition actuator eliminated the huge and complex external supply system, making its structure smaller, more compact, and lighter. (2) Low driving voltage. (3) The small voltage can produce large displacement and force.
• According to the principle of thermodynamics, the kinematics of the gas–liquid phase transition actuator are analyzed, the relationship between time and displacement is deduced, and experiments are designed to verify the correctness of the mathematical model.
• Deep reinforcement learning is used to optimize the control strategy of the soft gripper to judge whether and how to grasp.
• The actuator and soft robot are tested and verified.

2. Principles and Method

2.1. Design, Driving Principle, and Mathematical Model of Actuator

The driving principle of the gas–liquid phase transition actuator is inspired by the hydraulic principle and operates according to the thermodynamic phase transition principle, as shown in Figure 2. The main components of the gas–liquid phase transition actuator are the heating element and the elastic damping element spring. A phase transition liquid is injected into the actuator to fully immerse the heating element. After the heating element is energized, the temperature, volume, and air pressure inside the cavity will change. When there is a current I through the heating element, the heating element will rise in temperature and generate a large amount of heat Q. The generated heat is then transferred to the liquid, converting the liquid from the liquid to the gas phase. Because of the appearance of a large amount of gas, when no external load is applied, the pressure and volume inside the actuator will increase, and then the push rod will move upward. Reverse actuation is achieved by cutting off the current to stop heating and by passive cooling of the components. This type of gas–liquid phase transition actuator does not need external pneumatic and hydraulic compressors to provide power, so the structure is simpler and more compact.

As shown in Figure 3a,b, the gas–liquid phase transition actuator is composed of a hollow cylindrical shell, a cylindrical push rod, a spring, a ceramic heating plate, and a cable connector. The overall structure adopts the structure of liquid-like injection instruments that are commonly used in medical systems. The internal end of this structure itself has good tightness, and the structure is simple and easy to manufacture, which is an important reason for using liquid-like injection instruments. At the end of the actuator, to meet the conditions of connecting the internal heating plate with the external power supply and ensuring the gas–liquid tightness, a cable connector with good gas tightness that is commonly used for underwater cables is used. The structure of this connector is shown in Figure 4; It is composed of a nut, rubber gasket, basic accessory body, tightening rubber, and tightening nut. The most critical one is the special grasping claw on the basic accessory body. The spacing design of the grasping claw enables it to shrink with the compression and contraction of external components. The specific use process is as follows. First, inject sufficient phase transition liquid from the end of the actuator to ensure that these liquids can always soak the heating sheet. Then screw the basic accessory body to the tightening nut so that the special grasping claw on the basic accessory body will shrink with the rotation and compression of the thread so that the internal rubber ring will also shrink and compress the cable passed through. This makes it so that there is no gap between the nylon ring and the cable and finally achieves the function of liquid and gas leakage prevention. The heating element for heating the liquid is a ceramic heating sheet, which is internally made by a special tungsten paste printing process. It is characterized by rapid temperature rise, high density, and acid and alkali resistance. At the same time, to improve the performance of the actuator and solve the problem of the slow recovery of the actuator because of friction during recovery, two springs are added to the structure of the liquid-like injection instrument, and the springs are connected on both sides between the push rod and cylindrical shell through connectors.

According to the principle of thermodynamics, the dynamic mathematical model of the actuator is derived with reference to the thermodynamic change process of water vapor. When the liquid temperature is lower than the saturation temperature, it is called supercooled liquid, or unsaturated liquid, as shown in Figure 5a. When the supercooled liquid is heated, the liquid’s temperature gradually increases, and the specific volume of the liquid slightly increases. Heating the liquid from an unsaturated state to a saturated state is called the preheating stage, and the heat required is called the temperature rise heat. Continuing to heat the liquid eventually leads to saturation temperature, and the liquid begins to boil and vaporize. At this time, the saturation pressure does not change, and the saturation temperature does not change either. This mixture of vapor and liquid is called wet saturated vapor (wet vapor for short), as shown in Figure 5c. As the heating process continues, the liquid gradually decreases, and the steam gradually increases until all the liquid turns into steam. At this time, the steam is called dry saturated steam (hereinafter referred to as saturated steam), as shown in Figure 5d. In the process of heating saturated liquid to dry saturated steam at constant pressure, the specific volume of the working medium increases rapidly with the increase in steam, but the temperature of the steam and liquid does not change, and the absorbed heat is transformed into the external expansion work caused by the increase in internal potential energy and specific volume of steam molecules. This heat is the vaporization heat, Q_V. The heat released by isobaric condensation of 1 kg saturated steam is equal to the latent heat of vaporization at the same temperature. If the saturated steam is heated continuously, the steam temperature will increase, and the specific volume will increase. Here, the steam is called overheating steam, as shown in Figure 5e. The value at which the temperature exceeds the saturation temperature is called superheat. The heat absorbed by steam during overheating is called superheat heat.

In the current study, the amount of ethanol liquid injected into the actuator cavity can ensure that it would not be completely converted to gas within the working time, so we only needed to study the first three stages. In the process of converting unsaturated liquids into saturated liquids, the volume change tends to be small because of the time period, so the volume change has been ignored in the calculation. According to whether the volume changed, the three stages have been divided into two steps. The first step is the conversion of unsaturated li-quid to saturated water, and the second step is the conversion of saturated liquid to wet saturated steam.

The energy required for the phase transition of the gas–liquid phase transition actuator comes from the energy generated by the heating plate, and the generated heat, Q, is:

$$\begin{array}{c}Q=I^{2}Rt\end{array}$$

(1)

The destination of the heat is divided into three parts: one is the heat required for the liquid to be heated to the boiling temperature, which is the heat of the temperature rise; the second is the heat that the liquid needs to absorb when it is converted from liquid to gas of the same mass, which is the latent heat of gasification; and the third is the heat lost in the air during the process:

$$\begin{array}{c}Q=Q_w+Q_v+Q_l\end{array}$$

(2)

$$\begin{array}{c}{Q}_w=m_1\mathrm{c}\left(T_b-T_v\right)\end{array}$$

(3)

$$\begin{array}{c}{Q}_v=mr\end{array}$$

(4)

$$\begin{array}{c}{Q}_l=\gamma Q\end{array}$$

(5)

where Q is the total heat generated by the ceramic heating strip, Q_W is the heat of the temperature rise when the liquid is heated to the boiling temperature, Q_V is the latent heat of gasification, Q_l is the heat loss, γ is the heat loss coefficient, c is the specific heat capacity, T_b is the boiling temperature, T_V is the initial temperature, r is the heat of gasification, m₁ is the initial liquid mass, and m is the mass of the phase transition gas.

In this process, the amount of gas produced is determined by the mass of the gas produced and molar mass of the corresponding substance.

$$\begin{array}{c}n=\frac{m}{M}\end{array}$$

(6)

where n is the amount of substance, m is the mass of the object, and M is the molar mass. Because of the strong volatility of ethanol, the liquid will change phases before boiling. Therefore, in order to reflect the phenomenon that the liquid will change phase earlier, it is assumed that Q_w during heating is

$$\begin{array}{c}{Q}_w=\frac{1}{2}{m}_{1}c\left(T_b-T_v\right)\end{array}$$

(7)

Simultaneous (1)–(6) show that the molar amount of phase to gas is

$$\begin{array}{c}\left\{\begin{array}{c}n=\frac{\left[\left(1-\gamma \right)I^{2}Rt-\frac{1}{2}{m}_{1}c\left(T_b-T_v\right)\right]}{rM} When \left(1-\gamma \right)I^{2}Rt-\frac{1}{2}{m}_{1}c\left(T_b-T_v\right)>0 \\ n=0 When \left(1-\gamma \right)I^{2}Rt-\frac{1}{2}{m}_{1}c\left(T_b-T_v\right)<0 \end{array}\right.\end{array}$$

(8)

Generally, the gas after the vaporization of liquid can be regarded as an ideal gas, so it can be calculated by the ideal gas equation, whose equation is

$$\begin{array}{c}PV=nRT\end{array}$$

(9)

According to the actuator cavity structure

$$\begin{array}{c}PV=PS\left(x+x_1\right)\end{array}$$

(10)

where P is the gas pressure inside the actuator, R is the molar gas constant, s is the cross-sectional area of the internal rubber, and x is the internal gas height before starting work.

The force on the end of the actuator is analyzed, and the force diagram is shown in Figure 6. The equation is

$$\begin{array}{c}PS=F_0+F_s+F_m\end{array}$$

(11)

where F₀ is the constant force during the working process, including force produced by atmospheric pressure F_a, and friction F_f.

$$\begin{array}{c}{F}_0=F_a+F_f\end{array}$$

(12)

$$\begin{array}{c}{F}_a=P_a{S}_1\end{array}$$

(13)

where P_a is the atmospheric pressure, S₁ is the cross-sectional area of the push rod in contact with the air outside the actuator, F_m is the external load received when the actuator encounters obstacles during movement, F_s is the restoring force provided by the spring, and the phase transition actuator is two identical springs in parallel, so the expression of F_s is

$$\begin{array}{c}{F}_s=2kx\end{array}$$

(14)

where k is the elastic coefficient of the spring and x is the deformation of the spring moving with the push rod.

The following equations can be obtained by (8)–(14)

$$\begin{array}{c}\left\{\begin{array}{c}2kx+P_a{S}_1+F_f+F_m\left(x+x_1\right)=nRT\\ PV=PS\left(x+x_1\right)\end{array}\right.\end{array}$$

(15)

In the process of liquid conversion from supercooled liquid to saturated liquid, the heating element in the current study could heat quickly, so its volume change can be ignored. When the saturated liquid boils and vaporizes, the temperature of the gas and liquid is constant, so T can be approximately regarded as a constant. Therefore, the push rod displacement x, and the gas pressure P, inside the actuator can be calculated directly from Equation (15).

2.2. Design, Manufacture, and Mathematical Model of Soft Robot

According to the motion characteristics of the actuator, the soft robot has been designed with passive compliant soft gripper evenly distributed around the center of the circle 120° apart. The overall structure is divided into two main parts. The upper part is the driving part, composed of a single gas–liquid phase transition actuator and various connectors. This part provides the driving force for the soft robot. The lower part is the grasping part, which is responsible for grasping the object in contact with the object. It is mainly composed of fixed parts and three silicone soft grippers evenly distributed around the center of the circle at an interval of 120°. The structure only needs a phase transition actuator to drive three silicone soft grippers to move at the same time to adapt to the shape and size of the grasped object. Because of the strong flexibility of silica gel itself, it can adapt well to the shape and size of the grasped object while grasping to avoid damage to the object. The production of silicone soft grippers is shown in Figure 7. These were made by pouring 20° medical silicone rubber PS6600 mixed with 1:1 AB silicone solution into a 3D printing mold. The cost of the soft silicone produced by this method is low. First, we poured the AB silicone solution into the PLA (Polylactic Acid) mold printed with a 3D printer in advance and then took it out after letting it rest for 8 h.

Because the soft robot is a 120° uniformly distributed structure, a group of structures can be selected separately for kinematic analysis. The simplified structure diagram and its coordinate diagram are shown in Figure 8. Suppose the movement of the push rod is x; that is, point C moves -x along the negative half axis of Z. According to the geometric relationship, the coordinate positions of the upper and lower points (D, E) corresponding to the contact surface between the silicone soft gripper and the object are deduced. Assume the coordinates of point D is (X_d, Z_d) and the coordinates of point E is (X_e, Z_e).

Then, the coordinates of point D (X_d, Z_d) and point E (X_e, Z_e) are

$$\begin{array}{c}{l}_6·sin{\theta }_2 -l_5·sin{\theta }_1=l_4-x-l_3\end{array}$$

(16)

$$\begin{array}{c}{l}_5·cos{\theta }_1+l_6·cos{\theta }_2=l_2\end{array}$$

(17)

$$\begin{array}{c}AB=l_6,BC=l_5\end{array}$$

(18)

$$\begin{array}{c}{X}_b=l_1+l_5·cos{\theta }_1\end{array}$$

(19)

$$\begin{array}{c}{Z}_b=l_4-x+l_5·sin{\theta }_1\end{array}$$

(20)

$$\begin{array}{c}BD=l_7, DE=l_8\end{array}$$

(21)

$$\begin{array}{c}{\theta }_4=\frac{\pi }{2}-{\theta }_3+{\theta }_2\end{array}$$

(22)

$$\begin{array}{c}{\theta }_5=\pi -{\theta }_2+{\theta }_6\end{array}$$

(23)

$$\begin{array}{c}DB=l_9,DE=l_{10}\end{array}$$

(24)

$$\begin{array}{c}{X}_d=X_b+l_9·sin{\theta }_3\end{array}$$

(25)

$$\begin{array}{c}{Z}_d=Z_b+l_9·cos{\theta }_3\end{array}$$

(26)

$$\begin{array}{c}{X}_e=X_d+l_{10}·sin{\theta }_5\end{array}$$

(27)

$$\begin{array}{c}{Z}_d=Z_d+l_{10}·cos{\theta }_5\end{array}$$

(28)

2.3. Control Principle and Method

As shown in Figure 9, the experimental control system is mainly composed of PC (Personal Computer), voltage-stabilized source, STM32, ceramic heating plate, display sensor, MOSFET, and vision sensor. The function of the ceramic heating plate is to supply energy and heat to the liquid of the actuator inside the soft robot, making the liquid undergo gas–liquid phase transition, which then makes the soft robot grasp the object. The function of displacement sensor is to obtain the displacement information of actuator in real time. Vision sensor has two functions. One is to obtain the shape information of the clamped object in real time, and transmit the shape information to the DQN network to select the clamping strategy. The second is to obtain the size information of the clamped object and transmit it to STM32.

Figure 10 shows the overall diagram and connections of the control system as well as embedded controller (STM32) plus soft gripper. The main purpose of the controller unit is to adjust the voltage of the ceramic heating plate by outputting PWM signal from STM32 so that the ceramic heating plate can heat the phase transition liquid with appropriate voltage. Measure the size of the clamped object through the vision sensor, and then convert the size into the displacement (disp) of the actuator through calculation. Compare the calculated disp with the actual displacement read by the displacement sensor to obtain the error signal. Finally, through the PI (Proportion Integration) controller, STM32 outputs the appropriate PWM signal to adjust the voltage. The selection strategy after Four-Classification DQN processing returns to the host computer to control the rotation motion of the manipulator.

The basic idea behind reinforcement learning is to learn the optimal strategy to achieve a goal by maximizing the cumulative rewards obtained by agents from the environment [24]. Therefore, reinforcement learning can interact with the environment. Reinforcement learning learns strategies by interacting with the environment and obtaining state and feedback.

Because reinforcement learning can interact with the environment, we chose deep reinforcement learning as the basic control method for soft robots. The soft robot could continuously obtain the shape of the grasped object in the environment through the visual sensor and then use reinforcement learning to determine how to grasp the object through the grasped state geometry, action set, state transition table, and return.

We have used Deep Q-learning (DQN) to construct a four-classification neural network to judge the contour category of objects when looking down. If its contour belonged to the first category, strategy 1 would be adopted. If its contour belonged to the second category, strategy 2 would be adopted. If its contour belonged to the third category, strategy three would be adopted. If its outline did not belong to the first three categories, it would not be pinched. Figure 11 shows the specific classification strategy and pinching strategy [25,26].

To build a deep reinforcement learning neural network to adapt to special environments, we collected four common images of grasped object contours in the laboratory as the data set. There were 400 images in the data set, and each type of image accounted for the same proportion, accounting for one-fourth of each, that is, 100. The ratio of the training set to the test set was 4:1; that is, 80 pieces of each type were randomly selected as the training set, and 20 pieces were the test set to verify the performance of the model. When the data set is small, the model training will produce a serious overfitting phenomenon, so before the model training, the data set was preprocessed and expanded at the same time. First, we mapped the original range of 0–255 pixel values to 0–1. The image data were centralized by deaveraging so that the pixel mean value of the image was 0 and standard deviation 1. In this way, data centralization could conform to the law of data distribution, and it was easier to achieve the generalization effect after training. At the same time, random image rotation, random image translation, color jitter, and random brightness adjustment were carried out on the image. The image processing technology of data enhancement was used to expand the dataset of small samples. Data enhancement had the function of regularization, which could reduce the structural risk of the model, improve the robustness of the network, and improve the accuracy of model recognition of the four kinds of grasped object contours.

The basic idea of DQN is to use a convolutional neural network (CNN) to encode the feature of the picture first and then use the full connection layer to output the corresponding Q value. Equation (29) uses a CNN to achieve the best action-value function. In the iteration of step I in Q-learning, the following loss function is defined as Equation (30), and the Q value is the estimated value calculated by the neural network function. Finally, the Bellman optimization equation can be used to calculate the long-term return and the mean square error function to update each weight. The loss function is differentiated according to the network weight, and the gradient Equation (31) can be obtained. As shown in Figure 12, the captured image is cropped into a color image with 150 × 150 pixels. The input image is transformed into a 128-dimensional feature vector through the use of a CNN. The full connection layer takes the feature vector as the input and then outputs four Q-values. The action taken in this state uses one hot coding as another input, and dot multiplies these four Q-values. Other values, save for this action, are set to 0 and then added up to obtain the Q-function.

$$\begin{array}{c}{Q}^{*}\left(s, a\right)=\underset{\pi }{\mathit{max}}E\left(r_t+\gamma r_{t+1}+{\gamma }^2{r}_{t+2}+\dots |s_t=s, a_t=a, \pi \right)\end{array}$$

(29)

The calculated value represents the sum of the reward r_t discounted by strategy π = P(a|s) for the observation s and action a at each time step t. In order to solve the instability problem caused by reinforcement learning when the neural network is used to approximate the state value function, empirical playback [27] and the target network are used.

$$\begin{array}{c}{L}_i\left({\upsilon }_i\right)=E_{\left(s,a,r,s^{\prime }\right)~U\left(D\right)}\left[{\left(r+\gamma \underset{a^{\prime }}{\mathit{min}}Q\left(s^{\prime },a^{\prime };{\upsilon }_i^{-}\right)-Q\left(s,a;{\upsilon }_i\right)\right)}^2\right]\end{array}$$

(30)

γ is the discount factor that determines the vision of the robot, ${\upsilon }_i$ is the parameter of Q network in step i, and ${\upsilon }_i^{-}$ is used to calculate the goal of step i. Target network parameters ${\upsilon }_i^{-}$ use Q network parameters for each C step ${\upsilon }_i$ is updated and then remains unchanged during the two updates.

$$\begin{array}{c}{\nabla }_{{\upsilon }_i}{L}_i\left({\upsilon }_i\right)=E_{s,a,r,s^{\prime }}\left[\left(r+\gamma \underset{a^{\prime }}{\mathit{min}}Q\left(s^{\prime },a^{\prime }; {\upsilon }_{i-1}\right)-Q\left(s,a; {\upsilon }_i\right)\right){\nabla }_{{\upsilon }_i}Q\left(s,a; {\upsilon }_i\right)\right]\end{array}$$

(31)

3. Results and Discussion

3.1. Experiment on the Relationship between Displacement and Time of the Actuator

To measure the relationship between the displacement and time of the gas–liquid phase transition actuator, an experimental platform for measuring the relationship between the displacement and time was built, as shown in Figure 13. Time–displacement comparison experiments were carried out for the gas–liquid phase transition actuator with a spring and the gas–liquid phase transition actuator without a spring, respectively. The two actuators were compared at 5.5 V, 6.5 V, 7.5 V, 8.5 V, and 9.5 V. The measured time–displacement curves are shown in Figure 14. As shown in Figure 14, the maximum displacement of the actuator was 34.4 mm. Of course, this depended on the size of the actuator, the initial position of the installation, and the position of the end limit. In theory, a larger maximum displacement can be achieved. Figure 14f shows that the time to reach the maximum displacement of the actuator decreased with the increase in voltage in this range and that the recovery time also decreased with an increase in the voltage. It can be seen from Figure 14a–e that after adding the spring, the time for the actuator to move outwards did not increase much when compared with the situation of not adding the spring, and even when the voltage reached 8.5 V and 9.5 V, the difference was only 1 s. Based on this, the time for the actuator to recover and return can be greatly shortened. For example, at 8.5 V, compared with the actuator without spring, the gas–liquid phase transition actuator with spring had a 5% difference in reaching the maximum displacement time, but the time from reaching the maximum to returning to 15 mm was faster by 58%. At 9.5 V, compared with the actuator without a spring, the gas–liquid phase transition actuator with a spring had a 20% difference in reaching the maximum displacement time, but the time from reaching the maximum to returning to 15 mm was faster by 70%. Therefore, it was a comprehensive consideration of energy efficiency, driving time, and speed. Here, 8.5 V was selected for subsequent experiments and physical control. In addition, comparing the two actuators, it was concluded that the performance of the actuator could be greatly improved by adding a spring. At the same time, Figure 14 shows that the actuator with spring will have a certain time delay before obvious displacement changes after starting and stopping heating. Figure 15 shows the time delay of the actuator after starting and stopping heating when the voltage is 8.5 V. It can be seen from Figure 15 that after the heating starts, it needs to be delayed for 8 s to produce a significant increase in displacement. After the heating is stopped, it needs to be delayed for 28 s to produce an obvious displacement reduction. There are two reasons for the time delay after the start of heating. First, it takes time for phase transition liquid to absorb heat during the conversion from supercooled liquid to saturated liquid. Second, after the phase transition liquid turns into gas, it takes a period of time to generate enough pressure to overcome the friction. There are also two reasons for the time delay after heating is stopped. First, after the heating is stopped, the temperature of the heating plate is still higher than the boiling temperature of ethanol. It takes time to fall below the boiling temperature before the phase transition liquid can change into the liquid phase. The second is also the cause of friction.

Furthermore, according to Equation (15), we describe and compare the time–displacement relationship between the mathematical model and experiment in Figure 16. It can be seen that, in general, the mathematical model was slightly higher than the experimental data, but the upward trend was the same and the overall maximum displacement time similar, proving the correctness of the mathematical model. This difference existed because, in practice, the volume change in gas pressure and temperature during the phase transition process will cause changes in the physical properties of the liquid itself, such as the boiling point, heat of vaporization, and density.

3.2. Experiment on the Relationship between Force and Time of Actuator

In order to measure the force–time relationship of the gas–liquid phase transition actuator after a given displacement, an experimental platform for displacement measurement was built, as shown in Figure 17. The experimental platform mainly had a force sensor and fixed platform. Under a voltage of 8.5 V, the actuator was placed at a distance of 0 mm, 3 mm, 5 mm, 8 mm, 10 mm, 13 mm, 15 mm, 18 mm, 20 mm, 23 mm, and 25 mm from the dynamometer for testing, and the data of the time–displacement force was measured. We fit these data to a three-dimensional surface, which showed the law of the force generated when the actuator end encountered obstacles in the range of 0 mm to 25 mm. The plane part in Figure 18 was specially processed to obtain the fitting data, and the slope part was the corresponding time–displacement force obtained when the corresponding distance was loaded. When there was no load, the end force was 0, and the displacement time is referred to in the time–displacement diagram in Figure 14. When the voltage was 8.5 V, and there was no load, the average rate of increase in the force of the actuator was about 1 N/s.

3.3. Control Effect Comparison of the Different Control Strategy

Table 2. Experiment results of gripper grasping.

Method	Accuracy	Error
DQN	91.3%	8.7%
BPNN	85.6%	4.4%
CNN	83.1%	6.9%

shows the accuracy of strategy selection of the BPNN (Back Propagation Neural Network), CNN (Convolutional Neural Network), and DQN control method. The table shows that the accuracy of strategy selection of the BP control method is 83.1%, the accuracy of strategy selection of the CNN control method is 85.6%, and the accuracy of strategy selection of the DQN control method is 91.3%. It can be seen that the accuracy of strategy selection of DQN is 5.7% higher than that of BP and 8.2% higher than that of CNN.

3.4. The Grasping Experiment of the Soft Robot

In the grasping experiment, some common objects were selected and grasped by a soft robot. Some representative grasping work was summarized in Figure 19, and the corresponding parameters of the grasped object are shown in

Table 3. Experiment results of gripper grasping.

Number	Target	Size (mm)	Weight (g)
a	apple	Φ75H68	207
b	adhesive tape	Φ75H48	108
c	rectangular mold	55 × 70 × 25	32
d	USB Disk	65 × 20 × 10	9
e	box with weights	138 × 101 × 68	330
f	triangular prism mold	98 × 67 × 32	58
g	mouse	120 × 65 × 40	117
h	pillow	350 × 250 × 75	410

. The experiment was repeated 10 times for each object, with all experiments showing it was effective for grasping, except that the (h) grasping experiment had only a 50% success rate (generally, grasping an object for 10 s without falling is considered effective grasp). It can be seen from Table 3 that the safe cross-sectional size of the object that could be grasped by the soft robot was in the range of Φ20~Φ100 mm, and the maximum weight of the object that could be grasped was 410 g. Compared with its own weight of 320 g, the maximum weight that a soft robot can grasp is about 130% of its own weight.

4. Conclusions

In the current paper, a soft robot based on a gas–liquid phase transition actuator with elastic damping structure has been proposed. The gas–liquid phase transition driving mode enables the soft robot to generate a large driving force and large displacement only by using a low voltage. Because there is no need for an external large power source, the whole structure is small and compact. The addition of an elastic damping structure greatly reduces the working return time of the actuator. At the same time, the actuator with a new structure solves the problem of repeated liquid filling caused by gas leakage in the field of gas–liquid phase transition soft robots. This paper also added the Deep Reinforcement Learning algorithm (DQN algorithm) to select the appropriate clamping strategy for it. The experimental results show that the maximum weight that the soft robot can grasp is about 130% of its own weight.