Closed-loop active control of the magnetic capsule endoscope with a robotic arm based on image navigation

This paper proposes a novel approach for the closed-loop active control of the magnetic capsule endoscope based on image navigation. The proposed method uses a robotic arm to control the rotation of an external permanent magnet (EPM) to generate a rotating magnetic field. Subsequently, this rotating magnetic field controls the motion of the magnetic capsule endoscope (MCE). A gyroscope was included inside the MCE to obtain its posture. Using the image captured by the camera installed in the MCE, a relationship between the relative position of the MCE and the intestine was established. Based on the relative position relationship and the MCE ’ s posture, the MCE was controlled to move along the intestinal center by the robotic arm, thus, the closed-loop active control of the MCE was achieved. Furthermore, the feasibility of closed-loop active control of the MCE was verified through the isolated porcine small intestine experiment. Experimental results show that the closed-loop active control combined with image navigation is not only easy to operate, but also offers high stability in terms of control mechanism, and suitability for use in clinical applications.


Introduction
Gastrointestinal (GI) diseases are one of the leading public health problems around the world [1][2][3].The high prevalence and healthcare costs due to GI diseases constitute a serious social and financial burden.Consequently, there is a dire need to develop low-cost and easy-tooperate approaches to assist in the treatment/diagnosis of GI diseases.Traditionally, physicians diagnose and treat GI diseases by inserting endoscopes such as gastroscope and enteroscope.Certainly, such traditional ways are inconvenient and not easy to use.Also, it is challenging for patients, in terms of both physical and mental aspects, due to the pain associated with the process.One of the potential solutions is to use a wireless capsule endoscope (WCE) that allows painless endoscopic imaging [4].With the advancement of technology, there has been significant improvement in the performance of WCE for satisfactory clinical applications [5][6][7].At the beginning of the diagnosis process, the WCE's movement in the intestine is achieved by its gravity and the peristalsis of the GI tract, and then it is discharged from the body in 6-8 h.The lesions are accurately diagnosed based on the information captured by the camera installed in the WCE [8,9].Although the use of WCE enables the painless diagnosis of GI diseases, WCE has disadvantages such as low pixel, passive operation mode, uncontrollable, unable to provide active control, and consumes a significant amount of time [10].Therefore, many researchers are interested in developing approaches for active control of the WCE to make it move forward or backward stably in the GI tract.This will help physicians repeatedly observe the parts they are interested in.
In order to realize an active control of the WCE, previous studies have proposed several methods such as the electric stimulation method [11], bionic robot method [12][13][14], magnetic control method [15][16][17][18], and so on.Among these methods, the magnetic control method has demonstrated several advantages such as excellent control, and clear gastric pictures; and can be tolerated [15].
The magnetic control method mainly introduces external magnetic sources such as electromagnetic coils and permanent magnets.The electromagnetic coil control method uses electricity to generate a magnetic field, which may cause harm to the human body and hence, is not suitable for use.In contrast, the permanent magnet control method uses an external permanent magnet (EPM) to drag WCE to generate its motion.Hence, it may produce an uneven force, which may cause great damage to the intestinal wall.
In order to find a potential solution to these issues, our previous work [9] proposed a magnetic control method to control the motion of the magnetic capsule endoscope (MCE).The method mainly uses a stepping motor to drive the EPM to generate a rotating magnetic field.Subsequently, this rotating magnetic field drives the MCE to rotate synchronously.Due to the screw structure on the surface of the MCE, it can generate a friction driving force, so that the MCE can move forward or backward stably in the intestine.However, there are still some issues in the active control method of MCE such as poor accuracy, instability of active control and the magnitude of the error is significant.Furthermore, the control is open-loop which is one of the key issues.
To address the above-mentioned issues, a method for closed-loop active control of the MCE based on image navigation is proposed in this paper.The robotic arm controls the MCE's motion based on image navigation.The rest of the paper is organized as follows.Methods and background research on the active control of the MCE is presented followed by the discussion on the closed-loop active control experimental setup and experimental analysis of the robotic arm carried out in this work.Subsequently, the limitations and future work, and the conclusion are reported.

Active control method of the MCE
There are limitations to the active control achieved by directly using EPM to drag MCE to generate its motion.Under active control, the MCE is subjected to its own gravity and the magnetic force of the EPM.When the EPM drags the MCE to move, due to the uncertain bending of the intestine and the dragging motion, it may very easily lead to a collision, and friction between the MCE and the intestinal wall may result in the injury of the intestinal wall, aggravating the pain of patients and threatening the health of patients.
In order to avoid such potential risks to the human body, the EPM can be rotated to generate a rotating magnetic field to drag the MCE to generate its motion.The MCE is used because the magnetic shell is radially magnetized, and the magnetization direction of radial magnetization can make the MCE rotate synchronously under the action of the rotating magnetic field.The EPM rotates around the axis, and the MCE will rotate around the axis in the opposite direction.A double-threaded structure is added to the outer surface of the MCE, as shown in Fig. 1.This double-threaded structure increases friction between the MCE and the intestine, converting the MCE's rotating motion into a linear motion.
By rotating the EPM clockwise [19] (as shown in Fig. 2), the MCE rotates counterclockwise.Due to the double-threaded structure, the MCE moves to the left along the intestinal tract.Similarly, if the EPM is rotated counterclockwise, the MCE will rotate clockwise and move to the right along the intestinal tract.In this way, the MCE can be actively controlled.

Balance position and synchronous rotation of the MCE
The MCE is subjected to its own gravity, and the EPM has a magnetic force on the MCE.So, when the EPM is directly above the MCE, and if the magnetic force is equal to gravity, then the MCE is in equilibrium (suspension state), which is also the ideal position.In this situation, the MCE will not cause damage to the upper and lower walls of the intestinal tract, hence, the most perfect control of the MCE can be achieved.
The EPM used in this paper and the annular magnet in the MCE are radially magnetized.Therefore, when the EPM rotates, the MCE will also rotate.As shown in Fig. 3, when the EPM rotates a quarter of a turn, the MCE also rotates exactly a quarter of a turn, which can also be considered as synchronous rotation [20].

Closed-loop control of the MCE based on image navigation
The MCE's simple linear motion in the intestinal tract can be achieved by controlling the rotation of the EPM and moving the EPM in the direction of desired linear motion.When a curved intestine is encountered, the position of the EPM needs to be adjusted to change the MCE's posture.As shown in Fig. 4, the intestinal information taken by the MCE was extracted through the computer vision system and fed back to the manipulator.The manipulator adjusts the robot arm to tune the MCE's posture in the corresponding direction until the head of the MCE is aligned with the intestinal center.Then it continues to control the forward motion of the MCE in this direction.If there is a deviation during the forward process, it needs to be adjusted again.The above operations were performed repeatedly so that the image navigation of the MCE in the intestinal tract can be completed.
Since the naked eye cannot observe the posture of the MCE in the intestine in detail, a gyroscope was added to the MCE to obtain its posture.However, the posture of the MCE relative to the intestine tract    still can not be obtained, and hence, the motion of the MCE along the intestine can not be controlled.
Through the image captured by the camera, the central point of the intestinal lumen can be obtained.As demonstrated in Fig. 5, the relative position of the intestine and the MCE was established.In this situation, the MCE is in the middle of the intestinal tract.It is necessary to control the rotation of the MCE at a certain angle to orient the MCE's head towards the central position of the intestine tract.The central point of the camera is O, and the head camera coordinate system is O-XcYcZc, so the vector S → in the Fig. 5 represents the approximate direction of the intestinal lumen [14].Vector S → can be represented using the swing angle (α) and pitch angle (θ).Where, α represents the angle between the projection of vector S → on the plane O-XcYc and Xc.θ represents the angle between the vector S → and the Z-axis.Through these two angle parameters, the relative position of the MCE in the intestine can be obtained.By using the keyhole imaging principle, the image captured by the camera is inverted.For convenience, the virtual imaging plane adopted for analysis is the reverse direction of the imaging plane.As shown in Fig. 6, O i is the center of the image in the virtual imaging plane, S i is the central point of the intestinal lumen.Therefore, the swing angle (α) is calculated as follows: where, x = x o − x oi , y = y o − y io .In addition, the angle between S i and Z C axis is θ, and the distance of OO i is the focal length (f) of the camera.The equation used to calculate the pitch angle (θ) is as follows: In ( 2), the pixel size p s and the focal length (f) are the built-in parameters of the camera.

Introduction to the gyroscope and other experimental equipment
The experimental equipment used in this experiment are: capsule, external permanent magnet (EPM), permanent magnet fixing device, annular magnet, robotic arm, gyroscope, endoscope module, medical silicone oil, plastic hose, and isolated pig small intestine.
The capsule used in the experiment was printed by a 3D printer, as shown in Fig. 7.The middle part of the capsule is a cylinder, which can place an annular magnet, an endoscope (camera), and a gyroscope.Both sides of the capsule are transparent hemispherical shells.The size of the capsule is 13.5 mm × 28.0 mm, and its mass is 4 g.The outer thread of the capsule is a double thread structure, which is composed of two equal pitch raised threads.The thread material is Acrylonitrile Butadiene Styrene (ABS), the diameter is 2 mm, the number of threads is 2, and the pitch is 17 mm.
The gyroscope used in this experiment is the YIS100 series gyroscope produced by Wuhan Yuansheng Innovation Technology Co., Ltd.The YIS100 series integrated industrial-grade MEMS three-axis gyroscope and three-axis magnetic sensor, through multi-sensor fusion algorithm and sensor test calibration ensures excellent motion and posture measurement performance in harsh environments.It meets the requirements of autonomous motion, control, and navigation.The parameters of the gyroscope are shown in Table 1.The sensor axes are defined as shown in Fig. 8.
In this experiment, the gyroscope needs to be connected to the USB interface and used at the PC end.Therefore, it is necessary to know the pin diagram and pin description of the YIS100 gyroscope.The pin diagram of the YIS100 gyroscope is shown in Fig. 9 and the pin description is reported in Table 2.
The YIS100 gyroscope used in the experiment was connected to the PC through the circuit cable.In order to reduce the resistance generated by the circuit cable during the rotation and motion of the MCE, it is necessary to select a thin circuit cable to reduce the winding force generated by itself.The circuit cable selected in this experiment was a 36 # cable, and its outer diameter was 0.28 mm (as shown in Fig. 10).This circuit cable was welded on the pads of the YIS100 gyroscope pins VCC, UART_TX, UART_RX, and GND, and connected with the USB port  pins 3V3, TXD, RXD, and GND, respectively.Later, the gyroscope was placed inside the capsule.The overall experimental set used in the work is shown in Fig. 10.
The camera used in this work was an industrial endoscope module (as shown in Fig. 11) with a diameter of 4.5 mm, a pixel of 1 million, and a resolution of 1280*720.The endoscope module was connected to the USB using a 36 # circuit cable.Once proper connections were established, the endoscope module was placed inside the capsule as shown in Fig. 12.

Closed-loop active control experiment of the isolated porcine small intestine
In order to demonstrate the closed-loop active control of the MCE, an experiment on the isolated porcine small intestine was performed.In the experiment, an annular magnet, gyroscope, and endoscope module were added to the capsule as shown in Fig. 13.
Before the experiment, the MCE was placed in the isolated porcine small intestine with an EPM directly above it, and the robotic arm controlled the rotation of the EPM to control the MCE's motion.During the experiment, the images captured by the endoscope and the MCE's posture were been displayed on the computer screen, and the data collected was used to align the MCE to the center of the intestinal lumen.The experiment measured the speed of the MCE moving in a straight line in the isolated porcine small intestine under the control of the robotic arm.The experimental results showing the relationship between the velocity of the MCE and the rotation velocity of the EPM are shown in Fig. 14.In the figure, the horizontal axis represents the rotation speed of the EPM, whereas, the vertical axis represents the movement speed of the MCE.It can be observed in Fig. 14 that initially, the EPM rotates slowly resulting in a slight movement of the MCE.However, as the speed of the EPM increases, the speed of the EPM is linearly related to the speed of the MCE.Thus, the linear relationship demonstrates the MCE's motion can be controlled stably.When the speed of the EPM is greater than a certain value (approximately 3 r/s), the MCE cannot quickly follow up the speed of the EPM, which leads to asynchronism between the MCE and the robotic arm.The experimental results show that as long as the rotation speed of the EPM is controlled within a certain range, the    MCE can be stably and linearly controlled to move forward and backward in the isolated porcine small intestine.The stable control offered is advantageous in terms of safety and can prevent the MCE from colliding with the intestinal wall.

The experimental results
Several experiments were performed to demonstrate the feasibility of the closed-loop active control of the MCE.The experimental setup and the process for the different motions of the MCE, carried out in this work are as follows: Based on the experimental results obtained for the setups shown in Figs.15-19, the closed-loop active control of the MCE on the isolated porcine small intestine was successfully achieved.Here, the sixth axis of the robotic arm was controlled to rotate clockwise and counterclockwise to drive the rotation of the MCE, and subsequently, to control the forward and backward motion of the MCE in the isolated porcine intestine.Initially, in the experiment, the posture of the MCE can not be clearly visualized, and the relative position between the MCE and the intestine was not known.For these reasons, the YIS100 gyroscope and endoscope module were used.Thus, the posture of the MCE can be seen clearly using the gyroscope, and the relative position relationship between the MCE and the intestinal tract can be obtained through the endoscope module.Thus, based on the results achieved in the experiments shown in this work, the MCE can be actively controlled to move along the center of the intestine tract through the robotic arm.Overall, the results validate the approach presented for the closed-loop active control of the MCE with a robotic arm using image navigation.

Conclusion and outlook
In this work, a closed-loop active control method for the MCE is proposed based on the synchronous rotation control principle of the MCE.The robotic arm controls the MCE's motion based on image navigation.The approach presented was validated using the isolated porcine small intestine experiment.Results achieved demonstrate different motions of the MCE actively controlled by the robotic arm.The posture of the capsule and the positional relationship between the capsule and the intestinal tract were obtained by including a gyroscope and endoscope module integrated with a camera in the capsule, respectively.Thus, the presented closed loop-active control approach of the MCE can control the motion of the capsule along the center of the intestinal tract.The feasibility of the closed-loop active control method was verified by making the MCE move forward and backward, left and right, uphill and downhill in the intestinal tract.Although the experimental results achieved in this work demonstrate successful implementation of the approach for the closed-loop active control of the MCE with a robotic arm, there are some limitations to the work presented.The limitations and potential future works are summarized below:

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
B.Ye et al.

B
.Ye et al.

Fig. 6 .
Fig. 6.Imaging plane used for the closed-loop control analysis of the MCE.

Fig. 10 .
Fig. 10.The experimental setup and the connection diagram of the YIS100 gyroscope and USB interface.

Fig. 11 .
Fig. 11.The endoscope module used in this work.

Fig. 12 .
Fig. 12.A diagram showing the placement of the endoscope module in the capsule used in this work.

Fig. 13 .
Fig. 13.Capsule used in the isolated porcine small intestine experiment.

Fig. 14 .
Fig. 14.Experimental results showing the relation between the rotating speed of EPM and the moving speed of MCE.

( 1 )
The experimental setup to achieve forward and backward motions of the MCE in the intestinal tract is shown in Fig. 15.(2) The experimental setup to test a left-turn motion of the MCE in the intestinal tract is shown in Fig. 16.(3) The experimental setup to demonstrate a right-turn motion of the MCE in the intestinal tract is shown in Fig. 17. (4) The experimental setup to verify an uphill motion of the MCE in the intestinal tract is shown in Fig. 18. (5) The experimental setup to demonstrate a downhill motion of the MCE in the intestinal tract is reported in Fig. 19.

Fig. 15 .
Fig. 15.Forward and backward motions of the MCE in the intestinal tract.

Fig. 16 .
Fig. 16.Left-turn motion of the MCE in the intestinal tract.

Fig. 17 .
Fig. 17.The right-turn motion of the MCE in the intestinal tract.

Fig. 18 .
Fig. 18.The uphill motion of the MCE in the intestinal tract.

( 1 )
Limited by the size and magnetism of the EPM, the control distance of the EPM for the MCE is not far enough, so the control distance of the EPM should be increased further.(2)The MCE in this experiment is limited by the circuit cable.The gyroscope and the endoscope must be connected with the PC through the circuit cable.Even if the circuit cable is thin enough, it will interfere with the experiment setup.Hence, the gyroscope and endoscope module can be replaced with a wireless module to avoid wired interference.(3) The performance of the closed-loop active control of the MCE can be improved further using the positioning technology of the MCE suitable for the closed-loop active control system.CRediT authorship contribution statement Bo Ye: Writingoriginal draft, Investigation, Methodology, Writing review & editing.Yingbing Fu: Writingoriginal draft, Investigation, Methodology.Shicong Zhang: Data curation, Formal analysis, Methodology.Hao Wang: Software, Methodology.Guo Fang: Software, Validation.Wei Zha: Resources, Visualization.Amit Krishna Dwivedi: Project administration, Supervision, Writingreview & editing.

Fig. 19 .
Fig. 19.The downhill motion of the MCE in the intestinal tract.

Table 1
Parameters of the gyroscope YIS100.

Table 2
Pin description of the YIS100 gyroscope.