Drone is widely used in various field such as military, aerial photography, search and rescue mission, and environmental monitoring. However, drone safety accidents are increased due to poor pilot control and mistakes. Though dual-stick controller is the most widely used as controller, a fair amount of training time is required for the successful accomplishment of mission. To solve this problem, a well-designed remote controller is needed.
In this study, the primary goal is to develop a user-friendly drone controller. Deep learning-based hand gesture recognition with vibrotactile feedback are conducted to control drone. For appropriate vibration amplitude considering a distance between drone and obstacle, simulation is conducted. Moreover, preliminary experiments are carried out to collect the hand gesture patterns from the IMU sensors attached on the back of hand, which are related to the drone’s pose which consists of roll, pitch and yaw. Then, sensitivity analysis is performed to identify the key input parameters in associated with the deep learning algorithm. The gesture control is classified into two groups which are direct mode and gesture mode. Direct mode is for roll, pitch and yaw command with intuitive hand gesture and gesture mode is for direction command with hand signal of army and guidance. Also drone controller is combined with deep learning-based gesture recognition and vibrotactile feedback. To evaluate the controller performance, virtual drone flight system is implemented. Also mission complete time, gesture recognition accuracy and obstacle avoidance status are estimated. Then, a subjective evaluation is carried out for controller usability. Finally a flight experiment with real vehicle experiment is carried out to validate the proposed drone controller.