Hand gesture recognition has emerged as new means to control smart electronic devices, video games and mobile phones. Currently commercialized hand gesture technology is simply to sensing the direction of hand movement or to recognizing hand posture/gesture using color camera. However, in no light condition or complex environment, it is challenging to recognize hand posture/gesture. To solve this problem, we propose a novel method for improvement of hand gesture recognition. It is using both the depth camera and infrared camera for robust recognition in complex environment. The proposed method sets the candidate hand region based on depth data, and extracts hand position in infrared image, then recognize hand posture using HOG-SVM. The experimental results show that proposed method can improve the accuracy of hand posture than conventional method.