User Interface based on vision for controlling TV or PC has advantage that user could control TV or PC naturally and freely from all kind of device. So there have been variety of researches for recognition of human hand move by using RGB camera and Depth camera. but they have disadvantages that costing too much in setting and lack of accuracy of camera. Therefore proposing solution is that separating hand region from background accurately by using low cost infrared rays camera. and we can recognize the Click motion and Move motion by correlation of fingertips that extracting from contour, not using template matching or complicate study solution.