Robot-assisted surgery that uses robotic systems, such as DaVinci®, with minimal invasive procedure has Been growing over the past years rapidly. However, the technical development for more dexterous and safer operation is still in early stage, and there are several problems remain to be overcome in the robotic surgery. Typical medical problems using a surgical robot system are collision of surgical instruments and tissue damages caused by miscontrolling surgical tools. Therefore, identifying and tracking the position of each surgical instrument in real time during robot-assisted surgery can prevent medical accidents. However, it is difficult to detect the surgical instruments on endoscopic vision due to its complexity, non-uniformity of scene, time-varying lighting condition, back-light, and high surface reflectance, etc. This study proposes several algorithms for tracking surgical instruments, which is robust against the various characteristics of endoscopic images in robot-assisted surgery.