HEVC encoder encodes every possible CU(Coding Unit) to determine CU depth which has the best compression efficiency based on RD-Optimization. Generally, a current CU’s spatial neighbor CU and co-located CU have similar features. Therefore, a fast CU depth decision method in this paper is proposed by using the similarity of neighbor CU’s feature. The simulation results are showed 7.27% decreased encoding time and degraded 0.31% BD-rate.