Purpose: Directional deep brain stimulation (DBS) leads have become widely used in the past decade. Understanding the asymmetric stimulation provided by directional leads requires precise knowledge of the exact orientation of the lead in respect to its anatomical target. Recently, the DiODe algorithm was developed to automatically determine the orientation angle of leads from the artifact on postoperative computed tomography (CT) images. However, manual DiODe results are user-dependent. This study analyzed the extent of lead rotation as well as the user agreement of DiODe calculations across the two most common DBS systems, namely Boston Scientific’s Vercise and Abbott’s Infinity, and two independent medical institutions. Methods: Data from 104 patients who underwent an anterior-facing unilateral/bilateral directional DBS implantation at either Northwestern Memorial Hospital (NMH) or Albany Medical Center (AMC) were retrospectively analyzed. Actual orientations of the implanted leads were independently calculated by three individual users using the DiODe algorithm in Lead-DBS and patients’ postoperative CT images. Deviation from the intended orientation and user agreement were assessed. Results: All leads significantly deviated from the intended 0° orientation (p