In the new power system, a large number of new energy sources and power electronic equipment are connected to the AC power grid. When a fault occurs in the busbar area, affected by the control strategy, the amplitude and angle of the fault current are controlled, and the harmonic content is high. As a result, the operating performance of the ratio differential protection declines. The basic principle of the traditional ratio differential algorithm is introduced, and the problems of the traditional ratio differential algorithm in the new power system are analyzed. An improved algorithm for ratio differential protection for busbar protection is proposed, which is not affected by the angle difference and harmonics of the fault currents. The current phasors of each bay with different angles are mapped to the same coordinate system. Then, the differential current and restraint current are calculated. The operating performance of the improved ratio differential algorithm is analyzed under internal and external faults of the busbar area and current transformer (CT) saturation during external faults. An improved logic for the ratio differential protection is proposed. The real time digital simulation (RTDS) test results compare the operating performance of the traditional ratio differential protection and the improved ratio differential protection. It is proved that the improved ratio differential protection enhances the operating sensitivity without reducing the reliability of protection.