The traditional detection method for the defects of the pin on the power connection fitting in the aerial survey of the drone is dependent on manual marking. Aiming at this problem, the deep learning algorithm RetinaNet is used to automatically extract the features of normal and defective samples and complete the fusion of low-level features and top-level features to achieve automatic labeling of defects. Considering the fact that the number of defective samples is much smaller than the normal number of samples, Firstly, the influence of the category imbalance caused by the deficiency of the defect sample on the recognition result is analyzed. The results show that the trained model in this case will make a large number of defective samples be mistakenly recognized as normal classes. Therefore, at the data level of RetinaNet, class balanced sampling is proposed to ensure that each category has the same opportunity to participate in training. The experimental results show that the proposed method can significantly improve the average precision of defect categories under the premise of maintaining the high recognition rate of normal categories.