Relation Modeling and Distillation for Learning with Noisy Labels

RMDNet is a novel framework for learning with noisy labels in deep learning, combining relation modeling and knowledge distillation. It addresses overfitting to noisy data by using a self-supervised RM module for noise-resistant feature extraction through contrastive learning and a RGRL module to calibrate representations. RMDNet enhances model robustness by leveraging self-supervised learning and distillation to understand latent associations. The framework is versatile and outperforms existing methods on two datasets, showing improved discriminative representation learning despite noisy annotations. Key contributions include a noise mitigation strategy, compatibility with other methods, and a focus on representation error as a crucial factor for robustness. The paper explores various techniques, including robust loss functions, noise filtering, and contrastive learning, to handle noisy labels and presents empirical evidence of RMDNet's effectiveness.