Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation
Xinyue Chen, Miaojing Shi·June 01, 2024
Summary
The paper introduces Memory-guided Network with Uncertainty-based Feature Augmentation (MENUA) for few-shot semantic segmentation, addressing class distribution shift and intra-class variance. It consists of a Class-shared Memory (CSM) module that aligns base and novel class representations by re-encoding query features, and an Uncertainty-Based Feature Augmentation (UFA) module that generates diverse query features through Gaussian re-parameterization. The CSM stores base class information for improved feature reconstruction, while UFA enhances robustness to input variations. Experimental results on PASCAL-5i and COCO-20i datasets show significant performance improvements over existing methods, particularly in 5-shot settings, with the proposed method setting new state-of-the-art scores. The study also includes ablation studies and explores hyperparameters to optimize performance.
Introduction
Background
Class distribution shift in few-shot learning
Intra-class variance as a challenge for segmentation
Objective
To address class distribution shift and intra-class variance
Improve few-shot semantic segmentation performance
Method
Class-shared Memory (CSM) Module
CSM Architecture
Encoding of query features for base and novel classes alignment
Re-encoding mechanism for enhanced feature representation
CSM Functionality
Base class information storage for improved feature reconstruction
Alignment of base and novel class representations
Uncertainty-Based Feature Augmentation (UFA) Module
Gaussian Re-parameterization
Generation of diverse query features through uncertainty modeling
Robustness to input variations
UFA Integration
UFA in the context of few-shot segmentation pipeline
Enhancing feature diversity and generalization
Experiments
Datasets
PASCAL-5i: Evaluation on class distribution shift
COCO-20i: Comprehensive analysis with varying shot settings
Performance Evaluation
State-of-the-art comparison in 5-shot and few-shot scenarios
Quantitative results and comparison with existing methods
Ablation Studies
Analysis of CSM and UFA contributions
Hyperparameter optimization experiments
Impact of memory and augmentation on segmentation performance
Conclusion
Summary of the MENUA's effectiveness in addressing challenges
Future directions and potential improvements
References
Cited works and methodology inspirations
Advanced features