Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation

Xinyue Chen, Miaojing Shi·June 01, 2024

Summary

The paper introduces Memory-guided Network with Uncertainty-based Feature Augmentation (MENUA) for few-shot semantic segmentation, addressing class distribution shift and intra-class variance. It consists of a Class-shared Memory (CSM) module that aligns base and novel class representations by re-encoding query features, and an Uncertainty-Based Feature Augmentation (UFA) module that generates diverse query features through Gaussian re-parameterization. The CSM stores base class information for improved feature reconstruction, while UFA enhances robustness to input variations. Experimental results on PASCAL-5i and COCO-20i datasets show significant performance improvements over existing methods, particularly in 5-shot settings, with the proposed method setting new state-of-the-art scores. The study also includes ablation studies and explores hyperparameters to optimize performance.

Tables

Introduction

Background

Class distribution shift in few-shot learning

Intra-class variance as a challenge for segmentation

Objective

To address class distribution shift and intra-class variance

Improve few-shot semantic segmentation performance

Method

Class-shared Memory (CSM) Module

CSM Architecture

Encoding of query features for base and novel classes alignment

Re-encoding mechanism for enhanced feature representation

CSM Functionality

Base class information storage for improved feature reconstruction

Alignment of base and novel class representations

Uncertainty-Based Feature Augmentation (UFA) Module

Gaussian Re-parameterization

Generation of diverse query features through uncertainty modeling

Robustness to input variations

UFA Integration

UFA in the context of few-shot segmentation pipeline

Enhancing feature diversity and generalization

Experiments

Datasets

PASCAL-5i: Evaluation on class distribution shift

COCO-20i: Comprehensive analysis with varying shot settings

Performance Evaluation

State-of-the-art comparison in 5-shot and few-shot scenarios

Quantitative results and comparison with existing methods

Ablation Studies

Analysis of CSM and UFA contributions

Hyperparameter optimization experiments

Impact of memory and augmentation on segmentation performance

Conclusion

Summary of the MENUA's effectiveness in addressing challenges

Future directions and potential improvements

References

Cited works and methodology inspirations

Advanced features