FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezing

The paper "FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezing" presents a technique to address unauthorized use of text-to-image models for illegal content. The method selectively freezes critical tensors during fine-tuning, allowing for model adaptation in legal domains while limiting representation in illegal ones. This is achieved through model publisher APIs, saving resources and preventing relearning of illegal adaptations. The technique is effective in reducing fake public figure and copyrighted content generation, with minimal impact on legitimate model usage. FreezeAsGuard employs bilevel optimization, continuous mask learning, and efficient gradient calculations to balance between legal and illegal domains. Experiments using 1B-parameter models and the FF25 dataset show significant mitigation in illegal domains while maintaining or improving performance in innocent ones. The study also acknowledges limitations, such as image quality degradation and non-uniform freezing patterns, but overall, FreezeAsGuard demonstrates a promising approach to controlling model adaptation for ethical AI use.