FMVP: Fine-grained Meta Visual Prompt enabled domain-specific few-shot classification

Few-shot learning is a challenging and promising fundamental research. Inspired by recent advances in large language models (LLMs), visual prompt tuning has achieved notable performance gains in few-shot tasks by introducing only limited trainable parameters in the input space. Though effective, prompt tuning in few-shot settings heavily relies on well-initialized soft prompts and often lacks generalizability. Additionally, in certain specific fields, particularly in agriculture, there is a lack of high-precision fine-grained few-shot classification models. To our knowledge, this study is the first to employ prompt tuning for fine-grained few-shot plant disease classification (specific to disease severity). Specifically, we propose a novel Fine-grained Meta Visual Prompt tuning (FMVP) framework to systematically explore how visual prompts can enhance the generalizability of fine-grained few-shot domain-specific models. Firstly, a Sparsity-aware Meta Visual Prompt tuning (SMVP) sub-module is proposed to learn a universal visual prompt initialization. SMVP utilizes pixel-level optimizable visual prompts for input transformation, jointly with a novel sparsity-aware meta-learning paradigm for parameter updating, boosting generalizability to unseen classes. Secondly, a Fine-grained Cross-Alignment (FCA) module is introduced to explore intra- and inter-image relational patterns, enhancing fine-grained recognition by extracting object-level cross-image semantic discriminative features. Extensive experiments on datasets such as mini-ImageNet, CUB, and FPV have shown that our model outperforms state-of-the-art (SOTA) models. Our work constitutes a valuable addition to domain-specific models for practical applications.