Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence [131.4] タスク認識方式でアダプタを初期化する新しい手法であるコンテキスト指向分解適応(CorDA)を提案する。 本手法は,タスク認識により,知識保存モード (KPM) と命令レビューモード (IPM) の2つのオプション適応モードを実現する。 論文参考訳（メタデータ） (Mon, 16 Jun 2025 07:55:14 GMT)
knowledge-preserved mode (KPM) 、instruction- previewed mode (IPM)の導入、結果「Experimental results demonstrate that our method in KPM outperforms LoRA not only in downstream performance but also in maintaining zero-shot capabilities for both large language models and vision language models. Meanwhile, the IPM exhibits superior fine-tuning performance and faster convergence in both standard and quantized adaptation across various tasks.」とのこと。