Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add adapter过程中peft_config缺少属性 #1128

Open
hubosynapse opened this issue May 30, 2024 · 0 comments
Open

Add adapter过程中peft_config缺少属性 #1128

hubosynapse opened this issue May 30, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@hubosynapse
Copy link
Collaborator

Describe the bug/ 问题描述
在使用LoRA进行PEFT微调的过程中,模型加载后使用add_adapter,会出现peft_config缺少属性的报错。

  • Hardware Environment(Ascend/GPU/CPU) / 硬件环境:
    GPU

  • Software Environment / 软件环境:
    -- MindSpore version : 2.2.11
    -- Python version: 3.8.18
    -- OS platform and distribution: Ubuntu 20.04

  • Excute Mode / 执行模式:
    pynative

To Reproduce / 重现步骤

from mindnlp.transformers import AutoModelForSeq2SeqLM
from mindnlp.peft import get_peft_model, LoraConfig, TaskType

model = AutoModelForSeq2SeqLM.from_pretrained("google-t5/t5-small")
peft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)

# The approach that works
# model = get_peft_model(model, peft_config)

# The approach that gives error
model.add_adapter(peft_config)
model.enable_adapters()

model.print_trainable_parameters()

Expected behavior / 预期结果

Traceback (most recent call last):
  File "/home/ernaux/Projects/mindspore/tmp/test_adapter.py", line 9, in <module>
    model.add_adapter(peft_config)
  File "/home/ernaux/Projects/mindspore/mindnlp/mindnlp/transformers/integrations/peft.py", line 244, in add_adapter
    inject_adapter_in_model(adapter_config, self, adapter_name)
  File "/home/ernaux/Projects/mindspore/mindnlp/mindnlp/peft/mapping.py", line 131, in inject_adapter_in_model
    if peft_config.is_prompt_learning or peft_config.is_adaption_prompt:
AttributeError: 'LoraConfig' object has no attribute 'is_adaption_prompt'
@hubosynapse hubosynapse added the bug Something isn't working label May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant