You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug/ 问题描述
在使用LoRA进行PEFT微调的过程中,模型加载后使用add_adapter,会出现peft_config缺少属性的报错。
Hardware Environment(Ascend/GPU/CPU) / 硬件环境:
GPU
Software Environment / 软件环境:
-- MindSpore version : 2.2.11
-- Python version: 3.8.18
-- OS platform and distribution: Ubuntu 20.04
Excute Mode / 执行模式: pynative
To Reproduce / 重现步骤
frommindnlp.transformersimportAutoModelForSeq2SeqLMfrommindnlp.peftimportget_peft_model, LoraConfig, TaskTypemodel=AutoModelForSeq2SeqLM.from_pretrained("google-t5/t5-small")
peft_config=LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)
# The approach that works# model = get_peft_model(model, peft_config)# The approach that gives errormodel.add_adapter(peft_config)
model.enable_adapters()
model.print_trainable_parameters()
Expected behavior / 预期结果
Traceback (most recent call last):
File "/home/ernaux/Projects/mindspore/tmp/test_adapter.py", line 9, in <module>
model.add_adapter(peft_config)
File "/home/ernaux/Projects/mindspore/mindnlp/mindnlp/transformers/integrations/peft.py", line 244, in add_adapter
inject_adapter_in_model(adapter_config, self, adapter_name)
File "/home/ernaux/Projects/mindspore/mindnlp/mindnlp/peft/mapping.py", line 131, in inject_adapter_in_model
if peft_config.is_prompt_learning or peft_config.is_adaption_prompt:
AttributeError: 'LoraConfig' object has no attribute 'is_adaption_prompt'
The text was updated successfully, but these errors were encountered:
Describe the bug/ 问题描述
在使用LoRA进行PEFT微调的过程中,模型加载后使用add_adapter,会出现peft_config缺少属性的报错。
Hardware Environment(
Ascend
/GPU
/CPU
) / 硬件环境:GPU
Software Environment / 软件环境:
-- MindSpore version : 2.2.11
-- Python version: 3.8.18
-- OS platform and distribution: Ubuntu 20.04
Excute Mode / 执行模式:
pynative
To Reproduce / 重现步骤
Expected behavior / 预期结果
The text was updated successfully, but these errors were encountered: