Error: cannot import name 'ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST' from 'transformers' while using

[uni-manjunath-ke]

I am using PunctuationCapitalizationModel which was installed using:
python3 -m pip install git+https://github.com/NVIDIA/[email protected]#egg=nemo_toolkit[nlp]

But got below error:
Traceback (most recent call last):
File "/opt/api/batch_asr_server.py", line 13, in
from consumer import Consumer, add_consumer_arguments
File "/opt/api/consumer.py", line 9, in
from run import OfflineVadAsr, add_run_arguments
File "/opt/ds-offline-vad-asr/run.py", line 69, in
from nemo.collections.nlp.models import PunctuationCapitalizationModel
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/init.py", line 15, in
from nemo.collections.nlp import data, losses, models, modules
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/init.py", line 16, in
from nemo.collections.nlp.models.duplex_text_normalization import (
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/duplex_text_normalization/init.py", line 15, in
from nemo.collections.nlp.models.duplex_text_normalization.duplex_decoder import DuplexDecoderModel
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/duplex_text_normalization/duplex_decoder.py", line 34, in
from nemo.collections.nlp.models.nlp_model import NLPModel
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/nlp_model.py", line 31, in
from nemo.collections.nlp.modules import BertModule
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/modules/init.py", line 16, in
from nemo.collections.nlp.modules.common import (
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/modules/common/init.py", line 18, in
from nemo.collections.nlp.modules.common.huggingface import (
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/modules/common/huggingface/init.py", line 19, in
from nemo.collections.nlp.modules.common.huggingface.huggingface_utils import (
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/modules/common/huggingface/huggingface_utils.py", line 18, in
from transformers import (
ImportError: cannot import name 'ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/init.py)

I saw similar issue there : #9272 But, was not clear where this is fixed. Please help @titu1994 . Thanks

[yonas-g]

Please check #9272

[uni-manjunath-ke]

I tried using : python3 -m pip install git+https://github.com/NVIDIA/NeMo.git#egg=nemo_toolkit[all]

After this, I am getting different errors:

Fused optimized group norm has not been installed.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/nemo/collections/nlp/models/language_modeling/megatron/gpt_layer_modelopt_spec.py", line 16, in
from megatron.core.fusions.fused_bias_dropout import get_bias_dropout_add
ModuleNotFoundError: No module named 'megatron'

After this, I did pip install megatron, which gave another error:
ModuleNotFoundError: No module named 'transformer_engine'

Then I tried installing transformer-engine using
pip install git+https://github.com/NVIDIA/TransformerEngine.git@stable
, but engine crashed. Please suggest. Thanks

[uni-manjunath-ke]

Hi @yonas-g @titu1994
I am still facing this issue. Any suggestion. Thanks

[titu1994]

Transformer engine is not that easy to install, please look at installation instructions in NeMo readme or use container.

[raybellwaves]

Please check #9272

I checked and it using https://github.com/NVIDIA/NeMo/blob/main/requirements/requirements_lightning.txt#L7 it downgraded my transformers to 4.40.2 which fixed this problem as it pulled in https://github.com/huggingface/transformers/blob/v4.40.2/src/transformers/__init__.py#L1456