[huggingface_pytorch] Inference - update for HuggingFace Transformers to 4.41.2 - PyTorch 2.2

[JingyaHuang]

GitHub Issue #3871 :

Description

This PR updates Hugginface's PyTorch DLC for inference. Here are the corresponding updated dependencies versions:

• transformers: 4.41.2
• torch: 2.2.0
• diffusers: 0.28.2
• accelerate: 0.31.0
• peft: 0.11.1

Tests run

NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"

• I have run builds/tests on commit for my changes.

NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:

Expand

• sagemaker_remote_tests = true
• sagemaker_efa_tests = true
• sagemaker_rc_tests = true

Additionally, please run the sagemaker local tests in at least one revision:

• sagemaker_local_tests = true

Formatting

• I have run black -l 100 on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)

DLC image/dockerfile

Builds to Execute

Expand

Click the checkbox to enable a build to execute upon merge.

Note: By default, pipelines are set to "latest". Replace with major.minor framework version if you do not want "latest".

• build_pytorch_training_latest
• build_pytorch_inference_latest
• build_tensorflow_training_latest
• build_tensorflow_inference_latest

Additional context

PR Checklist

Expand

• I've prepended PR tag with frameworks/job this applies to : [mxnet, tensorflow, pytorch] | [ei/neuron/graviton] | [build] | [test] | [benchmark] | [ec2, ecs, eks, sagemaker]
• If the PR changes affects SM test, I've modified dlc_developer_config.toml in my PR branch by setting sagemaker_tests = true and efa_tests = true
• If this PR changes existing code, the change fully backward compatible with pre-existing code. (Non backward-compatible changes need special approval.)
• (If applicable) I've documented below the DLC image/dockerfile this relates to
• (If applicable) I've documented below the tests I've run on the DLC image
• (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list. See https://www.apache.org/legal/resolved.html.
• (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them.

NEURON/GRAVITON Testing Checklist

• When creating a PR:

• I've modified dlc_developer_config.toml in my PR branch by setting neuron_mode = true or graviton_mode = true

Benchmark Testing Checklist

• When creating a PR:

• I've modified dlc_developer_config.toml in my PR branch by setting ec2_benchmark_tests = true or sagemaker_benchmark_tests = true

Pytest Marker Checklist

Expand

• (If applicable) I have added the marker @pytest.mark.model("<model-type>") to the new tests which I have added, to specify the Deep Learning model that is used in the test (use "N/A" if the test doesn't use a model)
• (If applicable) I have added the marker @pytest.mark.integration("<feature-being-tested>") to the new tests which I have added, to specify the feature that will be tested
• (If applicable) I have added the marker @pytest.mark.multinode(<integer-num-nodes>) to the new tests which I have added, to specify the number of nodes used on a multi-node test
• (If applicable) I have added the marker @pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">) to the new tests which I have added, if a test is specifically applicable to only one processor type

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

This PR has been marked stale as a result of being open for 30 days without activity or updates. Please remove the stale label or comment in order to keep this open, otherwise the PR will be closed in 5 days.

[asai-carbon]

Any updates on this? I would like to use the new transformers version ASAP.

[JingyaHuang]

Hi @asai-carbon, I pinged sagemaker team for review, let's see how it goes. Sorry about the delay.

[asai-carbon]

@JingyaHuang I tried building the inference image on my M1 Mac with Transformers 4.40.1. I had some questions about the process outlined in the repo instructions.

1. However, I am getting stuck on the OpenMPI installation step specified in the Dockerfile.gpu, and each time its a different error for that step.

The command I ran was the following (after copying the files in this PR related to the new HuggingFace Pytorch inference image DockerFile):

REGION=<ENTER REGION HERE> ACCOUNT_ID=<ENTER ACCOUNT ID HERE> REPOSITORY_NAME=<ENTER REPOSITORY NAME HERE> python3 src/main.py --buildspec huggingface/pytorch/inference/buildspec.yml --framework huggingface_pytorch

Examples:

configure: error: in `/openmpi-4.1.5':
configure: error: cannot compute sizeof (float _Complex)
See `config.log' for more details

configure: WARNING: *** Problem running configure test!
configure: WARNING: *** See config.log for details.
configure: error: *** Cannot continue.

configure: error: in `/openmpi-4.1.5/opal/mca/pmix/pmix3x/pmix':
configure: error: cannot compute sizeof (ptrdiff_t)
See `config.log' for more details
configure: /bin/bash './configure' *failed* for opal/mca/pmix/pmix3x/pmix
checking if v3.x component is to be used... 
yes - using the internal v3.x library
configure: WARNING: INTERNAL PMIX FAILED TO CONFIGURE
configure: error: CANNOT CONTINUE

../../libtool: line 1766: 146647 Segmentation fault      (core dumped) gcc -DHAVE_CONFIG_H -I. -I../../opal/include -I../../ompi/include -I../../oshmem/include -I../../opal/mca/hwloc/hwloc201/hwloc/include/private/autogen -I../../opal/mca/hwloc/hwloc201/hwloc/include/hwloc/autogen -I../../ompi/mpiext/cuda/c -I../.. -I../../orte/include -I/openmpi-4.1.5/opal/mca/event/libevent2022/libevent -I/openmpi-4.1.5/opal/mca/event/libevent2022/libevent/include -I/openmpi-4.1.5/opal/mca/hwloc/hwloc201/hwloc/include -I/usr/local/include -I/usr/local/include -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -mcx16 -MT proc.lo -MD -MP -MF .deps/proc.Tpo -c proc.c -fPIC -DPIC -o .libs/proc.o

Do you know what would cause this and how to resolve?

2. Also , do you know how I could reproduce config.log that is generated from the Dockerfile so I can view locally? I don't see it generated anywhere within the repo when I try to build the inference image.
3. Would the first 2 images outlined in the large model inference section of the available images be adequate substitutes for this inference image?