[v.2.4.0] Release Tracker

[atalman]

We cut a release branch for the 2.4.0 release.

Our plan from this point from this point is roughly:

• Phase 1 (until 7/1/24): work on finalizing the release branch
• Phase 2 (after 7/1/24): perform extended integration/stability/performance testing based on Release Candidate builds.

This issue is for tracking cherry-picks to the release branch.

Cherry-Pick Criteria

Phase 1 (until 7/1/24):

Only low-risk changes may be cherry-picked from main:

1. Fixes to regressions against the most recent minor release (e.g. 2.3.x for this release; see module: regression issue list)
2. Critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks
3. Critical fixes to new features introduced in the most recent minor release (e.g. 2.3.x for this release)
4. Test/CI fixes
5. Documentation improvements
6. Compilation fixes or ifdefs required for different versions of the compilers or third-party libraries
7. Release branch specific changes (e.g. change version identifiers)

Any other change requires special dispensation from the release managers (currently @atalman, @PaliC , @huydhn, @malfet). If this applies to your change please write "Special Dispensation" in the "Criteria Category:" template below and explain.

Phase 2 (after 7/1/24):

Note that changes here require us to rebuild a Release Candidate and restart extended testing (likely delaying the release). Therefore, the only accepted changes are Release-blocking critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks

Changes will likely require a discussion with the larger release team over VC or Slack.

Cherry-Pick Process

1. Ensure your PR has landed in master. This does not apply for release-branch specific changes (see Phase 1 criteria).

2. Create (but do not land) a PR against the release branch.

   # Find the hash of the commit you want to cherry pick
   # (for example, abcdef12345)
   git log
   
   git fetch origin release/2.4
   git checkout release/2.4
   git cherry-pick -x abcdef12345
   
   # Submit a PR based against 'release/2.4' either:
   # via the GitHub UI
   git push my-fork
   
   # via the GitHub CLI
   gh pr create --base release/2.4

3. Make a request below with the following format:

Link to landed trunk PR (if applicable):
* 

Link to release branch PR:
* 

Criteria Category:
* 

1. Someone from the release team will reply with approved / denied or ask for more information.
2. If approved, someone from the release team will merge your PR once the tests pass. Do not land the release branch PR yourself.

NOTE: Our normal tools (ghstack / ghimport, etc.) do not work on the release branch.

Please note HUD Link with branch CI status and link to the HUD to be provided here.
HUD

Versions

2.4.0

[atalman]

Link to landed trunk PR (if applicable):

• NA

Link to release branch PR:

• [Release 2.4] Release only changes #128347
• [Release 2.4] Release only changes - use pinned triton. #128388
• [Release 2.4] Temp change to build triton from pin builder#1863

Criteria Category:

• Release only changes, temp changes to build triton from pin rather then branch

---

@atalman merged

[atalman]

Link to landed trunk PR (if applicable):

• Add AOTriton install step for ROCm manylinux images builder#1862

Link to release branch PR:

• Add AOTriton install step for ROCm manylinux images (#1862) builder#1867

Criteria Category:

• Required to fix rocm nightly build error AOTriton Cmake error breaking PyTorch nightly binary builds for ROCm #128420

---

@atalman merged

[atalman]

Link to landed trunk PR (if applicable):

• [dynamo] Fix for #127696 #128358

Link to release branch PR:

• [dynamo] Fix for #127696 #128530

Criteria Category:

• Critical fix, forward fix for Fix 'get_real_value' on placeholder nodes #127698 on release branch

---

@atalman merged

[atalman]

Link to landed trunk PR (if applicable):

• Revert of [cuDNN][SDPA] Remove TORCH_CUDNN_SDPA_ENABLED=1, enable cuDNN SDPA by default on H100 and 2nd on other archs >= sm80 #125343

Link to release branch PR:

• Revert "[cuDNN][SDPA] Remove TORCH_CUDNN_SDPA_ENABLED=1, enable cuDNN SDPA by default on H100 and 2nd on other archs >= sm80 (#125343)" #128539

Criteria Category:

• Reverted on main

---

@malfet merged

[atalman]

Link to landed trunk PR (if applicable):

• Revert of Set simdlen based on ATEN_CPU_CAPABILITY #123514

Link to release branch PR:

• Revert "Set simdlen based on ATEN_CPU_CAPABILITY (#123514)" #128541

Criteria Category:

• Reverted on main

---

@malfet merged

[atalman]

Link to landed trunk PR (if applicable):

• Revert of Deprecate torch._utils.is_compiling() and torch._dynamo.external_utils.is_compiling() #127690

Link to release branch PR:

• Revert "Deprecate torch._utils.is_compiling() and torch._dynamo.external_utils.is_compiling() (#127690)" #128542

Criteria Category:

• Reverted on main

---

@malfet merged

[zhuhaozhe]

Link to landed trunk PR (if applicable):

• [inductor] fix linear add bias pattern #128473

Link to release branch PR:

• [inductor] fix linear add bias pattern (#128473) #128577

Criteria Category:

• Critical fix, forward fix for Assertion error when running timm_models "levit_128" #128287 on release branch. It's a regression on inductor, will crash on many benchmark models without this fix.

---

@atalman merged

[zou3519]

Link to landed trunk PR (if applicable):

• Change Dynamo's custom ops warning message to be less spammy #128456

Link to release branch PR:

• Change Dynamo's custom ops warning message to be less spammy (#128456) #128581

Criteria Category:

• (1) Fixes to regressions. In 2.4, we started spamming warnings if someone used pybind'ed functions with torch.compile. There were no such warnings in 2.3. This PR adjusts the warnings to be less spammy.

---

@atalman merged

[etaf]

Link to landed trunk PR (if applicable):

• Clean up xpu ut to make CI happy #128383

Link to release branch PR:

• Clean up xpu ut to make CI happy (#128383) #128614

Criteria Category:

• (4) Test/CI fixes. Fix UT and unblock the xpu CI: before [CI] disable td for xpu ci test by default #127611 merged, the xpu-specific UT test/test_xpu.py was skipped temporarily because the test cases were disabled by TD. This PR aims to fix the UT bug introduced by Switch to torch.float16 on XPU AMP mode #127741 .

---

@atalman merged

[zou3519]

Link to landed trunk PR (if applicable):

• [custom_op] stop using nonlocals to store information #128547

Link to release branch PR:

• [custom_op] stop using nonlocals to store information (#128547) #128616

Criteria Category:

• (3) Critical fixes to new features. In 2.4 we are releasing a new torch.library.custom_op API. This PR fixes a critical bug that the API did not compose with FSDP and other distributed APIs.

---

@atalman merged

[chunyuan-w]

Link to landed trunk PR (if applicable):

• Revert of Make torch_geometric models compatible with export #123403 (the revert is done in Revert "Make torch_geometric models compatible with export (#123403)" #128377)

Link to release branch PR:

• Revert "Make torch_geometric models compatible with export (#123403)"… #128511

Criteria Category:

• Reverted on main to fix a critical regression issue ([AOTI] Conv-BN folding on CPU not working anymore after benchmark script change in https://github.com/pytorch/pytorch/pull/123403 #127513) that will impact the AOTI CPU performance on multiple CNN models in the dynamo benchmark.

---

@atalman merged

[etaf]

Link to landed trunk PR (if applicable):

• [Inductor] Update Intel GPU Triton commit pin. #124842

Link to release branch PR:

• [Inductor] Update Intel GPU Triton commit pin. (#124842) #128615

Criteria Category:

• Critical fix, forward fix for Test case test_torchinductor.py:test_nll_loss_backward_xpu failed on Intel GPU #126173. Also update Intel triton for release 2.4.

---

@atalman merged

[wanchaol]

Link to landed trunk PR (if applicable):

• [inductor] fix compile time regression by caching get_gpu_type #128363

Link to release branch PR:

• [inductor] fix compile time regression by caching get_gpu_type (#128363) #128717

Criteria Category:

• Critical fixes for compile time regression

---

@atalman merged

[wanchaol]

Link to landed trunk PR (if applicable):

• [tp] refactor and fix PrepareModuleInput for DTensor inputs #128431

Link to release branch PR:

• [tp] refactor and fix PrepareModuleInput for DTensor inputs (#128431) #128719

Criteria Category:

• Critical fixes for silent correctness

---

@atalman merged

[Xia-Weiwen]

Link to landed trunk PR (if applicable):

• [Quant][Inductor] Bug fix: mutation nodes not handled correctly for QLinearPointwiseBinaryPT2E #127592

Link to release branch PR:

• [Port][Quant][Inductor] Bug fix: mutation nodes not handled correctly for QLinearPointwiseBinaryPT2E #128591

Criteria Category:

• Critical fix for [inductor][cpu]abnormal performance improvement and accuracy drop for huggingface suit static/dynamic quantization #127402. Without the fix, transformer-based models will fail to run with PT2E quantization on X86Inductor backend.

---

@atalman merged

[xuhancn]

Link to landed trunk PR (if applicable):

• [cpp_extension][inductor] Fix sleef windows depends. #128770

Link to release branch PR:

• [cpp_extension][inductor] Fix sleef windows depends. (#128770) #128811

Criteria Category:

• Critical fixes for PyTorch Windows cpp_extension. If user use cpp_extension to build extension and its code contains VEC, it will occur sleef dependency issue.

---

Hi @xuhancn this looks like feature work to enable inductor on PyTorch Windows, however for release 2.4 we don't support this yet.
Do you have any tests with it ?

[zou3519]

Link to landed trunk PR (if applicable):

• [Inductor] Fix the High Order Op layout issue #128275

Link to release branch PR:

• [Inductor] Fix the High Order Op layout issue (#128275) #128834

Criteria Category:

• (1) Regression (from 2.3.x) and (2) Critical fixes for: silent incorrectness

---

@atalman merged

[lw]

Link to landed trunk PR (if applicable):

• Remove leftover warning causing log spew #128688

Link to release branch PR:

• Remove leftover warning causing log spew #128837

Criteria Category:

• Fixes to regressions against the most recent minor release (changes only logging, low risk)

---

@atalman merged

[clee2000]

Link to landed trunk PR (if applicable):

• Switch to ${{ runner.temp }} test-infra#5327

Link to release branch PR:

• Switch to ${{ runner.temp }} (#5327) test-infra#5348

Criteria Category:

• Release - needed to fix executorch release CI

---

@clee2000 merged

[pytorchbot]

Link to landed trunk PR (if applicable):

• [inductor][ci] Fix torchbench dependency issue with numpy #128968

Link to release branch PR:

• [inductor][ci] Fix torchbench dependency issue with numpy #129074

Criteria Category:
CI signal fix for torchbench

---

@atalman merged

[xuhancn]

Link to landed trunk PR (if applicable):

• [cpp_extension][inductor] Fix sleef windows depends. #128770

Link to release branch PR:

• [cpp_extension][inductor] Fix sleef windows depends. (#128770) #128811

Criteria Category:

• Critical fixes for PyTorch Windows cpp_extension. If user use cpp_extension to build extension and its code contains VEC, it will occur sleef dependency issue.

Hi @xuhancn this looks like feature work to enable inductor on PyTorch Windows, however for release 2.4 we don't support this yet. Do you have any tests with it ?

pytorch 2.4 not support Windows Inductor, but pytorch 2.4 enabled Windows x86 CPU vectorization(#118980). It will add sleef as dependency on cpp_extension. So, We need to cherry-pick this PR.

[atalman]

Link to landed trunk PR (if applicable):

• NA

Link to release branch PR:

• Revert "[Release 2.4] Release only changes - use pinned triton." #129139
• Revert "[Release 2.4] Temp change to build triton from pin" builder#1877
• [Release 2.4] Release only changes for triton 3.0.x build #129143
• [Release only] Temporary change to depend on pytorch-triton #129232

Criteria Category:

• Release only changes: Triton branch is now available here: https://github.com/triton-lang/triton/tree/release/3.0.x hence we don't need these temporary PRs anymore

---

@atalman merged

[pytorchbot]

Link to landed trunk PR (if applicable):

• Re-enable py3.12 nightly wheel builds and add triton dependency for ROCm  #128525

Link to release branch PR:

• Re-enable py3.12 nightly wheel builds and add triton dependency for ROCm  #129161

Criteria Category:
Critical - Build py3.12 and add Triton dependency for ROCm

---

@atalman merged

[atalman]

Link to landed trunk PR (if applicable):

• N/A (will be added later)

Link to release branch PR:

• [ROCm] [Triton] - Include roctracer headers in triton whl #129227
• [ROCm] Include hsa headers for rocm-triton whl #129235

Criteria Category:

• Release only change to fix triton rocm compilation. This change will be added to trunk later with moving of rocm pin in main

---

@atalman merged

[zou3519]

Link to landed trunk PR (if applicable):

• [docs] Redirect custom ops landing page to the correct place #129177

Link to release branch PR:

• [docs] Redirect custom ops landing page to the correct place (#129177) #129236

Criteria Category:

• Documentation improvements

---

@huydhn merged

[zou3519]

Link to landed trunk PR (if applicable):

• [custom ops] Switch out references from old landing page to new landing page #129178

Link to release branch PR:

• [custom ops] Switch out references from old landing page to new landi… #129237

Criteria Category:

• Documentation improvements -- we moved the custom ops docs and need to update the error messages.

---

@huydhn merged

[fegin]

Link to landed trunk PR (if applicable):

• [DSD] Correctly handle shared parameters for optimizer state_dict #128685

Link to release branch PR:

• [DSD] Correctly handle shared parameters for optimizer state_dict (#1… #129252

Criteria Category:

• Critical fixes to crashes for loading distributed optimizer state_dict with shared parameters. The bug is introduced in 2.4.

[fegin]

Link to landed trunk PR (if applicable):

• Support HSDP + Monolith Checkpointing #128446
• [DSD] Add unittest to verify HSDP1 + broadcast_from_rank0 #128755 (test)

Link to release branch PR:

• Support HSDP + Monolith Checkpointing (#128446) #129254
• [DSD] Add unittest to verify HSDP1 + broadcast_from_rank0 (#128755) #129255 (test)

Criteria Category:

• Critical fixes to crashes when loading HSDP1 + full state_dict with rank0 broadcasting feature.

[jingxu10]

Link to landed trunk PR (if applicable):

• Adding a note for Getting Started with PyTorch on Intel GPUs #127872
• add xpu to torch.tensors #127280
• add xpu for amp #127276
• update amp example to device-agnostic #127278
• add xpu to torch.compile #127279

Link to release branch PR:

• Documentations for XPU functionality to PyTorch #129266

Criteria Category:

Documentation improvements - Add documentation for XPU support to PyTorch

[atalman]

Link to landed trunk PR (if applicable):

• Add ability to switch channels during conda build builder#1880

Link to release branch PR:

• Cherry-Pick of Add ability to switch channels during conda build #1880 builder#1881

Criteria Category:

• Release conda build required change

---

@atalman merged

[etaf]

Link to landed trunk PR (if applicable):

• [Inductor][Intel GPU] Support reduction split. #129120

Link to release branch PR:

• [Inductor][Intel GPU] Support reduction split. (#129120) #129337

Criteria Category:

• Important Intel GPU Inductor performance improvement for release 2.4.

[xuhancn]

Link to landed trunk PR (if applicable):

• [cpp_extension][inductor] Fix sleef windows depends. #128770

Link to release branch PR:

• [cpp_extension][inductor] Fix sleef windows depends. (#128770) #128811

Criteria Category:

• Critical fixes for PyTorch Windows cpp_extension. If user use cpp_extension to build extension and its code contains VEC, it will occur sleef dependency issue.

Hi @xuhancn this looks like feature work to enable inductor on PyTorch Windows, however for release 2.4 we don't support this yet. Do you have any tests with it ?

Hi @malfet , @atalman , I have added some test case to proof it is a cpp_extension bug fixing: #128811 (comment) . Please don't forget to merge this PR.

[mikaylagawarecki]

Link to landed trunk PR (if applicable):

• Fix allowlisting of builtins for weights_only unpickler #129244
• Allow BUILD/NEWOBJ instruction for items added via torch.serialization.add_safe_globals #129251
• Add warning for weights_only #129239
• Add example for torch.serialization.add_safe_globals #129396

Link to release branch PR:

• Cherry pick #129244, #129251, #129239, 129396 into release/2.4 #129478

Criteria Category:

• 129244 and 129251 are critical fixes to torch.serialization.add_safe_globals added in 2.4 that caused the API to not work with weights_only for a large proportion of cases
• 129239 adds a warning, 129396 is a documentation improvement

[pytorchbot]

Link to landed trunk PR (if applicable):

• TunableOp hotfix #129281

Link to release branch PR:

• TunableOp hotfix #129499

Criteria Category:
Critical -

There is no associated issue. This was raised on slack from @mxz297 due to memory access faults using TunableOp.