Skip to content

Pull requests: Dao-AILab/flash-attention

New pull request New
41 Open 118 Closed
41 Open 118 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Support AMD ROCm on FlashAttention 2
#1010 opened Jun 26, 2024 by rocking5566 Loading…
6
Add return_softmax_lse in flash_attn_with_kvcache
#989 opened Jun 13, 2024 by ovowei Loading…
remove swizzle part of sV.data() to get a completely non-swizzle sVtNoSwizzle
#984 opened Jun 11, 2024 by soundOfDestiny Loading…
[Draft] support qk head_dim different from vo head_dim
#980 opened Jun 6, 2024 by defei-coder Loading…
1
Fix +/-inf in LSE returned by forward
#978 opened Jun 3, 2024 by sgrigory Loading…
1
Fixing argument checking when using seqlenq_ngroups_swapped.
#976 opened Jun 3, 2024 by Narsil Loading…
1
fix typo
#974 opened May 31, 2024 by jslhcl Loading…
add pyproject.toml with build dependencies
#958 opened May 17, 2024 by dhellmann Loading…
5
Relative position encoding
#956 opened May 14, 2024 by b-albar Loading…
1 of 4 tasks
Add softmax_d in mha_bwd
#905 opened Apr 1, 2024 by MayDomine Loading…
Fix KeyError handling for non-existing key in state_dict.pop()
#898 opened Mar 23, 2024 by JiedaokouWangguan Loading…
ALiBi for the non-flash code path
#858 opened Feb 29, 2024 by Markus28 Loading…
Add local version identifier to package metadata for pre-built wheels
#856 opened Feb 28, 2024 by yundai424 Loading…
1
1
Fix typos of statement about shape.
#837 opened Feb 19, 2024 by 66RING Loading…
Add support for small page sizes
#824 opened Feb 13, 2024 by skrider Loading…
22
Add C++ build support for use with LibTorch
#819 opened Feb 9, 2024 by shaltielshmid Loading…
1
meta tensor stuff
#769 opened Jan 15, 2024 by tsengalb99 Loading…
Animations for Flash Attention, Flash Attention2, and Standard Attention
#736 opened Dec 24, 2023 by LuisAVasquez Loading…
1
Jetson (aarch64) support
#724 opened Dec 14, 2023 by jasl Loading…
10
feat(attention): add Bi-Directional MLM attention model
#721 opened Dec 12, 2023 by TamirFriedman-RecoLabs • Draft
1
Update utils.py
#710 opened Dec 8, 2023 by adarshxs Loading…
2
[fix bug] Llama-2-70B crashed when prompt_len < ngroups
#708 opened Dec 7, 2023 by li2haipeng Loading…
Add flash_attn_varlen_func_with_kvcache.
#685 opened Nov 22, 2023 by garrett4wade Loading…
3
Custom attention bias
#617 opened Oct 19, 2023 by b-albar Loading…
2 of 5 tasks
29
Setup: Add extra compute targets
#605 opened Oct 15, 2023 by bdashore3 Loading…
1
ProTip! Exclude everything labeled bug with -label:bug.