New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add AttentionContext #730

Open

HIT-cwh wants to merge 2 commits into InternLM:main from HIT-cwh:attn_context

Collaborator

HIT-cwh commented May 29, 2024

Usage

from xtuner.utils import AttentionContext

seqlen_list = [1, 3]
AttentionContext.update(seqlen_list)
print(AttentionContext.get_cumulative_len())  # [0, 1, 4]
print(AttentionContext.get_max_seqlen())  # 3
AttentionContext.clear()
print(AttentionContext.get_cumulative_len() is None)  # True
print(AttentionContext.get_max_seqlen() is None)  # True

HIT-cwh added 2 commits

May 29, 2024 18:19


          add AttentionContext

468f432


          refactor AttentionContext

2ec31ed

HIT-cwh requested a review from pppppM

May 30, 2024 02:10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment