@cyberlangke/tokkit-arcee-ai
v1.11.0
Published
Arcee AI tokenizer families for tokkit.
Readme
@cyberlangke/tokkit-arcee-ai
Arcee AI 官方文本 tokenizer 的 tokkit 子包。
当前内置 family:
trinity-large-truebase- 覆盖
arcee-ai/Trinity-Large-TrueBase
- 覆盖
trinity-large- 覆盖
arcee-ai/Trinity-Large-Base - 也聚合当前确认完全复用同一 tokenizer 的
arcee-ai/Trinity-Large-Preview
- 覆盖
trinity-large-thinking- 覆盖
arcee-ai/Trinity-Large-Thinking
- 覆盖
当前不纳入:
arcee-ai/Arcee-Novaarcee-ai/Arcee-Sparkarcee-ai/Arcee-Agent
说明:
Trinity-Large-Base与Trinity-Large-Preview的tokenizer.jsonSHA256 一致,因此收口到同一个trinity-largefamily。Trinity-Large-TrueBase与Trinity-Large-Thinking都各自保持独立 family。Arcee-Nova当前许可证信号不是 MIT / Apache-2.0 兼容边界。Arcee-Spark、Arcee-Agent当前视为专项模型线,不作为这一轮公开支持目标。
使用方法
npm install @cyberlangke/tokkit-arcee-aiimport { getTokenizer } from "@cyberlangke/tokkit-arcee-ai"
const base = await getTokenizer("trinity-large")
const preview = await getTokenizer("arcee-ai/Trinity-Large-Preview")
console.log(base === preview)