Home Explore Project
Register Sign In
python
/
tiktoken
3
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: ef915bb425
Branches Tags
tiktoken
/
README.md

README.md 105 B
History Raw

tiktoken

openai模型BPE标记

Usage

pip install tiktoken

import tiktoken
import torch