Package Usage: npm: gpt-tokenizer
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
21 versions
Latest release: over 2 years ago
36 dependent packages
472,234 downloads last month
View more package details: https://packages.ecosystem.code.gouv.fr/registries/npmjs.org/packages/gpt-tokenizer