Radix/zml/tokenizer/sentencepiece
2025-09-26 13:38:11 +00:00
..
BUILD.bazel Revamp gather API with named indices (and add gather_ variant), improve topK handling, and add Yarn rope embedding support across core modules (buffer, nn, pjrtx, quantization, shape, tensor, testing, tokenizer, torch). 2025-09-26 13:38:11 +00:00
sentencepiece.i Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers. 2024-02-28 15:47:37 +00:00
sentencepiece.zig Add Zig 0.15 compatibility: update BUILD files, async primitives, stdx utilities, MLIR dialects, and PJRT FFI. 2025-07-28 13:54:28 +00:00