Tarry Singh e789e26008 Remove examples workspace and clean up related Bazel BUILD/MODULE files and Zig build scripts.		2025-06-19 09:30:29 +00:00
..
BUILD.bazel	Remove examples workspace and clean up related Bazel BUILD/MODULE files and Zig build scripts.	2025-06-19 09:30:29 +00:00
Cargo.lock	Bump hftokenizers dependency versions in Bazel and Cargo lockfiles (MODULE.bazel.lock, Cargo.toml, Cargo.lock)	2025-04-04 12:54:33 +00:00
Cargo.toml	Bump hftokenizers dependency versions in Bazel and Cargo lockfiles (MODULE.bazel.lock, Cargo.toml, Cargo.lock)	2025-04-04 12:54:33 +00:00
hftokenizers.h	Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.	2024-02-28 15:47:37 +00:00
hftokenizers.rs	Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.	2024-02-28 15:47:37 +00:00
hftokenizers.zig	Integrate TinyLlama support, restore the homemade tokenizer, and align Zig API naming across stdx and zml tokenizer modules.	2024-04-05 15:07:29 +00:00