Tarry Singh 959bc48c42 Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.		2024-02-28 15:47:37 +00:00
..
gguf	Add stdx utilities and rework async signature inference; tidy executable logging.	2023-06-21 14:45:14 +00:00
torch	Add stdx utilities and rework async signature inference; tidy executable logging.	2023-06-21 14:45:14 +00:00
gguf.zig	Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.	2024-02-28 15:47:37 +00:00
json.zig	Rename zml.aio.Value to zml.aio.Metadata, simplify its type variants, and update torch pickle/eval APIs accordingly.	2023-04-07 16:45:58 +00:00
nemo.zig	Add stdx utilities and rework async signature inference; tidy executable logging.	2023-06-21 14:45:14 +00:00
safetensors.zig	Add stdx utilities and rework async signature inference; tidy executable logging.	2023-06-21 14:45:14 +00:00
sentencepiece.zig	Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.	2024-02-28 15:47:37 +00:00
tinyllama.zig	Various minor fixes: rewrite tinyllama tokenizer newline token, prevent HostBuffer.isContiguous false trigger on 1‑dim axes, improve HostBuffer.slice1d error messages, simplify module.zig output to show .mlir file path, correct setFlags handling of comptime int/float, make tokenizer.zig return <oob> for out‑of‑range detokenization, and speed up Buffer.constant creation up to 2.5 GB/s on CUDA.	2024-02-19 12:34:18 +00:00
torch.zig	Add stdx utilities and rework async signature inference; tidy executable logging.	2023-06-21 14:45:14 +00:00
yaml.zig	Rename zml.aio.Value to zml.aio.Metadata, simplify its type variants, and update torch pickle/eval APIs accordingly.	2023-04-07 16:45:58 +00:00