|
|
30f6be0e2f
|
Update core Zig modules (async, mlir, pjrt, stdx) and third‑party Bazel definitions for the Zig 0.14.0 release.
|
2024-07-02 14:19:04 +00:00 |
|
|
|
959bc48c42
|
Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.
|
2024-02-28 15:47:37 +00:00 |
|
|
|
9b7eea8ac2
|
Add stdx utilities and rework async signature inference; tidy executable logging.
|
2023-06-21 14:45:14 +00:00 |
|
|
|
0189b71070
|
Rename zml.aio.Value to zml.aio.Metadata, simplify its type variants, and update torch pickle/eval APIs accordingly.
|
2023-04-07 16:45:58 +00:00 |
|
|
|
05d23beb23
|
Add Normalizer.fromHfJson to read HuggingFace tokenizer JSON and map to internal options, including a configurable magic space token and a debug flag for token merges. Adjust default handling of extra whitespaces to align with HF defaults.
|
2023-03-29 16:10:29 +00:00 |
|
|
|
266da6d4be
|
Add initial Bazel build configuration, async runtime implementation, and core MLIR dialect definitions for ZML.
|
2023-01-02 14:28:25 +00:00 |
|
