b67a3f9a0a
workspace: switch to libxev from Bazel Central Registry, completing migration away from custom registry
2025-10-29 17:43:22 +00:00
bcd43314a4
Migrate BUILD and MODULE files to upstreamed rules_zig (rename copts to zigopts) and adjust ZLS integration accordingly.
2025-10-13 15:26:42 +00:00
488a844a0f
Add Zig 0.15 compatibility: update BUILD files, async primitives, stdx utilities, MLIR dialects, and PJRT FFI.
2025-07-28 13:54:28 +00:00
1cf26756a1
workspace: run buildifier, drop rules_uv, refactor tools/hf dependencies
2025-07-16 10:01:41 +00:00
1427286716
runtimes/neuron: fix neuron runtime
...
This PR fixes the neuron runtime with the following:
Proxy the PJRT Api method to enforce the client struct sizes since the
neuron PJRT plugin doesn't use `>=` but `==` to assert them, breaking
PJRT compatibility guarantees.
Fixes https://github.com/aws-neuron/aws-neuron-sdk/issues/1095
Reimplement `libneuronxla` in Zig to control neuronx-cc sandboxing and
invocation.
Implement a python bootstrapper in Zig to create a full blown
`neuronx-cc` executable, avoiding the infamous chicken and egg problem
of python executables boostrapping when sandboxed (due to fixed path
shebangs).
---------
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-07-15 15:26:03 +00:00
cf00506dbb
Switch workspace build rules from zig_cc_binary to zig_binary, removing the hack and using the C linker directly.
2025-07-03 15:10:36 +00:00
78679817df
zml: remove nemo, sentencepiece loaders, remove zig-yaml
...
Progress towards
2025-06-24 15:39:55 +00:00
fe56f03f5d
third_party/xla: de-modularize xla
...
This PR intends to remove XLA as a bzlmod and transfer it as a non
bzlmod dep.
This is because this module will never be upstreamed as is, so keep it
private.
Also, we fetch llvm-raw and stablehlo from it, which is fine.
While there, dummify the various local_config XLA symbols to please the
imports, as we don't use those parts in ZML itself.
Closes
2025-06-23 09:13:43 +00:00
e789e26008
Remove examples workspace and clean up related Bazel BUILD/MODULE files and Zig build scripts.
2025-06-19 09:30:29 +00:00
1b58c34b8c
Update ZML build configuration to replace zig-protobuf with upb library.
2025-06-09 16:34:20 +00:00
f9280b1069
Add Bazel MODULE and tool build configuration (BUILD, requirements) for packaging the HF CLI.
2025-05-30 16:42:57 +00:00
1a2b862ec2
Add sandbox neuron dependencies: define a trampoline PJRT, create an empty repository for distroless deps, and update Bazel build files and Zig/C sources accordingly.
2025-05-19 17:35:33 +00:00
55c5b540f8
Add XLA 20250718.0‑6319f0d with ROCm 6.4.1 support, update Bazel module files and runtime configs, and apply migration, FFI‑handler and header‑cleanup patches.
2025-05-12 12:10:27 +00:00
cba9ce9615
protobuf, libxev: remove usingnamespace
...
remove usingnamespace from libxev and zig-protobuf
relevant PRs:
* libxev:
7f803181b1https://github.com/zml/zig-protobuf/pull/1/files
This means we can now build ZML with `-fincremental` (in debug mode only
and linux x86-64)
---------
Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2025-05-06 11:58:20 +00:00
47146fc088
workspace: remove glibc version constraint, update rules_rust, and re-enable frame pointers following sandboxing changes.
2025-04-29 12:09:13 +00:00
f5f3dc6b1b
Update rules_zig to 20250714.0-b14a4f1 and fix static‑lib inclusion in runfiles for non‑exe targets.
2025-04-18 12:09:38 +00:00
7d9fdf94e7
runtimes/rocm: sandbox ROCm dependencies and ensure they load on the main thread due to TLS usage in static C++ destructors.
2025-04-14 16:38:15 +00:00
2d321d232d
runtimes/cuda: sandbox CUDA dependencies by removing them from the leaf binary, sandboxing the dependency graph, marking dlopen direct dependencies as NEEDED, setting RPATH to the sandbox, loading the PJRT plugin from the sandbox, and enabling weak CUDA symbols without direct linking.
2025-03-26 11:18:29 +00:00
907577525f
Update MODULE.bazel and lockfile to bump toolchains_llvm_bootstrapped to version 0.2.4.
2025-03-18 11:47:22 +00:00
f27a524f31
Update rules_zig: add zig_srcs target, fix source handling bug, clean up BUILD files, adjust async/coro.zig tests, and disable nemo and yaml model loaders.
2025-03-13 12:27:21 +00:00
9488672d4b
workspace: bump xla to version 20250710.0-22ea002
...
Also:
- Bump XLA deps : `com_github_grpc_grpc` and `com_google_protobuf`
- Inject `rules_ml_toolchain`
- Fix `zig_proto_library` rule
2025-03-04 17:12:34 +00:00
1cafcc3c60
Workspace: bump XLA to newer version.
2025-02-05 17:35:27 +00:00
5464281c91
Update workspace configuration to use the toolchains_llvm_bootstrapped toolchain for Zig builds.
2024-12-03 13:50:58 +00:00
95453c7242
Update XLA dependency to version 20250527.0‑cb67f2f and refresh related Bazel BUILD, MODULE, overlay and patch files.
2024-11-22 16:50:20 +00:00
fa13287931
workspace: upgrade to Zig 0.14.1 and handle empty tuple syntax &.{} being detected as *const @TypeOf(.{}).
2024-11-19 11:45:36 +00:00
ea3ce685a9
runtimes/neuron: bump runtime version and expose nrt.h header to Zig.
2024-11-14 13:37:47 +00:00
59f99c4501
Enable Bazel debug/release Zig builds and add stdx.time.Duration.hz and stdx.math.roundeven utilities.
2024-10-08 09:39:00 +00:00
4a0b1cce50
Update Bazel workspace and XLA overlay (MODULE.bazel, BUILD files, patches) to prevent dual LLVM builds and apply migration/bump patches.
2024-09-27 14:00:44 +00:00
aec7072837
pjrt: add FFI bindings for custom calls
2024-09-10 09:14:28 +00:00
7df89301dc
Bump XLA version and import llvm, stablehlo, triton, and zig‑protobuf modules in workspace BUILD files.
2024-08-06 10:28:43 +00:00
ca4e061ad5
Add Bazel build configurations for macOS x86_64 CPU runtime and ZLS third‑party integration.
2024-07-25 15:58:14 +00:00
efcf955a4e
workspace, third_party/rules_zig: adjust ZLS to require --version as the first parameter and add missing keys to the BuildConfig object for code completion
2024-07-10 15:20:12 +00:00
30f6be0e2f
Update core Zig modules (async, mlir, pjrt, stdx) and third‑party Bazel definitions for the Zig 0.14.0 release.
2024-07-02 14:19:04 +00:00
967eeb928f
Update Bazel workspace and runtime configs: rework sandboxing, bump PJRT to 7.0.0, and upgrade CUDA (12.8), cuDNN (9.8), and ROCm (6.3.4).
2024-06-25 11:00:29 +00:00
8613a6da7d
workspace: update rules_cc to 0.1.1 and Bazel to 8.1.1 to resolve yanked version issue; adjust MODULE.bazel and .bazelversion accordingly; verified successful build and execution
2024-06-19 12:06:53 +00:00
5bcaf374c8
workspace: bump libxev to 20250222.0-07bcffa and introduce Dynamic to select between IO Uring and Epoll on Linux based on availability
2024-06-18 14:26:44 +00:00
ab5ad874c3
workspace: fix libxev version
2024-06-07 14:25:44 +00:00
72263aa9e3
workspace: fix mistakes in modules bumps
...
Fix missing = in stbalehlo integrity.
Changes back - by . in xla module name and folders.
Correctly depend on `xla@20250204.1-6789523`
2024-06-06 09:56:17 +00:00
f7450a2104
stablehlo: bump to head and use new dialect capi
...
This drastically reduce the number of build steps (from 3589 to 2553 steps)
2024-05-31 13:02:46 +00:00
f5ab6ff2c6
Update XLA to version 20250204.0-6789523 and adjust Bazel module and runtime files for Bazel 8 compatibility.
2024-05-03 15:57:56 +00:00
5a2171793d
workspace: MODULE.bazel cleanup
...
Title says it all !
2024-04-22 09:27:44 +00:00
65c28111a9
Update libxev to version 20252401.0‑31eed4e and apply patches and.
2024-04-15 13:03:25 +00:00
567210d1d7
bazel: depend on prebuilt protoc binaries to eliminate ~1300 build steps. Note: integration is currently blocked due to version constraints in rules_proto and toolchains_protoc.
2024-03-29 09:54:57 +00:00
959bc48c42
Add HuggingFace tokenizer bindings and SentencePiece integration; update BUILD files, async utilities, and FFI modules to support the new tokenizers.
2024-02-28 15:47:37 +00:00
b4b2490690
Upgrade rules_distroless to 0.4.2 in MODULE.bazel and refresh MODULE.bazel.lock accordingly.
2024-02-21 17:48:10 +00:00
169a24307c
Migrate workspace and XLA module definitions to Bazel 8, updating MODULE.bazel files, BUILD rules, and related migration patches.
2024-02-12 12:43:23 +00:00
7e6103d876
Upgrade XLA to version 20250122.0-cc075be, switch to nvptx compiler and nvlink with nvjitlink support, add warning for CUDA path in LD_LIBRARY_PATH, and revert the previous CUDA sandbox fix.
2024-02-06 09:31:48 +00:00
5120fe00dc
Update libxev epoll patch to resolve crashes and hangs in epoll and kqueue implementations.
2024-01-29 17:15:11 +00:00
a7b7ae0180
Fix async hangs by reworking the libxev epoll backend and using callBlocking for PJRT plugin loading, improving performance across async and runtime modules.
2024-01-16 14:13:45 +00:00
434cee3a6c
Fix CUDA and ROCm sandbox discovery, update epoll libxev patch to prevent high CPU usage, enable XLA GPU latency‑hiding scheduler, and upgrade cuDNN to 9.6.0.
2024-01-15 09:41:42 +00:00
