Precision, profiles & build

Storage widths, the fractional-bit split, the rounding rules per domain, how accuracy is validated, and how to build.

Profiles

Storage width is a compile-time choice via the GMATH_PROFILE environment variable (default: embedded), or the equivalent Cargo feature.

cargo build                           # embedded (default)
GMATH_PROFILE=scientific cargo build  # 77-digit precision

Custom integer/fraction split (realtime only). The realtime profile can move the binary point inside the 32-bit storage word via GMATH_FRAC_BITS (valid range 2–30 fractional bits); the other profiles use fixed splits.

GMATH_FRAC_BITS=10 GMATH_PROFILE=realtime cargo build   # Q22.10: wider range, 3 digits
GMATH_FRAC_BITS=24 GMATH_PROFILE=realtime cargo build   # Q8.24:  narrow range, 7 digits

Switching profiles compiles different code paths via cfg flags. Clear the incremental cache first, or stale artifacts can crash:

rm -rf target/debug/incremental/

Lookup tables are checked in, so a default build is fast. --features rebuild-tables regenerates them from build.rs (pure-Rust generation: π via Machin's formula, e via factorial series, √2 via continued fractions, zero runtime dependencies) and is slow.

Rounding

Rounding is defined per domain, and it is deterministic (integer arithmetic, so identical on every platform). The key property: everything beyond a lone storage-tier multiply or divide (every transcendental, dot product, decomposition, matrix chain, and fused op) computes at tier N+1 with double the fractional bits and rounds to storage exactly once, so the wide→storage downscale is the only rounding the result sees. The normative per-domain rounding table and the full explanation are in the rounding contract, CONTRACT.md §3. The certified interval types are the deliberate exception: they round each endpoint outward (toward −∞ below, +∞ above) so that the bracket provably contains the exact result; the scalar paths keep the per-domain rules and always land inside the corresponding interval.

Validation

Accuracy is defined by the test suite, not slogans:

• Reference values are generated with mpmath at 50–250 digit precision and embedded in the tests as exact strings, never computed with floats.
• Transcendentals are validated pointwise against those references on every profile; the wide-tier strategy means the final rounding step selects the nearest representable value for the storage format in the measured cases.
• Linear algebra, manifolds, Lie groups, ODE, and tensor tests combine structural checks (PA=LU, QᵀQ=I, exp/log roundtrips) with concrete mpmath-validated numerical comparisons.
• Both user-reachable decimal paths (imperative DecimalFixed and canonical gmath()) are graded against mootable/decimal-scaled's independent adversarial mpmath corpus in CI.

cargo test --release

Limits worth knowing

• Input representation. Within a single fixed representation some values aren't exact: 0.3 and 1/3 repeat in binary, so the binary domain approximates them (up to half an ULP before any computation). The canonical gmath(...) router sidesteps this by classifying each literal and routing it to a domain where it is exact (decimal for 0.3, symbolic for 1/3) and preserving the exact value in a shadow; the limit applies only when the binary domain is pinned (the imperative FixedPoint path or a forced binary mode).
• Conditioning. Error in a solved system scales with the condition number of the matrix. An ill-conditioned system (e.g. a Hilbert matrix) amplifies input error by orders of magnitude in any finite precision; iterative refinement recovers the residual but not the lost input information.
• Determinism. Whatever the error is, it is the same error on every platform; results are bit-identical across architectures.

Feature flags

No feature gates around core functionality: all domains, transcendentals, wide integers (I256/I512/I1024), and tiered overflow are always compiled.

The full determinism and cross-profile contract is in CONTRACT.md.

Disclaimer

This software is provided "as is", without warranty of any kind, express or implied. Use of this software is entirely at your own risk. In no event shall the author or contributors be held liable for any damages arising from the use or inability to use this software.

Built by Niels Erik Toren · support & donations.