Float Precision & Decimal Types
Working with numbers in Zolo: when to use
float,decimal, orbigdecimal, and how to compare floats safely.
Why float equality is tricky #
float in Zolo maps to IEEE 754 double-precision (f64). Binary floating-point can only represent a fraction of the real numbers exactly — familiar decimals like 0.1 and 0.2 are stored as the closest representable binary fraction, not the exact value. The accumulation of those tiny errors makes == unreliable:
print(0.1 + 0.2 == 0.3) // false — binary FP noise
print(0.1 + 0.2) // 0.30000000000000004 (not exactly 0.3)
Zolo's float-equality lint fires on any == / != between float values to remind you about this. Zolo gives you several better tools — choose the right one for your situation.
The decision tree #
Is the number exact by definition (money, quantities, totals)?
├── Yes, and ≤ 28 significant digits needed → use decimal
└── Yes, and unlimited precision needed → use bigdecimal
Is the number a result of continuous math (physics, graphics, stats)?
└── Yes → use float, compare with ~=
Do I need to detect NaN or infinity?
└── Yes → float predicates: x.is_nan(), x.is_finite(), x.is_infinite()
Do I need to test a math function to the last bit?
└── Yes → ~= within <n> ulps
The ~= operator family
~= is Zolo's approximate equality operator for float values. It replaces == in almost every float comparison. Its negation !~= replaces !=.
Bare form: adaptive tolerance #
The bare ~= uses an adaptive tolerance that automatically adjusts for the magnitude of the values. It combines an absolute guard (for near-zero) with a relative guard (for large magnitudes). This covers about 80% of everyday float comparison needs.
print(0.1 + 0.2 ~= 0.3) // true — FP noise within default tol
print(1.0 + 0.1 - 0.1 ~= 1.0) // true — rearrangement noise handled
print(1.0 ~= 2.0) // false — clearly different
print(1e10 + 1.0 ~= 1e10) // true — noise at this magnitude is expected
When to use: routine float comparisons where you don't have a specific tolerance in mind.
With absolute tolerance: within <tol>
a ~= b within <tol> checks that |a - b| <= tol. Use this when you know the scale of acceptable error in absolute units.
// Near-zero velocity check — anything below 1e-6 counts as "stopped"
fn is_stopped(velocity: float) -> bool {
return velocity ~= 0.0 within 1e-6
}
print(is_stopped(0.0000001)) // true
print(is_stopped(0.1)) // false
// Sensor tolerance: ±0.01 degrees is acceptable
fn temperatures_match(a: float, b: float) -> bool {
return a ~= b within 0.01
}
print(temperatures_match(36.6, 36.605)) // true
print(temperatures_match(36.6, 36.7)) // false
When to use: when you know the domain's measurement precision in absolute units (sensor tolerance, pixel distance, etc.).
With relative tolerance: within <tol> relative
a ~= b within <rtol> relative checks that |a - b| / max(|a|, |b|) <= rtol. Use this when values span orders of magnitude and you care about percentage error, not absolute error.
// 0.1% relative tolerance — works at any magnitude
print(1e10 ~= 1.0000001e10 within 0.001 relative) // true (0.000001%)
print(1.0 ~= 1.5 within 0.001 relative) // false (50% difference)
// Useful for comparing measurements of vastly different scales
fn is_within_percent(a: float, b: float, pct: float) -> bool {
return a ~= b within (pct / 100.0) relative
}
print(is_within_percent(100_000.0, 100_001.0, 0.01)) // true (0.001%)
print(is_within_percent(100_000.0, 101_000.0, 0.01)) // false (1%)
When to use: physics simulations, statistics, any domain where a % tolerance makes more sense than an absolute one.
With ULP tolerance: within <n> ulps
a ~= b within <n> ulps compares the bit-pattern distance between two floats. "ULP" stands for "unit in the last place" — one ULP is the smallest step between adjacent representable floats at a given magnitude. This is the most precise mode: it tells you whether two floats are the same up to the last N bits of precision.
use std::math
// 0.1 + 0.2 vs 0.3: differs by exactly 1 ULP at this magnitude
print((0.1 + 0.2) ~= 0.3 within 0 ulps) // false — not bit-exact
print((0.1 + 0.2) ~= 0.3 within 1 ulps) // true
print((0.1 + 0.2) ~= 0.3 within 4 ulps) // true (4 ULPs is very permissive)
// Exact match: 0 ULPs
print(1.0 ~= 1.0 within 0 ulps) // true (literally identical bits)
// Distinctly different values are many ULPs apart
print(1.0 ~= 1.001 within 100 ulps) // false (1.0 vs 1.001 ≈ 4.5e12 ULPs)
// sqrt round-trip: a few ULPs of error is typical
let two = math.sqrt(2.0) * math.sqrt(2.0)
print(two ~= 2.0 within 4 ulps) // true — roundtrip to within 4 bits
print(two ~= 2.0 within 0 ulps) // probably false — tiny error introduced
When to use: testing math function implementations, verifying bit-for-bit fidelity of algorithms, or any code where you want "exactly N bits of precision".
The negation: !~=
a !~= b is exact semantic sugar for not (a ~= b). It accepts all the same within clauses. Use it for guard clauses and convergence loops where the condition reads more naturally with the negation:
// Iterative convergence — loop while "not yet converged"
let mut x = 1.0
let mut prev = 0.0
while prev !~= x within 1e-10 {
prev = x
x = (x + 2.0 / x) * 0.5 // Newton step toward sqrt(2)
}
print("sqrt(2) ≈ {x}") // ≈ 1.4142...
// Guard clause in a test helper
fn assert_close(a: float, b: float, label: str) {
if a !~= b within 1e-9 {
print("FAIL {label}: {a} is not close to {b}")
} else {
print("OK {label}")
}
}
assert_close(0.1 + 0.2, 0.3, "fp_noise") // OK fp_noise
// With all tolerance modes
print(1.0 !~= 2.0) // true (default tol)
print(1.0 !~= 1.0001 within 0.001) // false (within tol → NOT not-equal)
print(100.0 !~= 101.0 within 0.001 relative) // true (1% > 0.1% rtol)
print(1.0 !~= 2.0 within 4 ulps) // true (distinctly unequal)
Summary table #
| Form | Check performed | Typical use case |
|---|---|---|
a ~= b |
adaptive (abs + rel combined) | everyday float comparison |
a ~= b within tol |
|a - b| <= tol |
known absolute precision |
a ~= b within rtol relative |
|a - b| / max(|a|,|b|) <= rtol |
percentage tolerance |
a ~= b within n ulps |
bit-pattern distance ≤ n | math function testing |
a !~= b [within ...] |
negation of corresponding ~= |
guard clauses, loops |
Predicates and special values #
IEEE 754 defines three "special" float values that don't behave like ordinary numbers: NaN, +∞, and −∞. Use predicates to detect them.
Detecting NaN #
NaN (Not a Number) is produced by indeterminate operations: 0.0 / 0.0, sqrt(-1.0), ∞ − ∞. It has a unique property: NaN is never equal to anything, including itself. Never use == to check for NaN.
use std::math
let nan = 0.0 / 0.0
print(nan == nan) // false — IEEE 754 rule, not a Zolo quirk
print(math.is_nan(nan)) // true — always use this
print(nan.is_nan()) // true — method form, same result
// NaN propagates through every operation
print(math.is_nan(nan + 1.0)) // true
print(math.is_nan(nan * 0.0)) // true
// ~= also returns false for any comparison involving NaN
print(nan ~= nan) // false
print(nan ~= 1.0) // false
// Use math.is_nan or .is_nan() for explicit sentinel detection
let unset: float = math.nan
if unset.is_nan() {
print("value not yet set")
}
Detecting infinity #
math.huge is the float constant for +∞. -math.huge is −∞. Division by zero produces ±∞ (not NaN). math.is_infinite matches both signs; math.is_inf is a shorter alias.
use std::math
print(math.is_infinite(math.huge)) // true
print(math.is_infinite(-math.huge)) // true
print(math.is_inf(1.0 / 0.0)) // true (+inf from division)
print(math.is_infinite(42.0)) // false
print(math.is_infinite(0.0 / 0.0)) // false (that's NaN, not Inf)
// Method form
print(math.huge.is_infinite()) // true
Checking for "normal" values #
math.is_finite (or .is_finite()) is the cleanest guard: it returns true only when the value is neither NaN nor infinite. Use it to validate inputs before doing arithmetic.
use std::math
fn area_of_circle(radius: float) -> float? {
if !radius.is_finite() { return nil } // reject NaN, ±inf
if radius < 0.0 { return nil }
return math.pi * radius * radius
}
print(area_of_circle(2.0)) // 12.566...
print(area_of_circle(-1.0)) // nil
print(area_of_circle(math.huge)) // nil
print(area_of_circle(float.NAN)) // nil
Constants: float.EPSILON, float.INFINITY, etc.
The float type exposes IEEE 754 constants as type-level (static) properties:
| Constant | Value | Description |
|---|---|---|
float.EPSILON |
~2.22e-16 | Smallest difference from 1.0 representable as f64 |
float.MIN |
~−1.8e308 | Most negative finite f64 |
float.MAX |
~1.8e308 | Largest finite f64 |
float.MIN_POSITIVE |
~2.23e-308 | Smallest positive normal f64 |
float.INFINITY |
+∞ | Positive infinity |
float.NEG_INFINITY |
−∞ | Negative infinity |
float.NAN |
NaN | IEEE 754 NaN sentinel |
use std::math
print(float.EPSILON) // 2.220446049250313e-16
print(float.INFINITY) // inf
print(float.NEG_INFINITY) // -inf
print(float.MAX) // 1.7976931348623157e308
print(float.MIN_POSITIVE) // 2.2250738585072014e-308
print(math.is_nan(float.NAN)) // true
// Using EPSILON as a lower-bound threshold
fn is_negligible(x: float) -> bool {
return x.abs() <= float.EPSILON * 8.0
}
The math module also exposes shorthand constants for the most common ones:
math.* |
Equivalent |
|---|---|
math.epsilon |
float.EPSILON |
math.infinity |
float.INFINITY |
math.nan |
float.NAN |
math.huge |
float.INFINITY (Lua-compatible alias) |
Methods on float
Every float value has methods that mirror the math.* stdlib functions. The method form (x.is_nan()) is preferred for chaining and readability; the function form (math.is_nan(x)) is preferred when you want a first-class callable.
| Method | math.* equivalent |
Return type | Description |
|---|---|---|---|
.is_nan() |
math.is_nan(x) |
bool |
True if NaN |
.is_finite() |
math.is_finite(x) |
bool |
True if not NaN and not infinite |
.is_infinite() |
math.is_infinite(x) |
bool |
True if ±∞ |
.abs() |
math.abs(x) |
float |
Absolute value |
.abs_diff(other) |
math.abs_diff(x, y) |
float |
|x - y| |
.approx_eq(other) |
math.approx_eq(x, y) |
bool |
Adaptive tolerance |
.approx_eq_abs(other, tol) |
math.approx_eq_abs(x, y, tol) |
bool |
Absolute tolerance |
.approx_eq_rel(other, rtol) |
math.approx_eq_rel(x, y, rtol) |
bool |
Relative tolerance |
.approx_eq_ulps(other, n) |
math.approx_eq_ulps(x, y, n) |
bool |
ULP tolerance |
use std::Array
let x: float = 0.1 + 0.2
// Predicate methods
print(x.is_nan()) // false
print(x.is_finite()) // true
print(x.is_infinite()) // false
// Arithmetic
print((-3.14).abs()) // 3.14
print(x.abs_diff(0.3)) // ~5.55e-17 (the actual FP noise)
// Approximate equality methods (parallel to ~= operator)
print(x.approx_eq(0.3)) // true — same as: x ~= 0.3
print(x.approx_eq_abs(0.3, 1e-9)) // true — same as: x ~= 0.3 within 1e-9
print(x.approx_eq_rel(0.3, 0.001)) // true — same as: x ~= 0.3 within 0.001 relative
print(x.approx_eq_ulps(0.3, 4)) // true — same as: x ~= 0.3 within 4 ulps
// First-class usage: pass a method as a callable
let values = [0.301, 0.299, 0.30000001, 0.5, 0.7]
let near = Array.filter(values, |m| m.approx_eq_abs(0.3, 0.01))
print(near) // [0.301, 0.299, 0.30000001]
The decimal type
Why decimal exists #
float uses binary arithmetic — it fundamentally cannot represent 0.1 exactly. This is fine for physics or graphics where continuous values don't have an "exact" answer. But it's wrong for money, quantities, and totals, where $0.10 + $0.20 must equal exactly $0.30.
decimal solves this by using decimal (base-10) arithmetic backed by the rust_decimal crate. It can represent up to ~28 significant decimal digits exactly.
// float: binary noise
print(0.1 + 0.2) // 0.30000000000000004 ← wrong for money
print(0.1 + 0.2 == 0.3) // false
// decimal: exact
print(0.1d + 0.2d) // 0.3
print(0.1d + 0.2d == 0.3d) // true — no lint warning, comparison is exact
Literals: d suffix
Append d to any numeric literal to get a decimal. Works with both decimal notation and integers:
let price: decimal = 9.99d
let tax_rate: decimal = 0.08d
let quantity: decimal = 100d // integer form — exact
print(price * quantity + price * quantity * tax_rate) // 1078.92
Arithmetic + comparison (exact, no lint warning) #
decimal supports the same operators as float: +, -, *, /, %, and all comparison operators (==, !=, <, <=, >, >=). All operations are exact within the representable range. The float-equality lint does not fire on decimal comparisons.
let a: decimal = 2.5d
let b: decimal = 1.5d
print(a + b) // 4.0
print(a - b) // 1.0
print(a * b) // 3.75
print(a / b) // 1.6666...
print(a % b) // 1.0
print(0.1d + 0.2d == 0.3d) // true (exact)
print(0.5d < 1.0d) // true
print(0.5d <= 0.5d) // true
Conversion via as
Convert between decimal and other numeric types using as. Note that converting a float to decimal reveals the binary noise that was already present in the float.
// decimal → float: may lose precision (28 digits → 15-17 digits)
let f: float = 0.3d as float
print(f) // 0.3
// float → decimal: reveals the binary FP representation
let d: decimal = 0.1 as decimal
print(d) // 0.1000000000000000055511151231...
// int → decimal: always exact
let n: decimal = 42 as decimal
print(n) // 42
// decimal → int: truncates toward zero (does not round)
let i: int = 99.9d as int
print(i) // 99
// decimal ↔ bigdecimal
let bd: bigdecimal = 3.14d as bigdecimal
print(bd) // 3.14
Key insight: Always use decimal literals (0.1d) rather than converting from float (0.1 as decimal). The float-to-decimal conversion carries the float's imprecision.
Methods: .round(), .ceil(), .floor(), .trunc(), .scale()
| Method | Behavior | Example |
|---|---|---|
.round(places) |
Half-to-even (banker's) rounding | 3.14159d.round(2) → 3.14 |
.ceil() |
Toward +∞ | 3.2d.ceil() → 4, -2.7d.ceil() → -2 |
.floor() |
Toward −∞ | 3.8d.floor() → 3, -2.3d.floor() → -3 |
.trunc() |
Toward zero | 3.9d.trunc() → 3, -3.9d.trunc() → -3 |
.scale() |
Number of decimal places | 3.14159d.scale() → 5 |
let d: decimal = 3.14159d
print(d.round(2)) // 3.14
print(d.round(4)) // 3.1416 (half-to-even)
print(d.ceil()) // 4
print(d.floor()) // 3
print(d.trunc()) // 3
print(d.scale()) // 5
let neg: decimal = -2.7d
print(neg.ceil()) // -2 (toward +∞, i.e., less negative)
print(neg.floor()) // -3 (toward -∞, i.e., more negative)
print(neg.trunc()) // -2 (toward zero, same as ceil for negatives)
Real-world example: invoice #
fn invoice_total(items: [decimal], tax_rate: decimal) -> decimal {
let mut subtotal: decimal = 0d
for item in items {
subtotal = subtotal + item
}
return subtotal + subtotal * tax_rate
}
let items: [decimal] = [19.99d, 4.50d, 12.00d]
let total = invoice_total(items, 0.08d)
print(total) // 39.7692 — exact, no floating-point drift
print(total.round(2)) // 39.77 — banker's rounding for presentation
The bigdecimal type
When to use bigdecimal vs decimal
decimal is backed by rust_decimal which is a fixed 96-bit representation, covering up to ~28 significant decimal digits. For most financial, scientific, and engineering work, 28 digits is more than enough.
Use bigdecimal when you need:
- More than 28 significant digits of precision
- Representing very large integers exactly (e.g., cryptographic key material)
- Arbitrary-precision intermediate results in scientific computation
decimal — ≤ 28 significant digits, fast, fixed size
bigdecimal — unlimited digits, heap-allocated, slower
Literals: bd suffix
Append bd to any numeric literal:
let a: bigdecimal = 0.1bd
let b: bigdecimal = 0.2bd
print(a + b) // 0.3 (exact)
print(a + b == 0.3bd) // true
// Values beyond decimal's range
let huge: bigdecimal = 1234567890123456789012345678901234567890bd
let one: bigdecimal = 1bd
print(huge + one) // 1234567890123456789012345678901234567891
Same arithmetic, methods, and conversions as decimal
bigdecimal supports the same operators, the same five methods (.round(), .ceil(), .floor(), .trunc(), .scale()), and the same as conversions. The interface is deliberately identical — the only difference is the precision ceiling:
let bd: bigdecimal = 99.999999bd
print(bd.round(2)) // 100.00
print(bd.ceil()) // 100
print(bd.floor()) // 99
print(bd.trunc()) // 99
print(bd.scale()) // 6
// Cross-cast with decimal
let d: decimal = 3.14d
let from_d: bigdecimal = d as bigdecimal
print(from_d) // 3.14
When to NOT use bigdecimal
bigdecimal is heap-allocated and significantly slower than decimal or float. Don't reach for it by default — use decimal for financial arithmetic, float for continuous math, and reserve bigdecimal for the specific cases where neither suffices.
The math.* functions
Every ~= operator form has a corresponding math.* function. Use functions when you need a first-class callable (passing to Array.filter, Array.map, higher-order functions, etc.) or when you want the tolerance mode to be explicit at the call site.
Approximate equality #
| Operator form | Function form | Description |
|---|---|---|
a ~= b |
math.approx_eq(a, b) |
Adaptive default tolerance |
a ~= b within tol |
math.approx_eq_abs(a, b, tol) |
Absolute tolerance |
a ~= b within rtol relative |
math.approx_eq_rel(a, b, rtol) |
Relative tolerance |
a ~= b within n ulps |
math.approx_eq_ulps(a, b, n) |
ULP tolerance |
use std::Array
use std::math
// Operator and function are semantically equivalent
print((0.1 + 0.2) ~= 0.3) // true
print(math.approx_eq(0.1 + 0.2, 0.3)) // true — same result
// First-class usage: use the function form to filter
let measurements = [0.301, 0.299, 0.30000001, 0.5, 0.7]
let near = Array.filter(measurements, |m| math.approx_eq_abs(m, 0.3, 0.01))
print(near) // [0.301, 0.299, 0.30000001]
// Reduce: find first sample within tolerance
fn first_near(samples: [float], target: float, tol: float) -> float? {
for s in samples {
if math.approx_eq_abs(s, target, tol) { return s }
}
return nil
}
print(first_near([1.0, 1.5, 1.999, 2.0001], 2.0, 0.01)) // 1.999
Predicates #
| Function | Method alias | Description |
|---|---|---|
math.is_nan(x) |
x.is_nan() |
True if NaN |
math.is_finite(x) |
x.is_finite() |
True if not NaN and not infinite |
math.is_infinite(x) |
x.is_infinite() |
True if ±∞ |
math.is_inf(x) |
— | Alias for math.is_infinite |
Constants #
| Constant | Value | Notes |
|---|---|---|
math.epsilon |
~2.22e-16 | Same as float.EPSILON |
math.infinity |
+∞ | Same as float.INFINITY |
math.nan |
NaN | Same as float.NAN |
math.huge |
+∞ | Lua-compatible alias for math.infinity |
math.pi |
3.14159... | π |
math.tau |
6.28318... | 2π |
math.e |
2.71828... | Euler's number |
The float-equality lint
What triggers it #
The float-equality lint fires on any direct == or != comparison where either operand is a float literal or has a known float type. It does not fire for decimal or bigdecimal comparisons (those are exact by design).
let x: float = 0.1 + 0.2
x == 0.3 // lint: float-equality — consider using ~=
x != 0.3 // lint: float-equality — consider using !~=
0.1 + 0.2 == 0.3 // lint: float-equality
The full lint message #
When triggered, the lint emits a prescriptive, multi-line message:
warning[float-equality]: direct == comparison of float values is unreliable
due to binary floating-point representation (e.g., 0.1 + 0.2 ≠ 0.3).
Consider one of:
• `a ~= b` — approximate equal (default adaptive tolerance)
• `a ~= b within 1e-9` — absolute tolerance
• `a ~= b within 0.001 relative`— relative tolerance
• `a ~= b within 4 ulps` — bit-pattern (ULP) tolerance
• `math.is_nan(x)` — to check for NaN specifically
• Use `decimal` or `bigdecimal` — for exact decimal arithmetic (money, totals)
How to fix it — four paths #
Path 1: Use ~= (most common)
// Before (lint fires)
if x == 0.3 { ... }
// After (lint clear)
if x ~= 0.3 { ... }
if x ~= 0.3 within 1e-9 { ... } // with explicit tolerance
Path 2: Use float methods
// Also lint-free; preferred in higher-order contexts
if x.approx_eq(0.3) { ... }
if x.approx_eq_abs(0.3, 1e-9) { ... }
Path 3: Switch to decimal
// When the values are exact decimals (money, fixed quantities)
let x: decimal = 0.1d + 0.2d
if x == 0.3d { ... } // exact — no lint
Path 4: Use NaN-specific predicate
// When the intent is to detect NaN — don't use x == float.NAN
if x.is_nan() { ... } // correct; lint-free
Suppressing the lint deliberately #
If you genuinely need a bit-exact float comparison (unusual), use within 0 ulps to make your intent explicit and suppress the lint:
// Explicitly: "I want exact bit-pattern equality"
if x ~= y within 0 ulps { ... }
Cross-reference #
- Operators —
~=,!~=,within, and all other Zolo operators - Variables and Types —
decimalandbigdecimalin the type table - Standard Library — complete
math.*function reference - Type Checker —
float-equalitylint rule details - Examples in
examples/features/17-stdlib/math/— runnable code for every feature:09-approx-eq.zolo—~=operator andwithinclauses10-approx-eq-not.zolo—!~=operator11-approx-eq-functions.zolo—math.approx_eq*function family12-nan-finite-infinite.zolo— predicates and special values13-approx-eq-ulps.zolo— ULP-based comparison14-decimal-type.zolo—decimaltype and literals15-bigdecimal-type.zolo—bigdecimaltype16-decimal-methods.zolo—.round(),.ceil(),.floor(),.trunc(),.scale()17-float-methods.zolo— float methods and type-level constants