Bits, numbers, and the cost of approximation

A bit has two states. An unsigned integer interprets a sequence of bits as a sum of powers of two: the rightmost position contributes one, the next two, then four. The pattern 1101 therefore denotes thirteen. With eight bits there are 256 distinct patterns, so unsigned values range from zero through 255. Signed integers, characters, and image pixels assign different meanings to patterns; bytes alone do not announce their interpretation.

A byte is eight bits. Distinguish decimal storage units, such as one megabyte meaning one million bytes, from binary units, such as one mebibyte meaning 1,048,576 bytes. If a tensor has one million elements and each stored element occupies two bytes, its raw element payload is two megabytes. Shape metadata, alignment, allocator overhead, gradients, and temporary buffers add costs outside that calculation. State the boundary before comparing memory estimates.

Floating-point formats represent a sign, a scaled significand, and an exponent, with special encodings for exceptional values. Like a finite decimal expansion of one third, many finite binary fractions approximate rather than equal their intended real value. Decimal 0.1 is a familiar example. Printing a short decimal can hide this approximation; a neat display does not turn the underlying computation into exact decimal arithmetic.

Precision describes how finely a format distinguishes nearby values. Accuracy describes closeness to the desired answer, which also depends on the algorithm, input quality, and model assumptions. A highly precise calculation can accurately compute the wrong physical model. Conversely, lower precision may suffice for some neural-network operations after validation. Cancellation, repeated accumulation, overflow, and very different operand magnitudes deserve attention; selecting a smaller type is an experiment with consequences, not a universally safe optimization.

For a measured length, an absolute error of one micrometre may matter more than a relative percentage. For values spanning many orders of magnitude, a relative tolerance may be more useful. Combine both intentionally when testing approximate outputs. Exact identifiers and counts should generally remain exact; a tolerance appropriate for a simulated temperature is inappropriate for deciding whether two user IDs are equal.

Keep a trusted baseline when changing precision. Compare representative outputs and difficult edge cases, record memory savings, and inspect whether downstream decisions change. The lab plots the exact real-valued model $y=ax^2$. Change coefficient $a$ or input position $x$ and inspect the tangent: greater slope magnitude means small input changes produce larger output changes locally. This illustrates sensitivity, not binary rounding or an accelerator. Predict where an input approximation would matter most, then distinguish that mathematical observation from evidence about an actual precision change.