2.5 Floating-Point Arithmetic & Errors

Theory: Floating-Point Arithmetic & Errors

Section 2.4 showed how floating-point numbers are stored. Here we compute with them: addition, subtraction, multiplication and division, all worked in the lecture’s 14-bit model (1 sign bit, 5-bit excess-16 exponent, 8 fraction bits with an implicit leading 1). Then we look at what can go wrong: limited range (overflow/underflow) and limited precision (rounding errors), and the programming habits that follow (Null & Lobur, Ch. 2).

Addition and Subtraction

You cannot add significands until they refer to the same power of the base. Decimal analogy:

1.5 \times 10^2 + 3.6 \times 10^3 = 0.15 \times 10^3 + 3.6 \times 10^3 = 3.75 \times 10^3

In general, for $X = X_S \times B^{X_E}$ and $Y = Y_S \times B^{Y_E}$ with $X_E \le Y_E$ :

X \pm Y = \left(X_S \times B^{X_E - Y_E} \pm Y_S\right) \times B^{Y_E}

Algorithm (FP add/subtract)

Check for zeros. If either operand is 0, the result is the other operand (negated if it is the subtrahend).
Align the significands. Shift the significand of the number with the smaller exponent to the right, adding 1 to its exponent for each place, until both exponents are equal.
Add or subtract the significands (taking signs into account).
Normalize the result back to $1.xxx \times 2^e$ : if it became $\ge 2$ (e.g. $10.xxx$ ), shift right and increase the exponent; if it has leading 0s (e.g. $0.0xxx$ , possible after subtraction), shift left and decrease the exponent.
Round/truncate to the available fraction bits. Check the exponent for overflow/underflow.

Always shift the smaller number. Shifting it right only loses its least significant bits, which matter least. Shifting the larger number left would push off its most significant bits.

Mini-example

$1.1 \times 2^1 + 1.0 \times 2^0$ (i.e. $3 + 1$ ): align $1.0 \times 2^0 = 0.1 \times 2^1$ ; add $1.1 + 0.1 = 10.0$ ; normalize $10.0 \times 2^1 = 1.0 \times 2^2 = 4$ ✓.

Multiplication and Division

Multiplication does not need alignment:

X \times Y = (X_S \times Y_S) \times B^{X_E + Y_E}, \qquad \frac{X}{Y} = \left(\frac{X_S}{Y_S}\right) \times B^{X_E - Y_E}

Algorithm (FP multiply)

Check for zeros (if either is 0, the product is 0).
Sign = $S_X \oplus S_Y$ (XOR: different signs → negative).
Add the exponents. With biased exponents, add the stored fields and subtract one bias: $E = E_X + E_Y - 16$ (otherwise the bias would be counted twice).
Multiply the significands (an unsigned integer-style multiplication).
Normalize (the product of two values in $[1, 2)$ lies in $[1, 4)$ , so at most one right shift is needed) and round/truncate.

Division is analogous: subtract the exponents (and add the bias back: $E = E_X - E_Y + 16$ ) and divide the significands, then normalize.

Summary of floating-point arithmetic

Operation	Significands	Exponents	Extra step
$X + Y$ , $X - Y$	add / subtract after alignment	make equal (the larger one)	align first, normalize after
$X \times Y$	multiply	add	normalize; sign = XOR
$X / Y$	divide	subtract	normalize; sign = XOR

Range, Precision, Overflow and Underflow

The exponent field decides the range (how large/small magnitudes can get).
The significand field decides the precision (how many significant bits; how finely values are spaced).

Overflow occurs when a result’s exponent is too large to store (there is no room for the high-order bits of the exponent). Underflow occurs when a nonzero result’s exponent is too small, i.e. the value is too close to zero to represent.

The real line splits into regions (shown for IEEE double precision, whose limits are about $10^{\pm 308}$ ):

 negative     |  expressible      | negative | positive |  expressible      |  positive
 overflow     |  negative numbers | underflow| underflow|  positive numbers |  overflow
--------------+-------------------+----------0----------+-------------------+-----------
         -1.0e308             -1.0e-308            1.0e-308              1.0e308

For the 14-bit model, true exponents run from $-16$ to $+15$ , so the largest magnitude is $1.11111111_2 \times 2^{15} = 65408$ , and anything requiring an exponent below $-16$ underflows.

Precision limits mean spacing: with 8 fraction bits, neighbouring values near $2^e$ are $2^{e-8}$ apart. Near $128 = 2^7$ the spacing is $2^{-1} = 0.5$ , so only $128, 128.5, 129, \dots$ exist.

Floating-Point Errors

The real numbers are infinite and dense (between any two there is always another), but a computer’s arithmetic is finite: it can only approximate real arithmetic. More bits give a better approximation, but there is always some error, no matter how many bits we use.

Kind	What happens	Example (14-bit model)
Blatant error	Obvious failure: the program crashes or produces something clearly wrong, typically overflow/underflow.	$2^{-18}$ needs exponent field $16 - 18 = -2 \lt 0$ : cannot be stored (underflow).
Subtle error	A plausible but slightly wrong result that is hard to detect.	$128.75$ is stored as $128.5$ (truncation).

Experienced programmers know that it is better for a program to crash than to produce incorrect but plausible results. Blatant errors get noticed and fixed; subtle errors silently propagate through calculations.

Never test floating-point values for exact equality, and in particular never test x == 0.0. Values that “should” be equal often differ in the last bit (and zero even has two encodings, $+0$ and $-0$ ). Instead compare with a tolerance: treat $x$ and $y$ as equal when abs(x - y) < epsilon for some small epsilon suited to the problem.

Mini-example

In IEEE double precision, adding $0.1$ to itself ten times gives 0.9999999999999999, not 1.0, because $0.1$ is a non-terminating binary fraction. So sum == 1.0 is false, while abs(sum - 1.0) < 1e-9 is true.

Worked Examples

Exercise 1

Add $1.11001_2 \times 2^2 + 1.10011_2 \times 2^0$ . Normalize the result, store it in the 14-bit model, and show what happens if only 5 fraction bits are kept. Verify in decimal.

Show solution ↓

Solution

Step 1, zeros: neither operand is 0.

Step 2, align: the smaller exponent is $0$ ; shift $1.10011$ right by $2 - 0 = 2$ places:

1.10011 \times 2^0 = 0.0110011 \times 2^2

Step 3, add:

   1.1100100
 + 0.0110011
 -----------
  10.0010111      (x 2^2)

Step 4, normalize: $10.0010111 \times 2^2 = 1.00010111 \times 2^3$ .

Step 5, store (8 fraction bits): the fraction $00010111$ fits exactly.

Sign $0$ , exponent $3 + 16 = 19 = 10011$ , fraction $00010111$
14-bit result: $0\ 10011\ 00010111$

With only 5 fraction bits, truncation drops the last three bits: $1.00010 \times 2^3 = 1000.10_2 = 8.5$ .

Decimal check:

$1.11001_2 = 1 + \tfrac12 + \tfrac14 + \tfrac1{32} = 1.78125$ , times $4 = 7.125$ .
$1.10011_2 = 1 + \tfrac12 + \tfrac1{16} + \tfrac1{32} = 1.59375$ , times $1 = 1.59375$ .
Sum $= 8.71875$ . And $1.00010111_2 \times 8 = 1000.10111_2 = 8 + 0.5 + 0.125 + 0.0625 + 0.03125 = 8.71875$ ✓.

The 5-bit version ( $8.5$ ) shows how truncating the significand introduces an error of $0.21875$ .

Exercise 2

In the 14-bit model, add $A = 0\ 10010\ 10000000$ and $B = 0\ 10000\ 01000000$ . Give the result as a 14-bit pattern and in decimal.

Show solution ↓

Solution

Decode:

$A$ : exponent $10010 = 18$ , true $2$ ; significand $1.1$ ; so $A = 1.1 \times 2^2$ ( $= 6$ ).
$B$ : exponent $10000 = 16$ , true $0$ ; significand $1.01$ ; so $B = 1.01 \times 2^0$ ( $= 1.25$ ).

Align: $B$ has the smaller exponent; shift right $2 - 0 = 2$ places: $1.01 \times 2^0 = 0.0101 \times 2^2$ .

Add:

   1.1000
 + 0.0101
 --------
   1.1101     (x 2^2)

Normalize: already of the form $1.xxx$ . Result $1.1101 \times 2^2$ .

Store: sign $0$ , exponent $2 + 16 = 18 = 10010$ , fraction $11010000$ :

A + B = 0\ \ 10010\ \ 11010000

Check: $1.1101 \times 2^2 = 111.01_2 = 7.25 = 6 + 1.25$ ✓.

Exercise 3

Compute $12 - 1.25$ using floating-point subtraction and store the result in the 14-bit model.

Show solution ↓

Solution

Convert and normalize: $12 = 1100_2 = 1.1 \times 2^3$ ; $1.25 = 1.01_2 = 1.01 \times 2^0$ .

Align: shift the smaller ( $1.01 \times 2^0$ ) right by $3$ places: $0.00101 \times 2^3$ .

Subtract:

   1.10000
 - 0.00101
 ---------
   1.01011     (x 2^3)

(Bit by bit from the right: $0 - 1$ borrow → 1; $0 - 0 - 1$ borrow → 1; $0 - 1 - 1$ borrow → 0; $0 - 0 - 1$ borrow → 1; $1 - 0 - 1 = 0$ ; integer $1 - 0 = 1$ .)

Normalize: the leading bit is still 1, so no shift needed: $1.01011 \times 2^3$ .

Store: sign $0$ , exponent $3 + 16 = 19 = 10011$ , fraction $01011000$ :

12 - 1.25 = 0\ \ 10011\ \ 01011000

Check: $1.01011 \times 2^3 = 1010.11_2 = 8 + 2 + 0.5 + 0.25 = 10.75$ ✓.

Exercise 4

Find the product of $12_{10}$ and $1.25_{10}$ using the 14-bit floating-point model. Show the exponent calculation using the biased fields.

Show solution ↓

Solution

Convert: $12 = 1.1 \times 2^3$ → $0\ 10011\ 10000000$ ; $1.25 = 1.01 \times 2^0$ → $0\ 10000\ 01000000$ .

Sign: $0 \oplus 0 = 0$ (positive).

Exponent: true exponents add: $3 + 0 = 3$ . With the biased fields: $19 + 16 - 16 = 19 = 10011$ (subtract one bias, otherwise it is counted twice).

Significands: $1.1 \times 1.01$ :

     1.01
   x  1.1
   ------
     101     (1.01 x 0.1, shifted)
    101      (1.01 x 1)
   ------
   1.111

(As integers: $101 \times 11 = 1111$ , i.e. $5 \times 3 = 15$ ; three fraction places in total give $1.111$ .)

Normalize: $1.111 \times 2^3$ , already normalized.

Store: $0\ \ 10011\ \ 11100000$ .

Check: $1.111 \times 2^3 = 1111_2 = 15 = 12 \times 1.25$ ✓.

Exercise 5

In the 14-bit model compute (a) $(-6) \times 3$ and (b) $18 \div 6$ . Show the normalization step and the 14-bit result.

Show solution ↓

Solution

$6 = 110_2 = 1.1 \times 2^2$ ; $3 = 11_2 = 1.1 \times 2^1$ .

(a) $(-6) \times 3$

Sign: $1 \oplus 0 = 1$ (negative).
Exponent: $2 + 1 = 3$ .
Significands: $1.1 \times 1.1 = 10.01$ (as integers $11 \times 11 = 1001$ , i.e. $3 \times 3 = 9$ , with two fraction places).
Normalize: $10.01 \times 2^3 = 1.001 \times 2^4$ (shift right 1, exponent $+1$ ).
Store: exponent $4 + 16 = 20 = 10100$ , fraction $00100000$ .

(-6) \times 3 = 1\ \ 10100\ \ 00100000

Check: $-1.001_2 \times 2^4 = -10010_2 = -18$ ✓.

(b) $18 \div 6$

$18 = 1.001 \times 2^4$ and $6 = 1.1 \times 2^2$ .

Sign: $0 \oplus 0 = 0$ .
Exponent: $4 - 2 = 2$ .
Significands: $1.001 \div 1.1 = 0.11$ (decimal: $1.125 / 1.5 = 0.75 = 0.11_2$ ).
Normalize: $0.11 \times 2^2 = 1.1 \times 2^1$ (shift left 1, exponent $-1$ ).
Store: exponent $1 + 16 = 17 = 10001$ , fraction $10000000$ .

18 \div 6 = 0\ \ 10001\ \ 10000000

Check: $1.1_2 \times 2 = 11_2 = 3$ ✓.

Exercise 6

Using the 14-bit excess-16 model:

(a) Store $128.75$ . What value is actually stored, and what is the error?

(b) Compute $128 + 0.75$ with the FP addition algorithm. What happens?

(c) Can $2^{-18}$ be stored? Can the product $2^{10} \times 2^{8}$ ?

(d) Classify each problem as a blatant or subtle error.

Show solution ↓

Solution

(a) $128.75 = 1000\,0000.11_2 = 1.00000001\underline{1} \times 2^7$ . The fraction has 9 bits ( $000000011$ ) but only 8 fit, so truncate to $00000001$ :

Sign $0$ , exponent $7 + 16 = 23 = 10111$ , fraction $00000001$ → $0\ 10111\ 00000001$
Stored value: $1.00000001_2 \times 2^7 = 1000\,0000.1_2 = 128.5$

Error $= 128.75 - 128.5 = 0.25$ ; relative error $= 0.25 / 128.75 \approx 0.0019 \approx 0.2\%$ .

The reason: near $2^7$ the 8 fraction bits give a spacing of $2^{7-8} = 0.5$ , so only $128$ , $128.5$ , $129$ , … exist.

(b) $128 = 1.0 \times 2^7$ , $0.75 = 0.11_2 = 1.1 \times 2^{-1}$ .

Align: shift $1.1 \times 2^{-1}$ right by $7 - (-1) = 8$ places: $0.000000011 \times 2^7$ .

Add: $1.000000000 + 0.000000011 = 1.000000011 \times 2^7$ .

Truncate to 8 fraction bits: $1.00000001 \times 2^7 = 128.5$ . The lowest bit of $0.75$ was shifted off the end during alignment, so the arithmetic itself loses $0.25$ : same subtle error as in (a).

(c) $2^{-18} = 1.0 \times 2^{-18}$ needs exponent field $-18 + 16 = -2$ , but the field cannot be negative (minimum $00000$ , true exponent $-16$ ). It cannot be stored: exponent underflow.

$2^{10} \times 2^{8} = 2^{18}$ needs exponent field $18 + 16 = 34 \gt 31$ . It cannot be stored: exponent overflow (the largest representable value is $65408 \lt 2^{18} = 262144$ ).

(d)

$2^{-18}$ (underflow) and $2^{18}$ (overflow): blatant errors. The value simply cannot be represented; the program fails or flags it.
$128.75 \to 128.5$ : a subtle error. A plausible number is produced that is wrong by about 0.2%, and nothing signals the problem.

Exercise 7

(a) Store $0.1$ in the 14-bit model. What value is actually stored?

(b) A program sums $0.1$ ten times in IEEE double precision and then tests if (sum == 1.0). What happens, and how should the test be written? Why is if (x == 0.0) also dangerous?

Show solution ↓

Solution

(a) $0.1 = 0.000110011001100\ldots_2$ (non-terminating; the block $0011$ repeats).

Normalize: $0.0001100110011\ldots = 1.100110011\ldots \times 2^{-4}$ .

Sign $0$ ; exponent $-4 + 16 = 12 = 01100$
First 8 fraction bits: $10011001$ (the rest are truncated)
Pattern: $0\ 01100\ 10011001$

Stored value: $1.10011001_2 \times 2^{-4} = \left(1 + \tfrac{153}{256}\right) / 16 = 0.099853515625$ . Error $\approx 0.000146$ (about $0.15\%$ ).

(b) Even with 52 fraction bits, $0.1$ is stored only approximately (in single precision it is $\text{3DCCCCCD}_{16}$ , already rounded). Each addition also rounds. After ten additions the double-precision sum is 0.9999999999999999, so sum == 1.0 is false and the branch is wrongly skipped. This is a subtle error: no crash, just wrong behaviour.

Correct approach: compare against a tolerance,

if (abs(sum - 1.0) < 1e-9) ...

Testing x == 0.0 is dangerous for the same reason: a result that “should” be zero is often a tiny nonzero value like $10^{-17}$ after rounding. (IEEE also has two zero encodings, $+0$ and $-0$ .) Use abs(x) < epsilon instead.