<span class="caps">ULP</span> Errors vs Relative Errors

Converting From ULP Errors to Relative Errors Let \(x\) be in the normal range, and \(|x-X|=\alpha\ulp(x)\). Then:

\begin{equation*} \left|\frac{x-X}{x}\right|\le\alpha\beta^{1-p} \end{equation*}

This is assuming there is no underflow.

The proof is that we know \(\ulp(x)=\beta^{e+1-p}\) for a normal number. And \(|x|\ge\beta^{e}\) for a normal number. Thus:

\begin{equation*} \frac{\ulp(x)}{|x|}=\frac{\beta^{e+1-p}}{|x|}\le\frac{\beta^{e+1-p}}{\beta^{e}} \end{equation*}

Converting from Relative Error to ULP Error

Let the relative error be \(\epsilon_{r}\). Then:

\begin{equation*} \left|x-X\right|\le\epsilon_{r}\beta^{p}\ulp(x) \end{equation*}

The proof:

We know that \(|x|\le\beta^{e+1}\) (i.e. go up one exponent). So \(|x-X|=|x|\epsilon_{r}\le\epsilon_{r}\beta^{e+1}=\epsilon_{r}\beta^{p}\ulp(x)\).

Note that this is true even for denormals. Example: Iterated Products

Say we are dealing with binary and have \(n\) floating point normal numbers. We multiply all of them together, using round to nearest. Assume no overflow occurs. Then the final result is \(P=x_{1}x_{2}\dots x_{n}K\) where:

\begin{equation*} \left(1-2^{-p}\right)^{n-1}\le K\le\left(1+2^{-p}\right)^{n-1} \end{equation*}

So the relative error of the final is less than or equal to \(\left(1+2^{-p}\right)^{n-1}-1\). We can take the linear approximation if \(n<<2^{p}\).

To derive this, consider just multiplying two numbers, \(a,b\). Now we know that \(|o(ab)-ab|\le\frac{1}{2}\ulp(ab)\). Thus, \(\alpha=\frac{1}{2}\). Converting to an inequality for the relative error, we see that \(\epsilon\) is bounded by \(\frac{1}{2}2^{1-p}=2^{-p}\). So now we can say that:

\begin{equation*} \left|\frac{o(ab)-ab}{ab}\right|\le2^{-p} \end{equation*}

Thus:

\begin{equation*} -2^{-p}\le\frac{o(ab)}{ab}-1\le2^{-p} \end{equation*}

This gives us:

\begin{equation*} 1-2^{-p}\le\frac{o(ab)}{ab}\le1+2^{-p} \end{equation*}

Since we are multiplying \(n\) numbers, we get:

\begin{equation*} \left(1-2^{-p}\right)^{n-1}\le K\le\left(1+2^{-p}\right)^{n-1} \end{equation*}