This is a work in progress that will eventually be merged with IEEE_754r. At this point this is just a list of notes at this point that will need to be fleshed out, cleaned up, and summarized.

Annex L is an informative annex in the works for IEEE_754r which will provide guidance to programming language developers on supporting the features of 754r.

We expect programs to leave modes unchanged, except subprograms whose whole purpose is to change the mode.

Contents

1 Parentheses issues 1.1 FMA examples 1.2 Issues 1.3 Languages 2 Levels of strictness 3 Compile time constants 4 Optimization level should be orthogonal from translation modes 5 Want to let programmers write generic programs 5.1 Issues 5.2 Widest need expression evaluation 6 Difficult to deal with languages which don't allow declaring types 7 Calculators and expression evaluation intended for human eyes should use decimal types. 8 Assignments 9 Meaning of exceptions and flags 10 Naming of formats 11 Operations 11.1 min / max

Parentheses issues

It is difficult to distinguish between "intended" parentheses and "non-intended" ones because, for example, many languages use C source code as their translation target. So the question is intended by whom?

The tension arrises because sometimes a programmer wants to have expressions interpreted in some particular way (for some good reason), and at other times the programmer might want to grant the system a license to interpret the expressions in any of a variety of ways with the goal of getting faster execution. In either case, it is desirable for the program to be clear. Of course, if we avoided infix notation, and just used prefix or postfix there would be no ambiguity, but this would limit the opportunities for optimization and likely also the clarity of the source text.

Superfluous parentheses are often left in by preprocessing that cannot be distinguished from intended parentheses

• Definition: Superfluous ()s are those that when removed wouldn't change the value in exact arithmetic (Level 1)

Optimizing compilers can take advantage of a license to compute "equivalent" expressions. They are included because it is safe to do so, in most cases extra ()s will not kill you. One can use FMA(a,b,c) to force an FMA: round(a*b+c).

If you have written an expression so you can tell how it will be evaluated, you want to be able to make that true.

FMA examples

Our difficulty is being able to tell when it is ok to use FMA in the face of superfluous ()s

• (a*b) + (c*d): add(prod(a,b),prod(c,d))
• (a*b) + c*d: FMA(c,d,prod(a,b))
• a*b + (c*d): FMA(a,b,prod(c,d))
• a*b + c*d: portable FMA(c,d,prod(a,b)), but native may optimize to FMA(a,b,prod(c,d)) if allowed

• a*b*c + d*e*f + g*h*i: portable same as (((a*b)*c) + (d*e)*f) + (g*h)*i
• (a*b*c) + (d*(e*f) + (g*h)*i): add(prod(prod(a,b),c),fma(prod(g,h),i,prod(d,prod(e,f)))
• ... Example from Tim's paper ...

• Could have a "mandate FMA" mode that requires all possible contractions be performed. If the order of contractions is ambiguous, then it is up to the system to define what happens. So this would eliminate strict results except for codes which contain no FMA-ambiguous expressions.
  • A problem here is that it is then very difficult to say you do not want an FMA in some place without an explicit product or cast. E.g., a += (double)(b*c)

Issues

Some languages have looser () than others. Want to have *some* way to put in () that we really mean. C compilers often aggressively remove ()s. So you can put superfluous ()s in until you are blue in the face in a*b+c*d and it won't change the way the machine looks at it.

Want to be able to:

• Want to avoid doing these without a licence to remove superfluous
  • (a-b)+c -> (a+c)-b
  • (a*b)/c -> (a/c)*b

Languages

• At the moment, Java doesn't support contractions. FMA could be supported in a math library.
• C99 supports contractions

Levels of strictness

1. Strict, completely reproducible expressions
   1. Strict, single, bottom-up, no-FMA.
   2. Strict, single, bottom-up, mandate-FMA.
   3. Strict, widest-needed, no-FMA.
   4. Strict, widest-needed, mandate-FMA.
2. License associativity, FMA synthesis, et al.
   1. License sum reductions, associativity.
   2. License sum reductions, associativity, FMA synthesis.
   3. License sum reductions, associativity, FMA synthesis, others?
3. Relaxed mode (Not supported by the standard, but likely people will do it, e.g., for benchmarks)
   1. Replace divides with multiplies, change the sign of zero, constant folding, et al.

Common subexpression that does not break any other rules are always permitted.

• Forbidden: never ignore distributivity
• Forbidden: discard intermediate assignments
  • Can never replace an assigned value by a precursor with a different value
  • This includes that t = a*b can not be used unevaluated in an fma
• Strength reduction in floating point should be discouraged. E.g.,

x = i-- * 0.1; //vs
x -= 0.1;

Compile time constants

• Need a warning that whatever expression evaluation modes you have need to be extended to compile time arithmetic
• W.r.t. generic functions: What about traditional Fortran functions which are neither generic nor have known signatures?

Optimization level should be orthogonal from translation modes

Want to let programmers write generic programs

You might want to write something that can be run and rerun in higher and higher precision. The programmer has to choose those parameters that depend on some known precision and those which should be defined implicitly.

Convergence criteria may complicate matters. E.g., if you need to converge until the error is less than some constant. The problem is that people often don't know when they should have something that depends on some absolute or something that depends on the precision.

This can be as much a programming style issue as a numeric issue and language issue.

Generic functions turn up naturally in math libraries. Overloaded functions are a bit more restrictive than generic functions. For each data type there is a function and algorithm.

• Examples: Root finding, numerical integration, numerical differentiation, ODE solving

Many of the issues that come up in the discussion of "generic" programming arise because float and double are primitive types that aren't related to one another in some deeper typing sense. In Java float and double are primitive types where a float will be widened to a double automatically at a method call site. However, if you want to capture that information about widening in a language like Java for Kahan's elliptic integral example, you need a number of overloaded methods:

• double integrate(double, double)
• double integrate(double, float)
• double integrate(float, double)
• double integrate(float, float)

Even if the implementation of most of these methods is trivial, like delegating to another method, the API is still cluttered with these marginal methods and the number of methods needed grows exponentially with the number of arguments. Method overloading in Java allows the same name to refer to different (but hopefully related) operations.

In contrast, it would be feasible to construct a system where float, double, and precisions from the tower were just special kinds of "FloatingPointNumbers" so one would write a single method

FloatingPointNumber integrate(FloatingPointNumber, FloatingPointNumber)

and the implementation may test if the arguments were actually float or double. Likewise, the float array versus double array issue comes from relying on an implementation artifact for performance rather than relying on a more abstract interface. That is, from a programming interface perspective arrays are just a map from a set of contiguous integers to some other kind of value, say double floating-point numbers. If that were the only aspect of arrays being relied on the "conversion" of a float array to a double array would be very easy with an adapter object that would convert an element perhaps only when it was accessed.

Of course the extra levels of indirection in such a hypothetical programming environment are inimical to easily getting good performance for operations close to the metal. For 754R, I'd recommend trying to state more abstract desired capabilities of the programming environment.

Could be approached with templates which expand to include the actual types required. This is similar to the tower approach except that an instance of the template will define one base type. Should have some mechanism for double_of and extended_of that basic type. If you want to know if there is a type you could ask is_there(double_of(type)). Probably really want to know if it is native: is_fast(double_of(type)). Would like to be able to take something written for variable precision could be specialized to double, for example. (Would be nice to talk to the LAPack people: they have experience with Fortran and C, for each precision for real and complex.) Some inputs may be special: integer or narrower type for some parameter.

It is important to know what we want to abstract over.

Issues

• C doesn't allow function overloading
• C99 has some support but it isn't tightly integrated into the language
• Fortran 90 allows overloading. In fortran the convention has been to append a prefix.
• Arrays as inputs create a problem
  • Would want the elements converted on the fly
  • But what if you pass the small array to another routine
  • Could pass a datastructure which maps integers to floating point numbers

Would like to say something to the language community to attract their support for allowing programmers to write programs which are numerically generic.

Would like to have predicates at compile time to make the needed inquiries to select the appropriate algorithm.

Widest need expression evaluation

(Can be postponed to variable precision, don't want to talk about it in general)

• Requires that it scans its generic subprograms and pass this need out
• This is more demanding than is typical
• Much more complicated if types aren't known until runtime

Difficult to deal with languages which don't allow declaring types

Calculators and expression evaluation intended for human eyes should use decimal types.

Actually, most interactive floating-point calculations should be done in decimal.

Assignments

• a += b*c may contract to a = fma(b,c,a) if allowed

Meaning of exceptions and flags

Naming of formats

E.g., we could identify the properties

• Internal floating point: 13 bytes, radix 10 (decimal), Emax 384, 16 digits
• Internal fixed point: 8 bytes, radix 10 (decimal), 13.3 digits

13 to the left of the decimal, 3 to the right

• 82-bit extended: 16 bytes, radix 2 (binary), Emax 655385, 64 bits
• X87 80-bit extended: 10 bytes, radix 2 (binary), Emax 16383, 64 bits

Programmer should be able to request a type which has at least certain values for these proporties. Beyond they they should be able to make environmental inquiries to determin what they actually get.

Operations

min / max

Define the selecting min and max as:

minsel(a,b) := if (a<b), a otherwise b. In C this is (a < b)? a : b

maxsel(a,b) := if (a>b), a otherwise b. In C this is (a > b)? a : b

From these an application programmer can define other related operations with known semantics.

we can define a windowing function:

winnum (a,b,x) := minsel(b,maxsel(x,a))

If numbers are preferred over NaN:

minnum (a,b) := minsel(b,minsel(a,+inf))

maxnum (a,b) := maxsel(a,maxsel(b,-inf))

If nans are preferred over numbers:

minnan (a,b) := isnan(a) ? a : minsel(a,b)

maxnan (a,b) := isnan(b) ? b : maxsel(b,a)

If the extra-numeric treatment of +/-0 is desired:

if(a == b == 0), a|b, otherwise min(a,b)

if(a == b == 0), a&b, otherwise max(a,b)