Title
Role of NaN and infinity in §[linalg] is unclear
Status
new
Section
[linalg]
Submitter
Jan Schultke

Created on 2026-11-09.00:00:00 last changed 6 days ago

Messages

Date: 2026-11-09.00:00:00

The handling of NaN and infinity inputs and intermediate results throughout [linalg] is not clearly specified. To name some examples:

  • [linalg.algs.blas1.scal] states that `std::linalg::scale` computes an "elementwise multiplication `αx`", but it's unclear whether this must be exactly the `*` operator. If it is a multiplication in the mathematical sense, how can this requirement be satisfied by `std::linalg::scale` if any NaN results or is input into a multiplication?

  • [linalg.algs.blas1.iamax] states that `std::linalg::vector_idx_abs_max` returns "the index of the first element of of `v` having largest absolute value". It is unclear whether a NaN is disqualified in this comparison (the `std::fmaximum_num` approach) or results in an unspecified index being returned. It is also unclear what element has the largest absolute if every element is NaN, since there exists no element that compares greater or equal to any other element. It is also unclear what the absolute value of NaN is. Is this "absolute value" the same as `std::fabs` in this description, meaning that NaN is its own absolute value?

The problem is not limited to just these two functions; there are fundamental questions such as:

When we describe results throughout [linalg] mathematically, do we mean real number arithmetic (no infinities), extended real number arithmetic (infinities but no NaNs), or a (sometimes unspecified) series of expressions ([expr])?

Possible solutions

  • Clearly specify for every function in [linalg] what happens when infinity or NaN is used as an input or when an intermediate calculation produces infinity or NaN (paper needed).

  • Make it undefined behavior to pass infinities and NaNs to <linalg> functions and state that any NaN or infinity produced in an intermediate calculation results in undefined behavior.

  • Something else.

History
Date User Action Args
2026-11-09 00:00:00admincreate