rlax | library built on top of JAX that exposes useful building | Reinforcement Learning library

Let f(a, b,…) be a function of signed objects a, b,… Let a', b',… be the values of those objects reinterpreted as unsigned. If, for all values of a, b,… (in their signed types), the reinterpretation of f(a, b,…) as unsigned equals f(a', b',…), then no sign extension of a, b,… is needed when evaluating f.

This is evident because, if the condition is satisfied, then f(a', b',…) produces the bits required to represent f(a, b,…). However, it may be incomplete. We might have some f(a, b) that requires a sign extension of a but not of b, and that is not directly addressed by the above. However, it could be considered included in that f(a, b) may be expressed as a function g_a(b), and then the above tells us no sign extension of b is needed when evaluating g_a. If this is true for all g_a, then evaluating f(a, b) does not require sign extension of b.

Also, the fact that a sign extension is not needed does not imply a compiler will necessarily detect this and generate code without sign extension. A compiler might generate sign extension even if it is not necessary. I think this may be seen in sint8fun; I would expect (int8_t) (a+b)*2 to be evaluable as an addition and a shift without sign extension. However, the compiler may be failing to account for the fact that the expression is converted to int8_t by the return. By itself, (a+b)*2 does require sign extension, as it could produce a negative int result that it would not if the signs were not extended. It is only after the conversion to int8_t that the result is then independent of sign extension.