pbrt-v3 | Source code for pbrt , the renderer | Graphics library

To inline a function call, the compiler has to know 1. which function is called and 2. the exact code of that function. The whole purpose of virtual functions is to defer the choice which function is called to run-time, so compilers can obtain the above pieces of information only with sophisticated optimization techniques that require very specific circumstances¹.

Both templates and virtual functions (i.e. polymorphy) are tools for encoding abstraction. The code that uses a CoefficientSpectrum does not really care about the implementation details of the spectrum, only that you can e.g. convert it to and from RGB - that's why it uses an abstraction (to avoid repeating the code for each kind of spectrum). As explained in the comment you quoted, using polymorphy for abstraction here would mean that the compiler has a hard time optimizing the code because it fundamentally defers the choice of implementation to run-time (which is sometimes useful but not strictly necessary here). By requiring the choice of implementation to be made at compile-time, the compiler can easily optimize (i.e. inline) the code.

^{1For example, some compilers are able to optimize away the std::function abstraction, which generally uses polymorphy for type erasure. Of course, this can only work if all the necessary information is available.}