for a data parallel problem like that you could vectorize it using the SIMD unit. It never ceases to amaze me at any point how much of silicon real estate just sits idle because no one really bothered to look behind the curtain and tailor the code for a specific architecture. for reference, a recent Intel chip will have 256b wide vector unit (512 for server class chips) that can be treated as a vector of 8/16/32/64b units to do parallel math. at a typical 16b ops you get 16x speedup for practically nothing. plus its considerably more if you have predictable but non standard data access patterns.
now <rant> IMO what the parent commenter was trying to say is often this punchline about premature optimizations used as catch-all justification for all the convenience/expediency related decisions taken. & also it just works well to create future work. unfortunately that never gets scheduled as I have seen management time frame shrink too. almost everyone is in quick & dirty 'working' version to book a success and move on. with this culture if you try to push resolving these 'technical debt' type issues you are just perceived as a downer. </rant>
I've heard that modern processors are designed with the expectation (or even a requirement) that no more than 10% of the chip will be active at once. In addition, there are power density constraints (even those 10% of the chip that can be active at once must not be concentrated in the same spot).
Nope, but processors, especially larger ones, have pretty advanced power and clock management which allows them to deal with power envelope and thermal envelope constraints. E.g. the two clocking and core voltage ladders in Xeon E5/E7 for AVX vs. non-AVX code. (Though this particular example is a bit of a sledge-hammer approach).
now <rant> IMO what the parent commenter was trying to say is often this punchline about premature optimizations used as catch-all justification for all the convenience/expediency related decisions taken. & also it just works well to create future work. unfortunately that never gets scheduled as I have seen management time frame shrink too. almost everyone is in quick & dirty 'working' version to book a success and move on. with this culture if you try to push resolving these 'technical debt' type issues you are just perceived as a downer. </rant>