Conv_forward Function: vectorized != loop version

The eps for binary64 floats is roughly 2 * 10^{-16}.

See this wikipedia page for 64 bit floats or the fully general IEEE 754 spec for more info.

Everything we do in floating point is an approximation and any change in the order of the operations can result in slightly different values. Here’s a thread that gives another example in which mathematically equivalent algorithms can yield different floating point results.

2 Likes