Remember what Donald Knuth said, "premature optimization is the root of all evil".

advantages and disadvantages of various regression methods, including non-negative least squares, elastic net, weighted least squares, least absolute deviations, and nearest neighbors