I too found that the steepest descent method works well for fitting a curve since it’s hard take derivatives there are so many summations in the equations. I too programmed a way to do the steepest descent method, but unfortunately, trying to fit a circle using least squares was very difficult because a bad initial guess kept making my solution diverge.

I select the step size (h) by computing the variance for the current parameters (a,b,c), and for the modified parameters at -hu and -2hu. Then I fit those values to a parabola (a la Simpson’s method), and use the results to calculate a revised step size.

It seems to work fairly well, though I’m still fine-tuning the code.