You can now undo the placement of points when building a new curve-on-surface. Use + Z or Edit > Undo to remove the last point. Use Edit > Redo to put it back.
As with any other operation, the maximum number of undos is set to 11 but can be modified in the Miscellaneous section of Preferences > General Preferences.