98 Thunderbird Sport Starts When Button Released

Does the engine turn over when you hold the starter button in?

 

Sounds to me like the starter is drawing down the voltage enough that the ignition voltage is low until you release the starter button. You need to figure out why. Is the battery weak ? Is the starter drawing too much power ? Are the ignition coils the problem ? Could be a switch problem but, I don't think so. ...J.D.

 

I had exactly the same issue with a car, as soon as l released the key it would start, this is exactly the point where the starter stops drawing current.
In the 70's it was popular to fit a 6v coil to get over this low voltage on cranking problem.
As someone has already said,probably a battery or starter motor fault.