Starting issues

[95se]

Just replaced starter with another O'Reilly re-mand one that has lifetime warranty. They said battery was ok. Put it in and it started ok couple of times then click click like before but now the radio turns on when I stop cranking it though the acc on eclipse stereo is turned off so it shouldn't be getting any power at all. Its like the battery has been disconnected and reconnected. U can hear the fuel pump getting power and lights work normal. I do have a bad valve cover leak for long time and my cel keeps coming on because of o2 sensor but replacing didn't affect it. Still throws a code even after computer has been cleared. Stock ka no mods