If it is agell battery they can sow as fine but under load fail. I had the same on my CX.
How did you put 12v directly onto the starter? Whas it the same battery?
It could be also a poor earth. They may look good but disconnect & clean them.