As long as you have offered the the dealer the chance to fix it himself, its up to you to choose what to do next.
A court would take a dim view of you say charging £20k for repairs if the dealer had offered you your money back because it was uneconomical to repair. There are limitations within the legislation - you cant make someone repair something that is irrepairable or only repairable at a cost that far outweighs the value of the original transaction.
In your shoes, if the dealer doesn't want to fix it, best bet would be to get a fixed price offer for a repair from a specialist & get agreement from the dealer for a refund of that amount. Its better for you to get the money & pay for the repair, as you cant then get caught with the car not being released because the dealer hasn't paid the bill, & any warranty on the repair would be between you and the repairer. You need to be careful though that the refund will cover all possible work.
You need to get every thing in writing, if the dealer won't put something in writing, confirm what was said in a letter to him straight away.