If ever happens I will have to remember to sift through the ashes and remove the USB charger

Actually, on thinking about it, I imagine it would not be that easy to find given that area is mainly wood and plastic.
I haven't checked but I would think all the circuits are fused and I doubt a 15mA load would cause any problems.