Are you drilling with a twist bit in there?
I'd imagine something like this would likely require removal mounting it in a jig so it didn't move and using a carbide tipped steel die drill.

https://goo.gl/KllomK

I've not used one of these in a hand drill, so I don't know if you'd be able to maintain pressure and keep it dead on so as not to damage the bit.

Just imagine the joy if you manage to get that twist drill deeper in there and it totally snaps off in the hole.