A rare mob will spawn by the name of Percival that will quietly appear behind you and attack when you least expect it.

The simple way of getting rid of him is by hitting him a few times when you see him before he attacks. He despawns after 30 seconds when he initially spawns.