If you factory defaulted it and it's still there, it's likely that you used the old add capability that existed whereby in certain situations it was possible to have the phone use an add method and DPMA would write an updated contacts XML file for the phone - provided that the file was sent to the phone using DPMA as a transport (not retrieved from HTTP) and that the file was in the file_directory and was writable by Asterisk and the edit flag was set for the contacts file and the phone had the use_local_storage option enabled.
Given the limitations of that capability its use was highly frowned upon and we've just removed support for it within the contacts app on the phone itself in the newest firmware. (I'd forgotten to remove mention of the use_local_storage option from the public wiki docs, which I'll now do).
So..go check the contacts.xml file for the phone and see if it's got the entry you'd like to remove. Otherwise, a factory default should wipe anything local to the phone itself.