If it bugs you get some cold blue... or even a sharpie XD sights are notoriously tough to get off and many XDs have marks in that area. I know how you feel though, I just about lost it when I found a scratch on my XD40 after having night sights installed at
Gander Mountain. Then I took 2 deep breaths and haven't worried about it since