Upper chain guide removal? Should I?

[jdkruger4]

I took my valve cover off to look at the upper chain guide cause something is clunking on startup and at idle and i thought it might be hte culpret. Well the bolts holding it are still tight but i can easily push the chain up onto it as there is only 1-3mm clearance between them. Should i go ahead and remove the guide? Or should i leave it on? Thanks!