You certainly need to remove the inner metal retainer, as well as the first metal sleeve before installation
here is what you should see (from the top) in regard to whats left before using ES bushings. You want to leave the bottom 'flange' as well when installating them.
