Ryan Singer wrote about this back in 2008.
When you add a feature to a product, you should be committed to that feature.
I had a situation this week where a web host performed some upgrades, and as a result, some of the lesser-known features of that web host were removed. The upshot was that I could no longer use their business.
Not every customer will use every feature, and not every customer will use the same features. If one feature of your product is used by 1% of your business, you will inflict pain on them by removing it.
Don't commit to a feature if you are unwilling to support the 1%.