If I need to get a web service up and I can save 4 minutes by setting a low TTL when I configure my DNS record why wouldn't I?

Because you're pushing the cost on to someone else.

If your DNS hosting provider charged you per query (some do, especially when adding features like health checks & load balancing), then it might make a big difference.