Android had a hidden feature in certain iterations of Android 4, called "App Ops," that would let you do this but it never made it to prime time. Though Cyanogenmod does leverage it for their "Privacy Guard" feature.