Look at your analytics and see which browsers your users are using. Sometimes, I’ve worked on apps without a big userbase but we had to support IE because a big percentage used it.

You’ll probably find out that you don’t have too many browsers that actually matter.