This is one of the items that is tested in your Windows 8 App when you use the WACK.
If you are in Southern California, please email these posts for more information on getting a token for your app by June 30, 2012!
All those user preferences and settings that users do not change often should be exposed only in the settings windows rather than throughout the app.
The guidelines for app settings has great insights into organizing your settings and what is appropriate for settings and what is not appropriate for settings. Review and implement this guidance to make your app easy to configure.