If you are setting up a 32 bit SharePoint farm, do not turn on the /3GB switch. If you have custom web parts that consume a large amount of memory, or some other need for more memory, you should build a 64 bit farm.
For more info: http://support.microsoft.com/kb/933560/en-us
If you never have a significant load on your SharePoint environment, you may never experience any problems with the /3GB switch turned on. But under load you may see things like:
- users get IE page not found error
- authentication prompts that result in an IE page not found
- application pools restarting unpredictably