Remoting does support stateful connections. You could use singleton or client activated objects to keep object state. This object state is not persisted however. If the server appdomain is unloaded all remoting state will be lost. Not only will the state be lost but client-activated objects will not be available. You will get a “service not found” exception if you try to use a proxy to a non-existent CAO.
If you have a recycle the server scenario, then you are required to use wellknown objects. Wellknown objects are always available when the endpoint is listening, so you need not worry about whether the server has been recycled or not. That said you have to carefull not to depend on any state within the wellknown object.