When destructing the server it is stopped if not done so far. You should ensure that no client is active anymore.