Retry removal of busy minors