Changing the permanent delegate of a token mint?

I’m using the Permanent Delegate extension of the Token-2022 Program. Is there any method to change the permanent delegate? Or to remove it entirely? I’m trying to get a sense for how permanent this delegate really is so that I can decide if I want to build with it.