Remove Transfer-Hook from my token

I am thinking of methods to write a whitelist functions on a token. Is there a more efficient way to do this, as I want only specific wallets to be able to buy/send/transfer my tokens, I see that there is a way to do this via transfer hooks.

So I have like another yes/no question, after using the transfer-hook function to whitelist my token to specific wallets, is there a way to now cancel this function from off my token ?