To delete a specific address, simply click delete address on the Approved Addresses page under settings.

To fully disable the approved addresses functionality on your Gemini account, you need to contact Gemini support. As you can see in the screen shot below, users can't disable the functionality on their own and need to reach out to the Gemini support team.