Yes, you can delete an account anytime. Purging exchange wallets and transactions happen in two different ways depending on the usage scenario. To further explain this scenario, we will use Coinbase as a reference.
Scenario 1 - If you have two portfolios where both the portfolios have the same Coinbase account connected
In this scenario, if you delete the Coinbase account from on portfolio, the wallet balances and the transactions will no longer be visible inside this portfolio. However, the transactions are still stored in the database as there is another portfolio connecting to the said Coinbase account
Scenario 2 - If you have only one portfolio connected to the Coinbase account
In this scenario, deleting the exchange will purge the wallet balance and the transactions there are no other portfolios with the exchange connected.