You're right. One of the transactions is already a duplicate, so you will need to remove one of them. You can just delete the deposit transaction since the payment is already in your bank register.
To avoid having duplicates next time, here's what you'll want to do.
Check out the video tutorials about invoices and payments for your additional reference.
That's it. If you have other questions in mind, you can always go back to this thread.