Yeah, I finally figured it out. It was a GST/HST tax credit (for over payment) I owed them in May, when they sent me the first notice. I paid it immediately, logged in a few days later and saw that the section of the MyCRA website labelled account balance was 0.00 so I figured it was fine. Got the second notice this week, paid it again. Then I noticed there's a separate section on their website where they put your GST/HST balance (entirely separate from the account balance) that still had a balance in the same amount that they sent both notices for.
Found yet another section where they showed that there was one, single GST/HST payment owed dated to May (when they sent the first notice). In an additional section where they list your account balance payments, they showed that I had a negative balance with them (they owed me) twice the amount I owe them for GST/HST, showing both payments, in May and this week.
So tomorrow I have to call and ask them to move one of the payments to my GST/HST account, and then ask them to refund me the amount I overpaid.