Yes, you'll have to create a journal entry to finalize the correction process of the accounts, tjspainting.
The changes on the Accounting preferences in the Payroll settings only apply moving forward. Hence, you'll have to create a journal entry to correct the affected accounts. You can use this link for reference, too: Manually enter payroll paychecks in QuickBooks Online.
Regarding to what accounts need to be debited or credit, I'd recommend reaching out an accountant. They know how to handle this kind of scenario. You can run payroll reports to see the accurate amount that needs to be corrected.
Again, we'd recommend asking your accountant to guide you in creating the journal entry. They know what's best for you and your books.
Keep us posted about this so we can further assist you in correcting your payroll data.