Removing the Oregon State from your payroll setup isn't possible, RGRandall. I'm here to provide some helpful insights on this matter.
Once you set up a state tax in your payroll setting, you cannot delete or remove it. While this option is unavailable, you can keep them as they are.
Also, your action to exempt your employee from tax was appropriate since you didn't withhold taxes from them. To better guide you in setting up your employees, you can open this link: Set up employees and payroll taxes in a new state.
Additionally, you can open this page to view useful info about your business and employees: Run payroll reports. It also contains steps to run, print, and customize payroll reports.
If there is anything else I can assist you with, or if you have any further questions about payroll, please let me know. I'll be glad to help you further. Have a wonderful day!