How can I get a report on a category by category basis, showing all the items within each category?

It's good to have you here, @TomR233. 

Let me help you how to get your report by category.

Here's how: 

1. Go to Reports. 
2. On the search bar, type Transaction detail. Then choose Transaction Detail by Account. 
3. Click Switch to classic view. 
4. On the top right, click Customize.
5. Under Rows/Columns, click Change Columns. Then tick the Product/Service.
6.  Under Filter, tick the Transaction type. 
7. You can choose what Transaction type you want by clicking the drop-down arrow. 
8. Click Run report.

If you want further customization, you can export your data and do it in your Excel. 

Moreover, you can also visit this site to explore more about customizing reports: Customize reports in QuickBooks Online

Feel free to ask again if you have another concern regarding customizing a report. Enjoy your day!