The ability to create mailing labels from your customer list isn't available in QuickBooks Online. Besides, it's workable with the Customer Contact List report that contains all of the customer's information.
You can utilize the mini Gear icon to show additional columns like shipping and billing addresses.
Let me show you how:
Select the Magnifying glass icon on the upper right.
Enter "Customer Contact List" and then select the Customer Contact List.
Click the mini Gear icon below the Save Customization button.
Click the space beside the mini Gear icon to run the report.
Once done, you can print the report for reference, so you won't need to go to the customer's information.