"Before that, let's make sure that the Import Style feature is turned on. Go to the Gear icon. Select QuickBooks Labs under Your Company."
This has nothing to do with importing products and services items into QBO. Enabling Import Style feature from the QuickBooks lab is able to import the DOCX template for the sales form like Invoice type.
I’m importing my existing inventory for the first time and I’m not sure how or if I need to show cost of the items so then I can process sales of the items in the future?
If you're importing inventory items with quantity on hand, you need to enter the COST field. This is your actual cost, you paid buying these items. It will create a transactional journal, adding value to the Inventory assets account and the other side goes to Opening Balance Equity account.
If you don't quantity on hand and you're importing the product/services items list, you can even enter "0" (or estimated cost) in the Cost field. Your actual Cost will pick-up when buying inventory from vendors using the Item details tab.