Fulfillment Services are top level product groups. Fulfillment services should be used for 3rd party vendor product catalogs. Your own products are technically considered part of the "Manual" fulfillment service.
Product Groups are used to group products within a fulfillment service, or within your own product catalog. Each fulfillment service always has a "Default" product group which can be used to set rates for products within that fulfillment service. Products within a fulfillment service, or your own product catalog can be further sub-divided into product groups.
These product groups are at the core of the app's functionality. Each product group can be configured with its own unique set of shipping rates for each shipping zone.