Wrong pagination for a filtered product collection

Hello
I used this event to remove duplicates from my product collection.
it is showing the correct items, but the wrong way.
Any idea how to correct the following:

• show correct items count (256 is the wrong items count for this filtered collection)
• show correct pagination (it shows 1-9 pages for 256. 30 items/page)
  but we only have 22 in our case.

public function execute(
            MagentoFrameworkEventObserver $observer
        ) {
            $collection  = $observer->getEvent()
                ->getCollection();
            $variations = [];
            foreach ($collection as $_product) {
                $variationGroupId = $_product->getVariationGroupId();
                if (!empty($variationGroupId)) {
                    if (in_array($variationGroupId, $variations)) {
                        $collection->removeItemByKey($_product->getId());
                    }
                    $variations[] =  $variationGroupId;
                }
            }
            return $collection;
        }