Pagination not working fine after filter product collection

We are using Magento 2.4.3-p2 and filter collection on advance search page and quick search page using custom attribute.

Our collection is properly filter using our observer but tool bar count and pagination not work properly.

Can any one let us know how to fix it?

We have tested same code with Magento 2.4.4 as well but not working fine.

Thanks