How do I return a 404 page for pagination pages where there are no products?

My website product landing pages have the current pagination:

https://www.example.com/bread/sourdough
https://www.example.com/bread/sourdough?p=2

But if I put in a page number that isn’t in the series, like:

https://www.example.com/bread/sourdough?p=123123

Then I still get a live page with 0 products – essentially an empty page.

How do I adjust this to return a 404 when this is the case?

Thanks