Override phtml from third party extension in theme folder

I have an extension installed via composer and wish to make some small changes so it fits better within my theme. The file is located at:

/vendor/<Namespace>/<Module>/view/frontend/templates/file.phtml.

I understand I can copy this file to my theme and override it without making xml updates. I just can’t get the folder right where it should live.

I have tried following the official magento guide and other posts on here but it’s not updating:

/<theme_dir>/<Namespace>_<Module>/templates/file.phtml