If you have Java installed then DocFetcher is a good lightweight utility. It will do what you want after you set the folders to be indexed.
https://docfetcher.sourceforge.io/en/index.html
But the greatest of them all is Recoll. It’s the graphical front end for the Xapian indexing engine. Looking at the download page I see that he asks for a small donation for the Windows build (5 EU) but if you cannot afford it or cannot use Paypal he will give you a download link for free. It is by far the best indexer out there.
Windows 11 has a search box in the upper right when you open File Explorer (it used to be called Windows Explorer). If you don’t see it there must be some option to make it visible which you have to set. I would go nuts if I didn’t have this, I use it all the time.