How many icons do you have on your desktop? That is one thing that will slow down an OSX Mac. Too many and it can slow things significantly. Keep them to just Aliases of folders elsewhere on your hard drive and the problem will go away.
Also, leave your Mac on overnight. The Mac’s UNIX system does housekeeping tasks such as that memory recovery that PAengineer mentioned and defragging and optimizing of the hard drives when the system has been idle for a period. . . but if you turn it off instead of just letting it go to sleep, these never get done. Leave it turned on.
Thanks