can it be deleted with no quirks?
Yes, but Windows will re-create it.
I have declined all Microsoft attempts to get me to take my "free" Windows 10. The folder is still created.
At this moment, the $Windows.~BT on my main computer is more than 5GB in size.