A curser gesture towards the upper right corner will give you the 4(or 5,6,etc.)again.
Yep, you have to keep track of windows in a 3D space. Not exactly the most user-friendly idea.