If it’s a server, it should default to not being accessible via remote desktop.
Have you enabled it (usually in the remote tab of computer properties, IIRC).
Also need to open up Windows Firewall if in use.
check system event log; off start menu if server, or control panel-administrative tools->event viewer if win 7/8
its bound to have logged something.
RD is turned on and Firewall was turned off and then turned on with port (I think 3389) open.