How was he contacted in order to send him to Washington? It seems to me it would have taken a long time to get a message to him, and then it would have taken him some time to get to Washington.
Read Freeman's biography. Lee was ordered back to Washington on February 4th with orders to report to the general in chief no later than April 1st. He was back home by March 1st.
As for how he was contacted? By telegraph.
Then the claim that he was in Texas on April 11th is incorrect. That would certainly explain things.