Interesting that he also says the same thing to his brothers later in Chapter 7 when they urge him to go to Jerusalem for the Feast.
But in Ch. 2 and 7 he does both anyway.
What's this to mean? That his Mother and brothers knew better than Jesus?
No. Rather it's likely that he had to correct them that his "time" would not be until his death on the cross. They were pushing him to take a leadership position on earth; he was telling them that his leadership would not really transpire until his death on the cross. He did the works they asked him to but they were erroneous as to the meaning of his work.
**he also says the same thing to his brothers**
Silly you, those are really cuzzinz, antz, an uncullz. (sarc)
What brothers? Jesus was an only child.