Well, it’s used by adults in all English speaking countries now - watching British cooking shows, you’ll hear it all the time; so I guess they don’t consider it ‘baby talk’. At worst it’s just a relatively new slang word.
I reiterate: “Veggie” is babytalk; it’s evidence of the ongoing infantilization of American people.