The only way the Normans and the Welsh these days can readily communicate is through English. Back in the 500-700 period they would have both spoken the same set of Gaelic dialects. Somewhere about 900+ the folks in Normandy, Eastern Brittany and along the Loire, began speaking several Gallo dialects.
It looks like French, but it's not!
With the Welsh speaking two different Gaelic languages (one cognate to Irish and the other cognate to Breton), and the Normans speaking Gallo and French, they would be mutually unintelligible.
Bretons and Welshmen, however, do, at least in part, speak the same language!
Actually, I think that was the point of the book. The Gaelic connection. Also, if I remember correctly, they did say it was older Welshmen. It was several years ago that I read it. Good book. I keep meaning to re-read it.