I made the pilgrimage to Walsingham and highly recommend it to all Catholics. Of course there is not much left of the old abbey. Westminster Cathedral Choir also has a great CD of hymns to Our Lady of Walsingham.
I think I saw that CD in one of the religious stores in Little Walsingham. I got a CD of the London Oratory choir, and it's great! I also heard them streamed online off BBC.