When I was on the JFK, the Soviets had smaller ships, trawlers tracking us whenever they could. I remember once standing on the fantail and seeing a Soviet trawler trailing us a few miles back. I had a pair of binoculars (there were no flight ops going on right then) and as I viewed them, I saw implacable, motionless, silent groups of men staring back from their bridge, hands holding binoculars, black disks where their eyes would be.
We were standing behind our last plane spotted on the flight deck and couldn't be seen from the island, so a bunch of us lined up and dropped our dungarees to expose our backsides to the enemy...:) When I looked again at them, they were motionless, still staring in what seemed like a disembodied machine-like way. I often wonder what they thought of that display...did they laugh? I know I would have if they had mooned us.
When we were about 400 miles off the coast of Ireland (I think) heading up the the Artic Circle, we had a Tomcat malfunction and go off the side of the carrier with a Phoenix on board. Unfortunately, there was a Soviet cruiser several miles in front of us that saw it happen, and they began to cut across our bow to get a better look, and one of our vessels in the area cut in front of it. However, they now had a fix on where the plane was, so our country had to engage in an all out effort to salvage the plane so they couldn't get their hands on the Tomcat or the Phoenix. 
LOL, I was a new Plane Captain, sleeping in my plane spotted along a line parallel to the waist catapult, facing the island, when I was awoken as the full throttle exhaust from the runaway Tomcat played across my plane jerking it violently from side to side as the Tomcat dropped off the deck behind me. I turned in time to see the twin tails going over the side and people running over to see it hit the water.
Several times during my deployments, we had a Russian Bear (Tu-95, big propeller driven plane) fly towards us, with two Tomcats flying in formation with it, gently shepherding it away from the carrier.
I understand what went on between smaller ships like the Soviet destroyers and subs and their American counterparts was far more confrontational and aggressive. Not nearly as close to Cold War as hot war.
It is thought that the Soviet sub raised by Howard Hughes' Glomar Explorer had been sunk in a collision with one of our subs that had tried to surface with the Soviet sub right above it, which is how we knew where it was, but the Soviets didn't. (I read something that the sub limped into Yokosuka a week or two later with its conning tower, scopes, and antennae crushed and distorted, where it was immediately covered with tarps. At least that is one account I read, I can't remember if that was accurate or supposition.