England's three-wicket win on Saturday with just three balls to spare levelled the series at 1-1 meaning Monday's game at Southampton will be the decider.
Australia set England a target of 228 and the bowlers started brilliantly with McKay snaring the key wickets of Kevin Pietersen, Jonathan Trott and Joe Root with the first three deliveries of his second over.
The hosts were reeling at 3-8 after just 15 balls.
McKay said the loss was heart-breaking given the team could have clinched the series.
"(For me) the ball's been coming out quite well all winter ... and today once again a bit of luck was on my side," the 30-year-old said.
"But at the end of the day it doesn't mean too much when you don't take home the points."
The last Australian to bowl a hat-trick in an ODI was Daniel Christian against Sri Lanka in Melbourne in early 2012.
McKay is ranked seventh in the world one-day bowling rankings - the only Australian in the top 10. He finished Saturday with 4-39 off 10 overs.
After the initial shock of losing three quick wickets opener Michael Carberry and captain Eoin Morgan set about putting England back in the game.
They piled on 104 for the fourth wicket.
The right-arm quick from Perth took 1-34 off his 10 overs.
When Ravi Bopara was out lbw it looked like Australian might cruise to victory.
But Jos Buttler - who successfully used the Decision Review System to have an lbw call reversed at Watson's expense - and Ben Stokes had other ideas.
The pair put on 75 for the seventh wicket to steer England to within sight of victory.
Needing seven off the last over, Buttler smashed Mitchell Johnson's first ball for six and then the third for four to finish 65 not out.
"We needed to make some more runs," Australia captain Michael Clarke said.
"Then we couldn't get the last couple (of English players) out ... and we didn't hit the stumps with run-outs as well."
But Clarke insisted: "As soon as we get on that bus tomorrow we've got a final now so we look forward to playing it."
The captain praised Coulter-Nile, 25, as a very good athlete who could bat and field as well as bowl.
"He's certainly good enough to play at this level and I think he's got an extremely bright future in all three forms of the game," Clarke said of the right-arm quick from Perth who ended with 1-34 off 10 overs.
Whether Coulter-Nile retains his spot for Monday's match depends on the pitch conditions.
Australia's innings got off to a shaky start with Aaron Finch falling to the first legitimate ball and Watson a few overs later for six.
George Bailey, however, thumped 87 off 91 balls for his highest ODI score against England.
The Australian tally of 227 should have been a better after they'd fought back to be 5-209 with almost 10 overs to play.
But they lost their last five wickets for just 18 runs in the space of six overs.