They say the responsibility lands fairly and squarely with under-fire umpire Aleem Dar, who inexplicably missed a giant nick from Broad's bat that presented Australia captain Michael Clarke a catch at first slip late on day three of the opening Test at Trent Bridge.
"The Australians I have played with and have watched, with the exception of (former wicketkeeper) Adam Gilchrist, believe in standing and it's up to the umpire to give you out - there shouldn't be a moral argument," legendary batman Geoffrey Boycott said.
"They should be upset, disappointed and angered by the umpires. If they keep making poor decisions, it's up to the ICC to do something about it."
The astonishing blunder again called into question the point of the decision review system, with Australia unable to appeal to the third umpire having already burned their two challenges.
"The review system was brought in to get rid of the howler," former England captain Michael Vaughan said.
"I don't see why umpire Dar couldn't have had someone is his ear saying 'you've got that one wrong let's just overturn that quickly'."
Champion Australian bowlers Shane Warne and Glenn McGrath said Broad was well within his rights to stand his ground - taking the opposite stance of their former teammate Gilchrist, who famously gave up his wicket in a World Cup semi-final against Sri Lanka.
"To me, it has to be the umpire [at fault]. A player is allowed to stand his ground," McGrath said.
"If Australia had one appeal left, Broad would have walked.
"The fact there was no referral left, he left it up to the umpire. It was a poor miss from Aleem Dar but Australia can't stew on it."
Warne, who has been critical of Dar in the past, went a step further - calling for the 2011 ICC umpire of the year to be sacked.
"He (Dar) always gets the crucial decisions wrong & always has, that's why he's not a great umpire," Warne said on Twitter.
"We all make mistakes & it's a very tough job being an umpire, but when Dar continually makes crucial mistakes why does he keep getting a gig?"