The image of the Sydney-raised boy posing with the rotting head of a soldier, posted on the Twitter account of his father Khaled Sharrouf, an Australian who fled to Syria last year and is now an Islamic State fighter, sparked outrage.
Prime Minister Tony Abbott and US secretary of defense Chuck Hagel said it showed how barbaric IS militants had become, while Australian Muslim leaders expressed widespread revulsion.
The seven-year-old boy's grandfather Peter Nettleton, who is estranged from his daughter Tara, Sharrouf's wife, begged the government to help bring the boy and his siblings home.
"I'm scared for the children. What life are they going to have now," the Sydney truck driver told Sydney's The Daily Telegraph.
"Can't the government do something to pull these kids away from that man?
"That (picture) brought me to tears because I don't know how to handle it."
Another photo published by newspapers in Australia showed Sharrouf dressed in camouflage fatigues posing with three young boys believed to be his sons.
All were holding guns in front of the flag of the Islamic State militants who have swept across Iraq and Syria, seizing swathes of territory.
Australia has an arrest warrant out for Sharrouf, who fled the country last year using his brother's passport after serving almost four years in prison when he pleaded guilty over a 2005 conspiracy to attack Sydney.
Nettleton, whose daughter cut off ties with him when she married and converted to Islam, said he thought his five grandchildren -- three boys and two girls -- were staying with Sharrouf's sister in Malaysia while he fought in Syria.
A friend of Sharrouf's, Mohamed Elomar, who is also fighting in Syria, reacted to the photo of the boy with a tweet to "keep them heads rolling", the Telegraph reported.
"What a flaming ripper, ayyy beauty mate, love it, keep them heads rolling," it reportedly said.
Officials have said up to 150 Australians are fighting alongside militants overseas, mostly in Iraq and Syria. Many other countries also have nationals in the war zones.
Foreign Minister Julie Bishop said jihadists whose "barbaric ideology" threatened Australia's way of life would be discussed at security talks with US Secretary of State John Kerry in Sydney on Tuesday.
"Countries including in Europe, in Southeast Asia, in north Asia are concerned about citizens... fighting overseas in Iraq, in Syria and other conflicts, returning home as hardened, home-grown terrorists, extremists, capable of carrying out terrorist activity in our own countries," she said.
In brief opening remarks at the annual Australia-US Ministerial Consultations (AUSMIN), Kerry agreed that "we do face new challenges", pointing to foreign fighters who "learn jihadism and come back with bad intentions" as an issue of concern.