"The Union Cabinet has unanimously passed the matter. We have done whatever could be done in the matter," Azad said.
The Cabinet has already approved the proposal for sending a presidential reference to the Supreme Court on the issue, which was seen as a step towards his removal as the West Bengal Human Rights Commission (WBHRC) chairman.
"I am not aware of any such thing," Azad said when asked about allegations that Ganguly was framed in the case.
Justice (retd) Ganguly, who faces charges of sexually harassing a law intern, on Monday said he was yet to take a decision on resigning from his post in the WBHRC.
He also declined to comment on his telephonic conversation with former attorney general Soli Sorabjee.