England great Gooch, the team's batting coach on their woeful 2013/14 tour of Australia, told Britain's Daily Telegraph on Wednesday that the former off-spinner's decision had been highly damaging to the team.
"It made us a laughing stock," said Gooch, one of England's most successful opening batsmen. "I cannot understand why he couldn't stick it out until the end of the trip. It left a bad taste."
Swann quit, with England 3-0 down, in December 2013 after deciding a longstanding elbow injury meant it was no longer possible for him to spin the ball properly.
"I understand (what he is saying)," Swann told several British newspapers. "Goochy is very old school. "Perhaps if he had gained my perspective on it he might have seen the reasons why I did it. I think he is a bit misinformed thinking I just left because of form. It wasn't because of form.
"It was because I just couldn't turn a cricket ball, which, as a spin bowler, means you are useless to your team. It wasn't a form thing, it was succumbing to the inevitable."