Howard Sternis calling outSimon CowellfollowingGabrielle Union‘s controversial departure fromAmerica’s Got Talent.

On Monday’s episode ofThe Howard Stern Show, Stern, 65, slammed Cowell —America’s Got Talent‘s creator and judge — accusing him of having “orchestrated” Union’s firing.

“How is it that Simon Cowell has orchestrated this?” Stern said. “He sets it up that the men stay, no matter how ugly they are, no matter how old they are, no matter how fat they are, no matter how talentless they are.”

Stern himself was a judge onAGTfor four seasons from 2012 through 2015.

“But what he manages to do on all his shows is he constantly replaces the hot chicks with hotter chicks and younger chicks. Which is so obvious,” Stern added.

In addition to Union andJulianne Hough‘s recent departure from the show, other female judges who have appeared onAGTinclude Brandy Norwood, who exited the show after its first season;Sharon Osbourne, who served as a judge from season 2 through season 7; and Mel B andHeidi Klum, who were judges from season 8 through season 13.

Of the male judges who have been judges onAGT,Piers Morganwas on the show from season 1 through season 6 and David Hasselhoff from season 1 through season 4.

Meanwhile, Cowell and Howie Mandel will both return as judges for the upcoming season 15, with Cowell having joined the series as a judge in the show’s 11th season and Mandel, 64, having started appearing as a judge in season 5.

A rep for Cowell did not immediately respond to PEOPLE’s request for comment regarding Stern’s remarks.

Debra L Rothenberg/FilmMagic; Gregg DeGuire/Getty; Frazer Harrison/Getty

Howard Stern, Simon Cowell, Gabrielle Union

“We are working with Ms. Union through her representatives to hear more about her concerns, following which we will take whatever next steps may be appropriate,” the statement concluded.

In her tweet, Monique outlined what makes for a “solid apology”: “1. Sincerely admit wrong doing directly to the offended party. 2. Be twice as loud correcting your mistake as you were making the mistake. 3. Lay out steps to correct your behavior in the future. Anything less is covering your own a–.”

Union retweeted Monique’s post, adding, “This! This! AND THIIISSSSSSS!!!!!!”

Union and Hough, 31, were also both subject to “excessive notes” on their physical appearance, sources alleged inVariety‘s report.

Trae Patton/NBC

Gabrielle Union, Julianne Hough

Representatives for Hough, Union and Leno have not responded to PEOPLE’s request for comment.

BlogLove B Scottwas first to report about complaints Union had allegedly made prior to her departure, with an NBC source saying that she had spoken out about “problematic” situations on set.

Unionbroke her silenceon her controversial departure last Wednesday, thanking her followers for their support during this difficult time. “So many tears, so much gratitude,” she tweeted. “THANK YOU! Just when you feel lost, adrift, alone… you got me up off the ground. Humbled and thankful, forever.”

source: people.com