There are no take-backs when it comes to conference expansion. Once you add teams to your league, you’re stuck with them until further notice.

That’s the unfortunate reality right now for the Big Ten, which would be justified in feeling a small measure of buyer’s remorse for Maryland and Rutgers, two schools that joined the league during its last round of expansion in 2014 and have spent much of this season making the wrong kind of headlines. The additions were designed to increase the league’s TV revenue potential and expanding its footprint to the Washington D.C. and New York/New Jersey markets—a logical business move at the time, although cord-cutting threatens to erode the long-term gains. But commissioner Jim Delany took criticism for other aspects of the move back then, and the decision surely doesn’t look much better now.

Let’s first take Maryland. Since joining the Big Ten, the Terps may not have upended the football power structure, but they have had their moments, beating Texas in consecutive years and making two bowl games (losing both). The men’s basketball program has been to three NCAA tournaments; the women’s team has won the conference three times and made it to the Final Four. The men’s and women’s lacrosse teams are ranked No. 2 and 3 in the country right now.

The athletic department’s off-field failings have overshadowed all of that. Last week Maryland completely botched the firing of head football coach DJ Durkin to cap a string of mismanaged decisions following the death of Jordan McNair in early June after the redshirt freshman had collapsed during an offseason workout. One day after reinstating Durkin after nearly three months of administrative leave while his program’s culture was under investigation, the school decided (with a gentle nudge from people at every corner of the college football world) that it had made the wrong decision and fired him.

The commission placed responsibility on Durkin for a culture where players were intimidated, abused and discouraged from speaking up, but its report of findings did not go as far as labeling the program’s culture “toxic” as the initial ESPN report had stated. (This all happened after a separate investigation had concluded with sharp criticisms of the care of McNair, identifying staffers’ numerous failures to follow proper procedure.) The Board of Regents initially thought that Durkin could return to his position as if none of this would be an issue moving forward in recruiting or within the locker room. Ultimately, they embarrassingly reversed course after overwhelming backlash.

Meanwhile, Rutgers has never been able to hold up its football program as a standard bearer for success, aside from 2006, when it started the season 9–0 under Greg Schiano and went as high as No. 7 in the AP poll. This year, the Scarlet Knights sit at 1–8, winless in the Big Ten with a lone win over Texas State and a 55–14 loss to Kansas. In 2016, they were shut out four times: 58–0 by Ohio State, 78–0 by Michigan, 49–0 by Michigan State, and 39–0 by Penn State. In ’15, former head coach Kyle Flood served a three-game suspension for illegally contacting a professor, five players were arrested and dismissed from the team after being charged with violent crimes, and a starting receiver was suspended for allegedly slamming a woman’s face into the ground.

Non-revenue sports have struggled, too—before the men’s lacrosse team held the top spot in the Inside Lacross Media poll during the 2017 season, the last time Rutgers was ranked No. 1 in any sport was 1995, when the men’s soccer team topped the rankings. Suffice to say, the school’s conference affiliation has not changed its athletic fortunes.

These program shortcomings don’t belong in the same conversation as the headlines the football team made last week, when walk-on linebacker Izaia Bullock was charged in a plot to murder two members of a friend’s family. He faces two counts of first-degree attempted murder and two counts of first-degree conspiracy to commit murder. Needless to say, he has been dismissed from the program. Rutgers coach Chris Ash said in a statement that he is “thankful that no one was hurt” and that Bullock “demonstrated the positive behaviors that we ask of all our student-athletes. He was a good teammate, performed well academically and participated in several community service events.” But over the last two weeks, staffers witnessed changes in the player’s behavior, made him aware of counseling services and, once they became aware of the issue at hand, got authorities involved. The program handled Bullock’s situation as well as it could have, but the incident opened the door to more jokes at its expense.

Before joining the Big Ten, Maryland ran up a $21 million program deficit and was bailed out by the conference upon joining the league. Rutgers is still in mounds of debt, tallying up a program record $47.3 million deficit in the 2016-17 fiscal year.

These are the schools the conference added four years ago. But then again, its entrenched members have had their fair share of football and non-football scandals or varying scope and severity, from Jerry Sandusky (Penn State) to Larry Nassar (Michigan State) to Zach Smith (Ohio State) to Kevin Wilson (Indiana) and Tracy Claeys (Minnesota). The league’s public image arguably would be no more pristine if it had remained at 12 teams.

Through some unfortunate headline timing, Maryland and Rutgers have been a source of considerable embarrassment lately. But then again, who is the Big Ten to judge?