The Supreme Court on Monday struck down a provision of the law that limited the president’s authority to remove the head of the Consumer Finance Protection Bureau (CFPB).

The court held that Congress has gone too far when it created the CFPB and put it under control of a single director who could only be removed by the president for “inefficiency, neglect of duty, or malfeasance in office.” That gave rise to an office exercising executive power too unaccountable to the president, the high court ruled.

The 5-4 decision was a victory for conservative attempts to pare back the administrative state. The bureau was the brainchild of Elizabeth Warren, who first proposed it in 2007 when she was a Harvard law professor, and was erected by the Dodd-Frank Act Congress passed in 2010. President Barack Obama initially considered naming Warren, by then serving as an assistant to the president and adviser to the Treasury secretary on the agency’s creation, to head the bureau, but she was dropped when it became clear her nomination could not overcome fierce Republican opposition.

In an effort to shield the bureau from political influence, the Dodd-Frank Act gave the bureau’s head a five-year term that a president could only foreshorten for reasons of “inefficiency, neglect of duty, or malfeasance in office.” In a 2011 Wall Street Journal op-ed, Sen. Richard Shelby lambasted the structure for its lack of accountability and said the agency should be run by a board or commission, as most other independent agencies—such as the Securities and Exchange Commission and the Federal Deposit Insurance Corp—are.

“The agency may … continue to operate, but its Director, in light of our decision, must be removable by the President at will,” Chief Justice John Roberts wrote for the majority.

The decision may disappoint some of the CFPB’s opponents. The court left the agency largely intact, declining to rule that the flawed structure meant the entire CFPB and all the regulations needed to be shelved. In a separate opinion, Justice Clarence Thomas argued that the court was wrong to sever the removal clause from the rest of the statute and should have pronounced the agency unable to act until Congress remedied the structure flaw.

The case was brought by a law firm that objected to a CFPB investigation of its debt relief related practice. The firm said the agency lacked the authority to conduct the investigation because its director was too removed from presidential control, violating the constitution’s separation of powers requirement and the mandate that executive power of the federal government rest with the president.

Conservatives have long criticized the CFPB on those grounds. In a different case, PHH Corp. v. Consumer Financial Protection Bureau, then judge Brett Kavanaugh wrote a majority opinion for the U.S. Court of Appeals for the D.C. Circuit that declared the structure unconstitutional. That decision, however, was overturned after a review by an en banc panel of that court. Monday’s decision, joined by Justice Kavanaugh and four other Republican-appointed justices, vindicated Kavanaugh’s opinion.

Congress could act to amend the law to create a board structure if it does not want the agency to be directly answerable to the president.

Kathy Kraninger, the current chief of the CFPB, has said she believed the president could fire her at any time. She was nominated in 2018 by President Donald Trump, replacing Acting Director Mick Mulvaney. The agency had initially argued that its structure met constitutional requirements but switched stances under the Trump administration after former Director Richard Cordray resigned in 2017.

The decision likely means that the only other similarly structured agency, the Federal Housing Finance Agency, is also unconstitutional. The regulator of Fannie Mae and Freddie Mac was established in 2008 and its director is also only removable for cause according to the statute. That puts the tenure of the current director, Mark Calabria, into doubt, possibly hampering plans to bring Fannie and Freddie out of conservatorship and sell them to private investors.

The case is Seila Law v. Consumer Financial Protection Bureau, No. 19-7, in the Supreme Court of the United States.