WARSAW, Poland (AP) — Poland’s president came under heavy criticism from political opponents Wednesday after he called the European Union an “imaginary community” of little relevance to Poles.

Andrzej Duda is aligned with the ruling Law and Justice party, which has been in conflict with the EU over an overhaul of the Polish judicial system which Brussels sees as violating the rule of law.

“When our affairs are resolved, we will deal with European affairs,” Duda said in a speech on Tuesday. “For now let them leave us alone and let us fix Poland, because this is the most important thing.”

Duda and government officials insist that the changes, which give the ruling party vast new powers over the courts, are democratic, making judges more accountable. The EU and a number of human rights groups say the changes erode the independence of the judicial branch.

Duda’s speech came as the EU is struggling with challenges on several fronts, including a similar conflict with Hungary, Britain’s departure next year and a new euroskeptic government in Italy. EU lawmakers on Wednesday voted to launch action against the Hungarian government of Prime Minister Viktor Orban for allegedly undermining the bloc’s democratic values and rule of law.

The reaction in Poland to Duda’s words was strong, and some commentators expressed fears that Polish authorities could be putting the country on a path of eventual departure from the union.

Chief opposition leader Grzegorz Schetyna said that Duda didn’t understand the “damage that such words bring to Poland’s image and reputation.”

Wladyslaw Kosiniak-Kamysz, the leader of the small agrarian Polish People’s Party, accused Duda, who is a practicing Catholic, of rejecting the teaching of Poland’s top authority, the late pope St. John Paul II, who advocated for EU membership.

In his speech to the southeast community of Lezajsk, Duda accused Europe of abandoning Poland to Soviet control after World War II. He said because of that history, Poland has the right to have expectations of Europe and that above all, Poles “have the right to govern here and decide what kind of Poland we should have.”

Poland joined the EU in 2004, a step that has brought 14 years of strong economic growth, fueled by billions of euros received from various EU funds.

EU membership offered new freedoms for Poles to travel, work and study across Europe. That opportunity has been grasped by millions of Poles. Opinion surveys repeatedly put support for EU membership around 80 percent.

The spokeswoman for the ruling party “protested strongly” against any suggestion that it was trying to “eliminate” Poland from the 28-member bloc.

“We are a part of the EU, we are in the EU and absolutely nothing is changing that,” Beata Mazurek said.