By Gregory Wallace

(CNN) -- The current "political environment" is so toxic that a large number of minorities may not reply to the 2020 Census, the Trump administration admitted in court Thursday, even without the addition of a controversial question asking about citizenship status.

Justice Department attorney Josh Gardner was arguing that critics of the proposed question had not proven definitively that the question itself, rather than the nation's political discourse, would drive non-responses -- which both sides acknowledged were likely.

"That has nothing to do with the citizenship question per se," Gardner told a federal court in Maryland. "They may be less inclined to trust the government" or the Trump administration.

The striking admission reveals the administration's own thinking about its standing in minority communities, despite the President casting himself as a boon for minorities' economic success.

Judge George Hazel probed Gardner's logic to see if he was conceding a key argument for the plaintiffs who want the question nixed: that the citizenship question would contribute to a less accurate count and the resulting disenfranchisement.

"It seems like the macro environment is going to exacerbate the impact of the citizenship question," Hazel said.

"Exactly. At the end of the day, it's entirely possible that the plaintiffs will be right," Gardner said.

But it's also possible, he added, that "the macro environment is just so challenging" that minorities and non-citizens would not respond anyway.

The failure to count people anywhere in the US would disenfranchise residents there from federal funding, and an undercount as low as 1.5% would impact the apportionment of congressional seats and delegates to the Electoral College, plaintiffs said.

The plaintiffs, a mix of immigrant and minority groups and individual citizens, strung together more than a year of emails, meetings and phone calls from Trump administration officials and argued the question resulted from a discriminatory conspiracy involving Commerce Secretary Wilbur Ross, then-Attorney General Jeff Sessions, then-White House strategist Steve Bannon, and Trump adviser Kris Kobach.

"Secretary Ross was a banker and the notion that he came up with this on his own is preposterous," said Denise Hulett, an attorney representing the Mexican American Legal Defense and Educational Fund, one of the plaintiffs. "The motive shared by those who shared the solution was to exclude non-citizens from being counted."

Gardner, the government attorney, said conversations about the issue were a normal part of the policy-making process, and that Ross had ultimately and reasonably decided to include the question based on a request from the Justice Department.

"He could have 20 other reasons, 50 other reasons, one other reason," Gardner told the court, but those do not matter, and the critics "failed to identify any other action by the secretary [Ross] that is discriminatory or targets any protected class."

Wait for Supreme Court?

Thursday's arguments wrapped up the last of three trials over the citizenship question. The Supreme Court is preparing for arguments in April appealing a New York federal judge's ruling that the question was unlawfully added to the Census, and a trial judge in California is expected to rule soon on a similar case.

Judge Hazel appeared to bristle at a suggestion he wait for the Supreme Court to decide a separate lawsuit over the citizenship question before issuing his ruling.

"So you're suggesting I take my spring vacation?" he asked Brett Shumate, a high-ranking Justice Department attorney, who suggested the forthcoming high court ruling would make this decision moot.

"I'll endeavor to get an opinion out as quickly as possible," Hazel said. "I'm not inclined to wait for something else to happen, whatever that is."