He can always just claim that his pardon does not shield him from state criminal charges and also from civil suits. From the article In Mason v. United States, the Supreme Court made it clear that pleading the Fifth is “confined to instances where the witness has reasonable cause to apprehend danger from a direct answer.” https://thefederalist.com/2026/08/04/why-faucis-refusal-to-answer-congress-puts-his-pardon-in-jeopardy/