conference paper published yes Filip Cano Cordoba author 708cad98-e86a-11ef-8098-bdae2d7c6af10000-0002-0783-904X ToHe department ICAART: International Conference on Agents and Artificial Intelligence Vigilant Algorithmic Monitoring of Software project Modern AI systems increasingly rely on opaque, highly complex models whose inner workings remain inaccessible even to experts. This opacity creates challenges for trust, accountability, and compliance with emerging regulatory expectations such as the “right to an explanation”. While traditional explainability methods—feature attributions, counterfactuals, surrogate models—and interpretable model classes provide valuable insights for engineers, they often fall short of delivering the contextual, conversational explanations that real users expect. Large Language Models (LLMs) offer a promising new avenue for explanation due to their ability to engage interactively, adapt to user needs, and translate technical outputs into more accessible reasoning. However, their tendencies toward hallucination, conflict avoidance, and oversimplification introduce serious risks when used as explanatory agents. This paper analyzes these opportunities and limitations, examines verification strategies for ensuring explanation fidelity, and situates LLM-generated explanations within broader concerns about public trust. The paper concludes by outlining best practices and future research directions for building robust, verifiable, and human-aligned explanation systems. Science and Technology Publications2026Marbella, Spain eng Explainable AILarge Language ModelsTrust in AI 2184-3589 2184-433X 978989758796210.5220/0014483200004052 54689-4696 Cano Cordoba, Filip. “Explaining Decisions One Conversation at a Time: Opportunities and Risks of LLMs as Explainability Assistants.” <i>Proceedings of the 18th International Conference on Agents and Artificial Intelligence</i>, vol. 5, Science and Technology Publications, 2026, pp. 4689–96, doi:<a href="https://doi.org/10.5220/0014483200004052">10.5220/0014483200004052</a>. Cano Cordoba F. 2026. Explaining decisions one conversation at a time: Opportunities and risks of LLMs as explainability assistants. Proceedings of the 18th International Conference on Agents and Artificial Intelligence. ICAART: International Conference on Agents and Artificial Intelligence vol. 5, 4689–4696. Cano Cordoba F. Explaining decisions one conversation at a time: Opportunities and risks of LLMs as explainability assistants. In: <i>Proceedings of the 18th International Conference on Agents and Artificial Intelligence</i>. Vol 5. Science and Technology Publications; 2026:4689-4696. doi:<a href="https://doi.org/10.5220/0014483200004052">10.5220/0014483200004052</a> F. Cano Cordoba, in:, Proceedings of the 18th International Conference on Agents and Artificial Intelligence, Science and Technology Publications, 2026, pp. 4689–4696. Cano Cordoba, F. (2026). Explaining decisions one conversation at a time: Opportunities and risks of LLMs as explainability assistants. In <i>Proceedings of the 18th International Conference on Agents and Artificial Intelligence</i> (Vol. 5, pp. 4689–4696). Marbella, Spain: Science and Technology Publications. <a href="https://doi.org/10.5220/0014483200004052">https://doi.org/10.5220/0014483200004052</a> F. Cano Cordoba, “Explaining decisions one conversation at a time: Opportunities and risks of LLMs as explainability assistants,” in <i>Proceedings of the 18th International Conference on Agents and Artificial Intelligence</i>, Marbella, Spain, 2026, vol. 5, pp. 4689–4696. Cano Cordoba, Filip. “Explaining Decisions One Conversation at a Time: Opportunities and Risks of LLMs as Explainability Assistants.” In <i>Proceedings of the 18th International Conference on Agents and Artificial Intelligence</i>, 5:4689–96. Science and Technology Publications, 2026. <a href="https://doi.org/10.5220/0014483200004052">https://doi.org/10.5220/0014483200004052</a>. 221032026-06-21T22:03:00Z2026-06-24T08:37:00Z