xai Explainable ML Models: what are explanations and why do we need them? – Part I Interpretability is a key element of trust for AI models. An explanation is an interpretable description of a model behavior. For an explanation to be valid it needs to be faithful to the model and it needs to be understandable to the user.