Pay Attention to What You Are Relying Upon