Lean Diagnostic Messages: Missing Kernel Errors

If you're working with Lean, you might have encountered situations where your diagnostic messages don't quite tell the whole story. Specifically, the lean_diagnostic_messages tool, which is super handy for catching errors in your code, sometimes seems to skip over certain kernel errors and even miss regular errors when they appear. This can be super frustrating when you're trying to debug your Lean projects. In this article, we'll dive deep into this peculiar behavior, explore why it might be happening, and what you can do about it.

The Problem with lean_diagnostic_messages

Lean diagnostic messages are a critical part of the development process, providing feedback on syntax errors, type mismatches, and more. The lean_diagnostic_messages tool aims to consolidate these into a usable format, often for integration with IDEs or other development tools. However, as reported by users, this tool isn't always reporting all the errors that Lean's compiler (the kernel) detects. A minimal example showcases this issue quite clearly. Imagine you have a simple Lean file with a structure definition that attempts to derive Repr, and a lemma that uses the rfl tactic, which is known to fail in certain contexts. When lean_diagnostic_messages is run on this file, it returns an empty list of errors, indicating no issues found. This is problematic because there are indeed errors present: a kernel error stemming from the deriving Repr clause (which often involves unsafe operations that Lean's kernel needs to handle carefully) and a regular error indicating that the rfl tactic failed.

This behavior is not just a minor inconvenience; it can lead developers to believe their code is error-free when it's not. The lack of reporting for kernel errors is particularly concerning, as these are often fundamental issues related to how Lean's type theory is being applied. Furthermore, it seems that if the kernel error is the very first error in the file, both it and subsequent regular errors are completely omitted from the diagnostic output. This suggests a potential cascading failure in how errors are collected or processed when the first error encountered is of a specific type, like a kernel error. It's as if the diagnostic system gets derailed early on and fails to recover the rest of the error information.

Reproducing the Issue

To understand the problem better, let's look at how this issue can be reproduced. The initial report involves a minimal Lean file like this:

import Mathlib.Data.Real.Basic

structure test where
  x : ℝ
  deriving Repr

lemma test_lemma : False := by rfl

When lean_diagnostic_messages is executed on this file, the output is an empty result array, suggesting no errors were found. This is counter-intuitive given the code. The deriving Repr part can trigger a kernel error because Repr might rely on unsafe implementations, and Lean's kernel is strict about such things. The tactic 'rfl' failed is a straightforward error that should also be reported.

Interestingly, the behavior changes dramatically if another error is present before the problematic ones. Consider this modified file:

import Mathlib.Data.Real.Basic

lemma test_lemma1 : False := by rfl

structure test where
  x : ℝ
  deriving Repr

lemma test_lemma2 : False := by rfl

In this scenario, lean_diagnostic_messages does report all the errors, including the kernel error related to deriving Repr and the rfl tactic failures. This strongly suggests that the presence of a preceding error somehow enables the diagnostic tool to correctly capture subsequent errors, including the critical kernel errors. Without this preceding error, the tool seems to fail to initialize or collect the error information properly, especially when the first error encountered is a kernel-related one.

This dependency on the order and presence of errors is a significant clue. It points towards a potential issue in the error collection or reporting pipeline within lean_diagnostic_messages. It might be that the error handling logic doesn't gracefully recover from an initial severe error (like a kernel error) and thus fails to process or report subsequent errors correctly. Understanding this pattern is key to diagnosing and potentially fixing the underlying bug.

The Role of Kernel Errors

Kernel errors in Lean represent fundamental issues that arise from the core type theory and its implementation. These are not mere syntax mistakes or logical inconsistencies within your Lean code; they are problems that the Lean kernel itself cannot resolve or process. When you see a kernel error, it often means that something you've defined or used has violated the foundational rules of Lean's mathematical framework. For instance, attempting to derive instances for certain type classes can sometimes lead to kernel errors if the derivation process involves operations that Lean's kernel considers unsafe or ill-defined. The deriving Repr clause in the example is a prime candidate for this, as deriving Repr might involve introspection or other mechanisms that require careful handling within Lean's strict environment.

When lean_diagnostic_messages fails to report these kernel errors, it's a serious oversight. Developers rely on diagnostic tools to provide a complete picture of their code's correctness. Missing a kernel error means a potentially deep-seated problem goes unnoticed, which can lead to unexpected behavior or failures later in the development cycle, possibly in ways that are much harder to debug. The fact that these errors are missed when they appear first in a file, and that their presence is somehow dependent on prior errors, suggests a flaw in the error-reporting pipeline. It's possible that the system initializing the diagnostic process or collecting errors encounters an early, severe kernel error, and this causes it to shut down or fail to proceed with the rest of the analysis. When a