How One Clippy Lint Was Sped Up 3,133×

The clippy::nonstandard_macro_braces lint makes sure that macros are called with the right kind of brackets. In Rust you create an empty vector with vec![], but formally nothing stops you from writing vec!() or even vec! {"???"}. Code like that annoys everyone who reads it, so for a range of well-known macros Clippy keeps a “canonical” bracket style and nudges the developer whenever they stray from the norm.

Why it was slow

The problem is that Clippy runs after macros have already been expanded. Instead of println! {"..."} it sees expanded code like std::io::_print(std::format_args_nl!("Hello, world!")), and there's no way to tell from that what brackets were in the source. Rust doesn't keep an explicit map of macro calls — by the author's own admission, this is one of Clippy's biggest pain points.

To recover the original brackets, the lint climbed up the expansion chain, reached into the “global session data” holding macro hygiene information, pulled the source text out by span, and then, through string manipulation (split on !, trim, look at the first character), figured out which bracket had been used.

Sounds tolerable — until you find out that this whole procedure was run for every expression, statement and item in the entire project. Each time it accessed the hygiene data twice, locked the symbol interner and kicked off recursion. In effect it was bogging down the whole compiler for the sake of a check that ended up eating around 25% of Clippy's runtime.

“Blame the maintainers”

The author makes a point of stressing this: it's not the fault of any particular contributor, and blaming them would be wrong. The responsibility lies with the maintainers, whose job is to make writing bad code as hard as possible. Rust frees you from buffer overflows and use-after-free, but that sometimes means you'll have to write more code, reach for macros less often, and not get so carried away with abstractions that a function you've “seen a hundred times” quietly eats up a quarter of the runtime.

The fix

The fix came in at under 200 lines: the problematic function was rewritten from a post-expansion lint into a pre-expansion one. There's no longer any need to fetch the source text and compute the bracket span with arithmetic, no need to lock either the symbol interner or the global session data. The author frankly admits that working with pre-expansion code in Clippy is considered a hack that shouldn't be trusted — but this hack saves hundreds of thousands of “compute dollars.”

To top it off, a long-planned benchmarking server finally materialized — self-hosted, with 200 days of memory and on a common processor architecture, so the numbers are close to real user experience. Building it was made possible by a contract with the Rust Foundation.

Source: Lobsters