RFC: --crate-attr

[jyn514]

Rendered

cc rust-lang/rust#138287

Differences from the current nightly implementation (note that this section is non-normative):

• -A -W -D -F become aliases for crate-attr
• Attributes are prepended, not appended.
• Everything about how this interacts with doctests (see RFC: --crate-attr #3791 (comment))
• The attribute shares the same Span as the crate root. In particular file! behaves properly now and the edition is no longer always 2015.

[jyn514]

after thinking about this, i think this should always apply to the crate being tested. to apply to the generated test people can use --crate-attr=doc(test(attr(...))) (which is a bit of a mouthful, but, at least it's unambiguous.)

[tamird]

such behavior would deviate from the semantics of existing flags like --warn which apply to both the crate being tested and its doctests.

[jyn514]

hmm, ok. i will change the RFC to specify it should apply to both.

cc @rust-lang/rustdoc

[jyn514]

i actually expanded this into something more complicated; see the RFC for details.

[Nemo157]

From what I can tell rustdoc --test --deny=... does not apply to either the crate being tested or its doctests. rustdoc --test -Zcrate-attr seems to only apply to the generated doctests crates, so -Zcrate-attr=doc(test(attr(...))) does not currently work.

[jyn514]

both of those seem like implementation bugs to me. i am happy to open an issue on rl/r if this RFC is merged.

[jieyouxu]

Question (not necessarily for you but the Cursed Code Specialists): do we have any wonky (stable) crate-level attributes like #![crate_name = EXPR] (whose validation was only recently fixed as a breaking change in rust-lang/rust#127581) that an injection like --crate-attr=crate_name=include_str!("crate_name.txt") can lead to... interesting behaviors?

Or perhaps, can crate-root attributes take a "value" that's a macro, possibly line!()?

[jyn514]

these should be questions for the individual attributes, imo. crate-attr does not try to special-case these in any way, it just forwards the macro invocation which gets rejected later in compilation. i do not think we should try to limit the syntax to things that "look normal".

$ rustc +r2-stage1 src/main.rs -Zcrate-attr=crate_name='line!()'
error: malformed `crate_name` attribute input
 --> <crate attribute>:1:1
  |
1 | #![crate_name=line!()]
  | ^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#![crate_name = "name"]`

(here r2-stage1 is a checkout of rust-lang/rust#138336)

[jieyouxu]

Yeah that seems completely fair (if anything, may be QoI issue on specific crate-level attrs)

[jieyouxu]

(I'll temporarily leave this thread open in case people do know of any Funny Things, but please resolve this comment chain before FCP as it is not intended to be blocking).

[jyn514]

oh, i misunderstood. you were asking what include! and line! should expand to in this context. my understanding is that we do not currently have any crate-level attributes which accept values, right? i'm tempted to say we just delay defining this until we need to.

[fmease]

I agree with Jyn that we shouldn't restrict the grammar of those attributes.

To answer the original question however, there are several. E.g., #![doc = …], #![deprecated = …], #![type_length_limit = …]. In these three cases you can use concat!("", line!()) as the RHS which expands to (lineno) "1" when passed via -Zcrate-attr (I'm using concat to essentially get rid of the numeric suffix).

(crate_name, crate_type and recursion_limit are fully locked-down nowadays in the sense that they no longer accept macro calls)

(since a lot of malformed and misplaced attrs are still accepted by rustc and only trigger the warn-by-default lint unused_attributes, you can currently also do things like #![must_use = compile_error!("…")] which isn't that interesting arguably)

[Nemo157]

rustdoc -Zcrate-attr='doc=concat!("test", line!())' foo.rs gave a doc/foo/index.html containing "test1".

[jyn514]

ok. of these, line!() really does not seem to have a meaning to me, maybe the implementation comes up with "1" but i don't think that's useful. so i'm tempted to ban that particular macro in this context.

include! and friends all seem fine to me, they should be relative to whatever #![doc = include_str!()] in the source is relative to.

[jieyouxu]

FWIW I was mostly thinking adversarially on what weird combinations this can end up with (e.g. what's prone to break, between a stable + stable feature/functionality interaction). I suppose this may also include the other "meta" ones like column!(). Not sure how to phrase this, but "observable implementation details" -- e.g. that line number 1 being observable through line!() is potentially interesting if it can become depended upon by the user.

Alternatively... explicitly carve out a stability caution/exception where it's explicitly remarked that depending on line!()/column!() etc. is not part of the stability guarantees? Idk how to word this though. It's not a big concern but it can be annoying if it ever comes up.

[jyn514]

i still think that all these concerns apply just as much to existing crate-level attributes in source code. if there is a concern here it should be specific to attributes passed via a CLI flag.

[jieyouxu]

Remark: +1, I think it's perfectly fine (and arguably good) to not support multi-attr in same flag. If anything, multiple --crate-attr flags is way easier to read too, IMO.

[Mark-Simulacrum]

IIUC, register-tool would have to be configured in the workspace -- passing it only when the tool was invoked likely doesn't work since then the "passive" state of the code has a bunch of non-registered attributes? Maybe I'm missing something.

[Mark-Simulacrum]

FWIW, I don't think this quite matches how expansion actually works -- in particular step 1 in the compiler can't parse the full crate, since that's inherently interleaved with macro expansion. But I don't think that fundamentally changes anything specified here.

[nagisa]

Hm, it might be a good idea to specify this feature in context of precise operation of today's expansion if only to get ahead of potential underspecification issues if the details of macro expansion were to change in the future. I don't expect this to change the structure much, but it may require author to give a closer read on the parser's code.

(I'd be also fine with this part of RFC being refined to describe what ends up being implemented later down the line…)

[Mark-Simulacrum]

Suggested change

	3. The attributes are injected at the top of the crate root.
	3. The attributes are injected at the top of the crate root (see below for specifics on ordering).

[Mark-Simulacrum]

@rfcbot fcp merge

I've wanted this functionality before myself, and it's a requested feature from Rust for Linux, so demand exists. I think the RFC does a good job motivating the feature and noting edge cases that need resolution, though personally nothing that necessarily seems like it blocks stabilization. So, proposing we merge it.

[rfcbot]

Team member @Mark-Simulacrum has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Mark-Simulacrum
• @Noratrieb
• @SparrowLii
• @cjgillot
• @compiler-errors
• @davidtwco
• @estebank
• @matthewjasper
• @nagisa
• @oli-obk
• @petrochenkov
• @spastorino
• @wesleywiser

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[wesleywiser]

and [build-std], which wants to inject no_std into all crates being compiled

Just as a note, RfL already does this exact thing and they would like to see -Zcrate-attr stabilized as well:

https://github.com/Rust-for-Linux/linux/blob/e6ea10d5dbe082c54add289b44f08c9fcfe658af/scripts/Makefile.build#L238-L239

[apiraino]

visited in meeting on Zulip

@rustbot label -I-compiler-nominated

[nagisa]

Hm, it might be a good idea to specify this feature in context of precise operation of today's expansion if only to get ahead of potential underspecification issues if the details of macro expansion were to change in the future. I don't expect this to change the structure much, but it may require author to give a closer read on the parser's code.

(I'd be also fine with this part of RFC being refined to describe what ends up being implemented later down the line…)

[nagisa]

Not to mention we already have @file to pass in arguments from a file if for some reason it is necessary to have a single argument in the list of arguments.

[nagisa]

Both line! and column! should refer to the original source code, as written (i.e. prior to expansion.) That's their current behaviour, isn't it? I see no reason why that ought to change. Any reference to these macros would expand to the first column of the first line, regardless of the number of attributes or their length in that case.

Why? Consider the primary use-case for these macros – inserting location info into debug messages. It'd be weird if the logs said that the location is at line 100, but the expression that produced the log line ends up being on line 95 (all because 5 attributes have been introduced.) Same thing stands for column! (there's nothing preventing one from typing out the entire rust module into a single line.) Could argue that the actual source-code line!s and columns! could retain their behaviour still, but at that point we end up with a weird situation where the same line:col could refer to two entirely distinct locations.

[Noratrieb]

The problem presented here is the span of a line! inside a --crate-attr. So like a --crate-attr doc(concat!("i'm on: ", line!()))

[nagisa]

I understand that. The load bearing part in my comment is the 2nd paragraph. Let me try rephrasing it.

If any non-trivial logic had to be prescribed to line!() and column!() inside these inserted attributes there are two ways to go about doing so:

1. Consider the inserted attributes as-if they were written in the source file. This then offsets every other line!() and/or column!() (thus breaking basic expectations of these macros); or
2. Drop the guarantee that a given file:line:col refers to exactly one location in the source.

Neither of these options are viable in my opinion. 1st, again, breaks basic expectations of these macros. As for 2nd option consider a situation where a diagnostic has to be shown at, say line 1 column 8~16. Does rustc display a snippet from an introduced attribute? A snippet from the source file before attributes were introduced? Maybe rustc isn't a great example, as error could be in either of these locations & rustc's source location tracking will need to be adjusted regardless. But what about other tools? Perhaps even tools that don't necessarily have any business parsing rust code (and thus no reason to accept --crate-attr as an argument,) but need to tie lines and columns back to the original source somehow?

These considerations ultimately lead to only one sensible implementation for line!()/column!() in introduced attributes: they both should result in 0, 1 or some similar placeholder of some sort that proper use of line!()/column!() cannot otherwise produce. Such approach can also be rationalized by describing attribute insertion as:

0 | insert_attributes!(); // attributes are being prepended to the source file & 0 comes before 1...
1 | // source code goes here, unperturbed...

Since macro invocations present a #[track_location]-like behaviour, all line!() and column!() invocations that insert_attributes!() expands to produce the same value (in this case 0:0, 1:1 or whatever else we pick.)

[programmerjake]

why not just do what clang does and consider the inserted attributes to be in the file <built-in> or similar?
https://gcc.godbolt.org/z/aYn3cz5c3
clang++ ... --include=cstddef --include=unknown2
gives:

<built-in>:2:10: fatal error: 'unknown2' file not found
    2 | #include "unknown2"
      |          ^~~~~~~~~~
1 error generated.
Compiler returned: 1