From a20f042bb390d516f4dae703ca9c8f1eeb7c1057 Mon Sep 17 00:00:00 2001 From: Without Boats Date: Mon, 14 Aug 2017 11:40:33 -0700 Subject: [PATCH 1/3] RFC for module redesign. --- 0000-modules/README.md | 34 +++ 0000-modules/detailed-design/loading-files.md | 194 ++++++++++++++++++ 0000-modules/detailed-design/migration.md | 167 +++++++++++++++ 0000-modules/detailed-design/paths.md | 115 +++++++++++ .../visibility-and-reexport.md | 174 ++++++++++++++++ 0000-modules/motivation.md | 146 +++++++++++++ 0000-modules/overview.md | 190 +++++++++++++++++ 7 files changed, 1020 insertions(+) create mode 100644 0000-modules/README.md create mode 100644 0000-modules/detailed-design/loading-files.md create mode 100644 0000-modules/detailed-design/migration.md create mode 100644 0000-modules/detailed-design/paths.md create mode 100644 0000-modules/detailed-design/visibility-and-reexport.md create mode 100644 0000-modules/motivation.md create mode 100644 0000-modules/overview.md diff --git a/0000-modules/README.md b/0000-modules/README.md new file mode 100644 index 00000000000..93d84d10ed2 --- /dev/null +++ b/0000-modules/README.md @@ -0,0 +1,34 @@ +- Feature Name: modules +- Start Date: 2017-08-07 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary +[summary]: #summary + +This is a redesign of the Rust module system, intended to improve its +ergonomics, learnability, and locality of reasoning. Because this is a +relatively large proposal, it has been broken into multiple text files. + +# Table of Contents + +* **[Motivation][motivation]** - why we propose to make this change +* **[Overview][overview]** - a high level overview of what it will be like to +use the new system as a whole +* **Detailed design** - the details of the proposal, broken into multiple +sections: + * **[Loading Files][loading-files]** + * **[Paths][paths]** + * **[Visibility and Re-exporting][visibility]** + * **[Migration][migration]** - this proposal involves migrating from one + system to another, and this section describes it in detail. + +Each of the detailed design subsections contains its own description of +drawbacks and alternatives. + +[motivation]: 0000-modules/motivation.md +[overview]: 0000-modules/overview.md +[loading-files]: 000-modules/detailed-design/loading-files.md +[paths]: 000-modules/detailed-design/paths.md +[visibility]: 000-modules/detailed-design/visibility-and-reexport.md +[migration]: 000-modules/detailed-design/migration.md diff --git a/0000-modules/detailed-design/loading-files.md b/0000-modules/detailed-design/loading-files.md new file mode 100644 index 00000000000..4774708da89 --- /dev/null +++ b/0000-modules/detailed-design/loading-files.md @@ -0,0 +1,194 @@ +# Loading Files + +When building a Rust project, rustc will and parse some files as Rust code in +addition to the root module. These will be used to construct a module tree. By +default, cargo will generate the list of files to load in this way for you, +though you can generate such a list yourself and specify it in your +`Cargo.toml`, or you can generate the list in another way for your non-cargo +build system. + +This eliminates the need to write `mod` statements to add new files to your +project. Instead, files will be picked up automatically as a part of your +module hierarchy. + +## Detailed design + +### Processing the `--module` list (rustc) + +rustc takes a new argument called `--module`. Each `--module` argument passes +the name of a file to treat as a module. rustc will attempt to open and parse +all of these files, and report any errors if it is unable to. It will mount +these files as a tree of Rust modules using rules which mirror the current +rules for looking up modules. + +It will not attempt to open or parse files if the paths meet these conditions: + +* The file name is not a valid Rust identifier followed by `.rs`. +* The file is not a subdirectory of the directory containing the root module. +* Any of the subdirectories of the root module in the path to this file are not +valid Rust identifiers. + +(Cargo's default system will not pass any files that would be ignored by these +conditions, but if they are passed by some other system, they are ignored +regardless.) + +Rust will mount files as modules using these rules: + +* If a file is named `mod.rs`, it will mount it as the module for the name of +directory which contains it (the directory containing the root module cannot +contain a `mod.rs` file; this is an error). +* Otherwise, it will mount it at a module with the name of the file prior to +the `.rs`. + +All modules mounted this way are visible to the entire crate, but are not (by +default) visible in the external API of this crate. + +If, during parsing, a `mod` statement is encountered which would cause Rust to +load a file which was a part of the `--module` list, Rust does not +attempt to load that file. Instead, a warning is issued that that `mod` +statement is dead code. (See [migrations][migrations] for more info.) + +If a module is mounted multiple times, or there are multiple possible files +which could define a module, that continues to be an error. + +Another result of this design is that the naming convention becomes slightly +more flexible. Prior to this RFC, if a module file is going to have submodule +files, it must be located at `mod.rs` in the directory containing those +submodules - e.g. `src/foo/mod.rs`. As a result of this RFC, users can instead +locate it at `src/foo.rs`, but still have submodules in the `foo` directory. +Some users have requested this functionality because their tooling does not +easily support distinguishing files with the same name, such as all of their +`mod.rs` files. + +In fact, in this design, it is not necessary to have a `foo.rs` or `foo/mod.rs` +in order to have modules in the `foo` directory. Without such a file, `foo` +will just have no items in it other than the automatically loaded submodules. +For example: + +``` +/foo + bar.rs + baz.rs +lib.rs +``` + +This mounts a submodule `foo` with two items in it: submodules `bar` and `baz`. +There is no compiler error. + +### Gathering the `--module` list (cargo) + +#### Library and binary crates + +When building a crate, cargo will collect a list of paths to pass to rustc's +`--module` argument. It will only gather files for which the file name +has the form of a valid Rust identifier, followed by the characters `.rs`. + +cargo will recursively walk the directory tree, gathering all appropriate +files, beginning with the directory which contains the crate root file. It will +ignore these files and directories: + +* The crate root file itself. +* Any directory with a name which is not a valid Rust identifier. +* If the crate root is in the `src` subdirectory of the Cargo manifest +directory, and there is a directory called `src/bin`, cargo will ignore that +subdirectory. + +In short, cargo will include all appropriately named files inside the directory +which contains the crate root, except that it will ignore the `src/bin` +directory. + +Packages containing multiple crates which wish to use the default module list +will need to make sure that they do not have multiple crates rooted in the same +directory, or within a subdirectory of another crate. The most likely +problematic crates today are those which have both a `src/lib.rs` and a +`src/main.rs`. We recommend those crates move their binary crate to the +`src/bin` directory solution. + +While gathering the default module list, cargo will determine if any other +crate is rooted in a directory which would be collected by the default module +list, and will instead not pass a `--module` list and issue a warning in +that case, informing users that they need to rearrange their crates or provide +a list of modules themselves. + +(**Note:** These projects will receive a warning, but will not immediately be +broken, because the `mod` statements they already contain will continue to pick +up files.) + +#### Tests, examples, and benchmarks + +Test, example, and benchmark crates follow a different set of rules. If the +crate is located in the appropriate top-level directory (`tests`, `examples`, +and so on), no `--module` list will be collected by default. However, +subdirectories of these directories will be treated as individual binary +crates: a `main.rs` file will be treated as the root module, and all other +appropriately named files will be passed as `--module`, using the same +rules described above. + +So if you have an examples directory like this: + +``` +examples/ + foo.rs + bar/ + main.rs + baz.rs +``` + +This contains two examples, a `foo` example and a `bar` example, and the `bar` +crate will have `baz.rs` as a submodule. + +The reason for this is that today, cargo will treat every file in `tests`, +`examples`, and `benches` as independent crates, which is a well-motivated +design. Usually, these are small enough that a single file makes sense. +However, today, cargo does not make it particularly easy to have tests, +examples, or benchmarks that are multiple files. This design will create a +pattern to enable users to do this. + +#### Providing your own module list + +The target section of the `Cargo.toml` will gain a new item called `modules`. +This item is expected to be an array of strings, which are relative paths from +the cargo manifest directory to files containing Rust source code. If this item +is specified, cargo will canonicalize these paths and pass them to rustc as the +`--module` argument when building this target. + +## Drawbacks + +The RFC authors believe that making mod statements unnecessary is a *net* win, +but we must acknowledge that it is not a *pure* win. There are several +advantages that mod statements bring which will not be fully replicated in the +new system. + +Some workflows have been convenienced by the fact that statements need to be +added to the source code to add new modules to files. For example, it makes it +easier for users to leave their src directories a little bit dirty while +working, such as through an incomplete `git stash`. If users wish to comment +out a module, it can be easier to comment out the `mod` statement than to +comment out the module file. In general, it enables users to leave code which +would not compile in their src directory without explicitly commenting it out. + +Some users have expressed strong concerns that by deriving the module structure +from the file system, without making additional syntactic statements, they will +not be able to as easily find the information they need to navigate and +comprehend the codebases they are reading or working on. To partly ease their +concern, the RFC allows users to explicitly specify their module lists at the +build layer, instead of the source layer. This has some disadvantages, in that +users may prefer to not have to open the build configuration either. + +This will require migrating users away from `mod` statements toward the new +system. + +## Alternatives + +An alternative is to do nothing, and continue to use `mod` statements. + +We could also put the file-lookup in rustc, instead of cargo, and have rustc +perform its own directory walk. We believe this would be a bad choice of +layering. + +During the design process, we considered other, more radical redesigns, such as +making all files "inlined" into their directory and/or using the filename to +determine the visibility of a module. We've decided not to steps that are this +radical right now. + +[migrations]: 0000-modules/detailed-design/migrations.md diff --git a/0000-modules/detailed-design/migration.md b/0000-modules/detailed-design/migration.md new file mode 100644 index 00000000000..6d1c013708b --- /dev/null +++ b/0000-modules/detailed-design/migration.md @@ -0,0 +1,167 @@ +# Migration + +The changes proposed by this RFC are significant breaking changes to the +language. To complete them entirely we will need to pass a checkpoint/epoch +boundary as described in [RFC #2052][checkpoints]. However, we will begin with +a multi-phase deprecation process to ease the transition. + +We will also introduce a tool that will not only automatically move a crate +onto a new system, but also *improve clarity* of the user's code by +intentionally drawing the distinction between the `export` and `pub` +visibilities that today does not firmly exist. + +## Initial, compatible mode + +In the first release supporting this RFC, we will begin supporting all of the +feature set of this RFC, without breakages to existing functionality. That is: + +* `--extern` and `--module` arguments will be automounted, and cargo will pass +`--module` arguments in its expected way. +* The `crate::` path prefix will be accepted. +* The `export` visibility will be accepted. +* The "vis path" re-export syntax will be accepted. + +However, existing semantics will not be broken, meaning: + +* Module-internal paths will be accepted from the crate root without the +`crate::` prefix. +* `pub` will continue to have its current semantics if a crate does not ever +use the `export` visibility + +In other words, as soon as this RFC is implemented, users will be able to +transition to the new system, but they will not be immediately *required* to. + +The only breakage during this phase arises if users have files with valid Rust +module names inside of the directory than defines a crate, which is not a +compilable module of that crate. Users are encouraged to fix that by renaming +those files so that they will not be picked up by cargo, or else deleting them +if they are not necessary. **Note** that this does not include overlapping +crates which are both managed by cargo, as when cargo detects that it will +instead pass no `--module` arguments. + +### Using `export` changes semantics + +One aspect of this RFC requires a bit more of a complex transition than simply +"add new features, then deprecate the old ones." That is the change in the +semantics of the `pub` keyword. + +Under the current checkpoint/epoch, the `pub` keyword will continue to mean +exactly what it means prior to this RFC **unless** the crate under compilation +contains at least one `export` statement. If an `export` statement is found in +the crate, `pub` is interpreted to have its semantics defined under this RFC +(public to this crate). + +## Deprecations + +Deprecation will proceed in two phases: + +1. First, we will deprecate code which is now superfluous as a result of this +RFC. +2. Then, at a later point, we will issue more opinionated deprecations, pushing +users to write code which is compatible with the future checkpoint transition. + +### Phase 1 - deprecating dead code + +#### Dead `extern crate` + +We will issue deprecation warnings on extern crate statements which do not have +a `#[macro_use]` attribute. In the majority of cases, users will need to delete +these statements and take no other actions. In some cases, they will also need +to do something else as well: + +* If they used `extern crate as` syntax, they will need to add an alias to +their dependency object in their Cargo.toml. +* If they mounted the extern crate somewhere other than the crate root, they +will need to use an appropriate import at that location. + +#### Dead `mod` + +We will issue deprecation warnings on mod declarations for which there is a +corresponding `--module` argument. For private or `pub(crate)` modules without +attributes, users will need to delete these statements and nothing else. In +other cases, they will need to do other things as well: + +* If they used a visibility modifier on the module, they will need a re-export. +* If they used an attribute, they will need to move it inside the module. + +Note that modules that do not have a `--module` argument are *not* considered +dead code, and continue to be used to load additional files. If cargo was not +able to generate a module list for a crate, and the user hasn't specified one, +mod statements will continue to be the way to find modules. + +#### Dead `pub(crate)` + +If the user is using `export` semantics, `pub` becomes equivalent to +`pub(crate)`. We will issue dead code warnings on the `(crate)` restriction if +the crate is being compiled under this RFC's semantics. + +### Cargo errors when unable to generate the module list + +When cargo recognizes that a crate contains the source of another inside of it, +it will not generate a module list. When it does so, it issues a warning, +encouraging users to do one of two things: + +* Set their module list manually in the `Cargo.toml`. +* Rearrange their package so that their crates do not have overlapping source +directories. + +### Phase 2 - opinionated deprecations + +#### Deprecating all `extern crate` and `mod` statements + +All `extern crate` and `mod` statements (without blocks) will be deprecating. + +Users will need to be using cargo's default module loading or a manual module +list to avoid warnings. + +### Deprecating crate paths not through `crate::` + +Absolute paths to items in this crate which do not pass through `crate` will +receive a warning. + +### Warning to move to `export` in libraries + +When compiling a library, if still using the pre-`export` semantics, users will +receive a warning providing them documentation on the change to `export` and +encouraging them to make that transition. + +#### Blockers on phase 2 + +In the second phase, all `extern crate` statements will be deprecated. Phase 2 +as proposed by this RFC is, for this reason, blocked on deprecating the +`#[macro_use]` attribute on extern crates. + +#### Staggering phase 2 + +Its possible that some of the phase 2 deprecations will be issuable sooner than +others. Its not necessary that phase 2 be initiated as a single unit, but +instead its deprecations could be introduced across multiple releases. + +### Phase 3 - next checkpoint + +Once phase 2 has been completely implemented, in the next checkpoint, all of +the phase 2 warnings will be made into hard errors. + +## Tooling + +Concurrent with the first release under this RFC, we will also release a tool +which will automatically transform projects to be consistent with this RFC (and +therefore work, without warnings, on both the current and future checkpoint). + +The exact interface and distribution of this tool is left unspecified for the +implementation, but as a rough sketch, it would perform these transformations: + +* If a package contains both a `lib.rs` and a `main.rs`, it will move the +binary crate into the `bin` directory. +* It will remove unnecessary `mod` and `extern crate` statements. +* It will transform `use` imports and absolute paths to include the `crate` +prefix when necessary. +* It will selectively replace `pub` with `extern` *only* when the type is +exposed in the actual external API of the crate. (For binaries, it will not +perform this change at all.) This way, the tool will automatically make code +more self-documenting. + +Our aspiration is that most users will have access to this tool and be able to +use it, making this transition automatic for most users. + +[checkpoints]: https://github.com/rust-lang/rfcs/pull/2052 diff --git a/0000-modules/detailed-design/paths.md b/0000-modules/detailed-design/paths.md new file mode 100644 index 00000000000..5e3f2bf10c7 --- /dev/null +++ b/0000-modules/detailed-design/paths.md @@ -0,0 +1,115 @@ +# Paths + +Today, the `::` "root" path refers to the crate root. Under this RFC, this will +be changed, so that instead it refers to a root which contains all `--extern` +dependencies, as well as the current crate under the special `crate` module. + +This means that imports from within this crate will be distinguished from +imports from other crates by being initialized with the `crate` keyword: + +```rust +// Import from a dependency +use std::fs::File; +// Import from within this crate +use crate::toplevel::Item; +``` + +## Detailed design + +As a result of [RFC #2088][extern-crate], `extern crate` will be deprecated. In +conjunction with that RFC, all `--extern` dependencies are mounted at the +absolute path root. However, that no longer corresponds to the root file of +this crate (usually `lib.rs` or `main.rs`). Instead, that root *contains* the +root file of this crate in a module named `crate`. + +`use` statements continue to take paths from the absolute root, but this no +longer corresponds to the crate root. Other paths continue to be relative by +default, but can be made absolute with an initial `::`. + +The use of the `crate` keyword has several advantages: + +1. Users can always tell whether a path comes from an external dependency or +from within this crate. This increases the explicit, local information. Today +you have to know if `::foo` refers to a top-level module or a dependency. +2. It is no longer the case that within the root module, relative paths and +absolute paths correspond. This correspondence has lead users to believe that +they will always correspond, causing significant confusion when they don't +correspond in submodules. +3. Our survey of the ecosystem suggests that external imports are more common +than internal imports and we believe this will continue to be true for the +reasons listed below. Thus if we are going to distinguish external dependencies +from internal ones, it makes more sense to put the distinguishing mark on +internal dependencies. + * We encourage users to modularize their code using the `crates.io` + ecosystem, leading to a proliferation of external dependencies. + * Many large projects are broken into several crates, making even + intra-project dependencies "external" dependencies for the purposes of + paths. +4. `crate` is already a reserved word, the current purpose of which is being +deprecated. This is easier to transition to than using new syntactic forms +which would complicate the grammar & require more learning about the meaning of +sigils. +5. Several other languages work this way; most use the name of this package as +the distinguisher, rather than a keyword, but a keyword has these advantages: + * If a crate is renamed, you do not need to rename all internal imports. + * Crates currently can depend on other crates with the same name; this is + commonly employed when you want a library with the same name as a binary + (for example, cargo). Using a keyword avoids breaking this pattern. + +`crate` will act like a "special path prefix," similar to `self` and `super`. +Relative paths can begin with `crate` without beginning with a `::` prefix, to +make them relative to the crate module. + +## Drawbacks + +This will require migrating users away from the current path syntax in which +`crate` is not required. + +These paths will be longer, being slightly less convenient to type. This is a +trade off to make it more explicit whether an import is from this or another +dependency. + +## Alternatives + +The primary alternative is to distinguish external dependencies instead of +distinguishing internal crate-rooted dependencies. For example, you could +imagine this instead: + +```rust +use extern::serde::Serialize; +use module::Item; +``` + +Various syntaxes, more radically departing from the current syntax, have also +been considered, such as: + +```rust +use [serde]::Serialize; +use :serde::Serialize; +from serde use Serialize; +``` + +We believe because of the slight weighting toward extern dependencies, the +similarity to other languages and - most importantly - the symmetry this +introduces for `use` statements in the root and other modules (reducing path +confusion), distinguishing local imports is the most pragmatic choice. + +We have also considered even more radical departures, such as making `use` take +local paths by default, but ultimately decided not to make such an enormous +departure from the current system. + +## Potential future extensions + +This RFC doesn't include any other sugar in the `use` syntax, because it +already is a large and complicated change. However, the cost of introducing +`crate` can be mitigated (and imports made shorter, in general) through some +sort of multi-tiered nesting, as in: + +```rust +use crate::{ + module::Item, + other_module::{AnotherItem, some_other_function}, +}; +``` + +[extern-crate]: https://github.com/rust-lang/rfcs/pull/2088 diff --git a/0000-modules/detailed-design/visibility-and-reexport.md b/0000-modules/detailed-design/visibility-and-reexport.md new file mode 100644 index 00000000000..de83a771dba --- /dev/null +++ b/0000-modules/detailed-design/visibility-and-reexport.md @@ -0,0 +1,174 @@ +# Visibility and Re-Exporting + +This RFC changes the hierarchy of visibility keywords in Rust, introducing a +new `export` visibility. It also introduces a new mechanism for re-exporting, +replacing `pub use`. + +## Detailed design + +### Visibility modifiers + +The set of valid visibility modifiers is changed to be: + +* `pub(self)` - visible to this module & its children +* `pub(super)` - visible to the parent module & its children +* `pub(in $path)` - visible to the given path, which must be a parent of this +module +* `pub` - visible to this entire crate +* `export` - visible in the API of this crate + +The first three exist today and have no change in meaning, but there has been a +shift in the final two, represented in this table: + +Today | After this RFC +---------- | -------------- +pub(crate) | pub +pub | export + +In addition, we introduce a `public-in-private` error for items marked +`export`. If a canonical item (that is, not a re-export) is marked `export` but +not actually reachable by a public path (whether through its canonical path or +through a re-export), the compiler raises a hard error about failing to +actually export that item. + +Enforcing this convention will enable greater clarity around intent - if a user +means for an item to be in the public API, they will have demarcated it +distinctly. It will also be less likely for users to accidentally expose +something in their API, because their 'default' move will be to mark things +`pub` instead of `export`, and if they mark something `export`, they *know* it +is part of their public API. + +Additionally, it will become more meaningful to search a crate for all +instances of `export` than it today is to search for all instances of `pub`. +After this change, that search will tend to give you a good sense of the entire +API this crate exposes, whereas today it includes many items which are marked +`pub` but are not exposed. + +Finally, though redefining `pub` may seem like a very significant breakage, +practical breakages will be limited by the [migration][migration] mechanism. +Repurposing the `pub` syntax will also limit some of the "long tail" effects of +breakages. For example, if outdated documentation or Q&A content uses the `pub` +visibility in a code snippet, that code will continue to compile, whereas if a +new keyword were used, that code would not work on a newer Rust checkpoint. + +The distinction between the two definitions of `pub` really only impacts +library authors who are in the process of migrating across versions. In +binaries, or even parts of libraries that aren't part of the external API, the +change in the meaning of `pub` does not make code stop working. + +### New re-export mechanism + +We also introduce a new mechanism for re-export, having the syntax: + +```rust +$visibility $path; +``` + +This is similar to the `pub use` system we have today, but the `use` keyword is +omitted. + +Other than the syntactic difference, another big difference is that these paths +are relative paths, the same paths used everywhere inside of modules (except +for `use` statements). + +Most commonly, the visibility used will be the export visibility. When you +compare a facaded module like `futures::future`, for example, it will go from +this: + +```rust +mod and_then; +mod map; +mod select; +mod then; + +pub use self::and_then::AndThen; +pub use self::map::Map; +pub use self::select::Select; +pub use self::then::Then; +// etc +``` + +To this: + +```rust +export and_then::AndThen; +export map::Map; +export select::Select; +export then::Then; +``` + +#### Exporting modules + +Because the blockless mod statement is deprecated, we need a new mechanism for +making modules a part of the public API. In this system, modules are made +public by exporting them: + +```rust +// Exports the `foo` module +export foo; +``` + +In this way, the syntax for mounting a module which is another file at this +location in the public API is the same as the syntax for mounting any other +item from other files at this point in the public API. + +#### Private-in-public & re-exporting + +It is not permissible to use this mechanism to re-export something at a greater +visibility than its explicitly declared visibility. That is, this would be +invalid: + +``` +mod foo { + pub struct Foo; +} + +// This is an error: you have exported something which is only marked `pub` +export foo::Foo; +``` + +The exception to this is modules which have not been explicitly created with +the `mod` keyword. Because they have no explicit statement of their visibility, +it is permitted (and idiomatic) to use a re-export to make a module more +visible by exporting it, as discussed in the previous section. + +You also cannot re-export something at the same location if it has been +explicitly declared there - again, this means modules are exempt from this +rule. For example, this code would be invalid: + +```rust +export struct Foo; + +// This is an error: you have re-exported something at a path it already is +// mounted at. +export Foo; +``` + +#### Deprecated `pub use` + +As a result of this RFC, visibility attributes on `use` will be phased out. +Ultimately, `use` will not take a visibility modifier, like impl blocks. + +## Drawbacks + +The biggest drawback of this change is that redefining `pub` involves a complex +[migration][migration]. This is the only change in this RFC that does not +follow a strict "add and deprecate cycle." For reasons outlined in the detailed +design section, we believe this is the right choice, but it carries some costs. + +## Alternatives + +An alternative is to use a new keyword (such as `public`) and deprecating `pub` +entirely. The migration here would be less complex, but more disruptive, in +that every `pub` statement would need to change, and not only those that are +meant to be `export`. + +We could also introduce `export` as a new kind of statement, similar to `use`, +but for the purpose of re-exporting. We explored some schemes along these +lines, and found that the visibility-based system proposed above had the most +appealing properties, such as grepability and clarity of naming. + +We could make the error on a non-exposed `export` item a warning instead of a +hard error. + +[migration]: 0000-modules/detailed-design/migration.md diff --git a/0000-modules/motivation.md b/0000-modules/motivation.md new file mode 100644 index 00000000000..3ca06718a59 --- /dev/null +++ b/0000-modules/motivation.md @@ -0,0 +1,146 @@ +# Motivation + +## Background + +For the past several months, we have been investigating the module system, its +weaknesses, strengths, and areas of potential improvement. Two blog posts have +been produced, which lay out a part of the argument in favor of changing the +module system: + +* [The Rust module system is too confusing][too-confusing] by withoutboats +* [Revisiting Rust's modules][revisiting] by aturon + +Both of these posts contain proposals for how to revamp the module system, +neither of which are representations of what this RFC contains. However, they +provide valuable background on the conversation that has been occurring, since +January of this year, about how the Rust module system could be improved. + +Fundamentally, we believe that the current module system is difficult to learn +how to use correctly, and contains unnecessary complexity which leaves new +users confused and advanced users annoyed. We have collected empirical data in +support of that belief, and also have formed a model of the underlying problems +with the system that we hope to resolve or mitigate with this proposal. + +Its important to keep this in mind: our contention is not that the module +system is the most difficult part of the language, only that it is +**unnecessarily** difficult. Some aspects of Rust have high inherent novelty - +such as ownership & lifetimes - and cannot be just *made* easier (though the +syntax could always be more obvious, etc). We do not believe this is true of +our modules, which provide similar benefits to systems in many mainstream +languages which do not have such a problem. + +## Evidence of a problem + +### Survey data + +In the survey data collected in 2017, ergonomics issues were one of the major +challenges for people using Rust today. While there were other features that +were raised more frequently than the module system (lifetimes for example), we +don't feel that modules ought to be considered an ergonomics problem at all. + +Here are some quotes (these are not the only responses that mention the module +system): + +> Also the module system is confusing (not that I say is wrong, just confusing +> until you are experienced in it). + +> a colleague of mine that started rust got really confused over the module +> system + +> You had to import everything in the main module, but you also had to in +> submodules, but if it was only imported in a submodule it wouldn't work. + +> I especially find the modules and crates design weird and verbose + +> fix the module system + +One user even states that the reason they stopped using Rust was that the +"module system is really unintuitive." Our opinion is that if any user has +decided to *stop using Rust* because they couldn't make modules work, the +module system deserves serious reconsideration. + +### Found feedback + +@aturon devoted some time to searching for feedback from Rust users on various +online forums about Rust (GitHub, the discourse forums, reddit, etc). He found +numerous examples of users expressing confusion or frustration about how the +module system works today. This data is collected in [this +gist][learning-modules]. + +## Underlying problems + +### Lots of syntax, all of it frontloaded + +The current Rust module system has four primary keywords: `extern crate`, +`mod`, `use`, and `pub`. All of these are likely to be encountered very early +by users - even if you're working on a small binary project, just to learn, you +are likely going to want to add a dependency and break your code into two +files. Doing those two things introduces the need to use all four keywords +that are a part of the module system, and to understand what each of them does. + +In our proposal, there are three primary keywords: `use`, `pub`, and `export`. +And in an early project, only `use` and `pub` are going to be necessary. Not +only that, the `export` keyword is just a visibility, like `pub`, and can be +viewed as an extension of the pre-existing knowledge about visibility learned +when your crate gains an external API. + +We believe we will simplify the system by requiring users to learn and do less +stuff. + +We also believe reducing the syntax improves the ergonomics for advanced users +as well. Many of these declarations tend to feel like 'boilerplate' - you write +the only thing you could possibly write. You may forget to write a `mod` +statement, leading to a compiler error (worsening your edit-compile-debug +cycle). We, the RFC authors, frequently have this experience. + +## Path confusion + +A problem, recognized by many users, is a confusion that exists about how paths +work in Rust. Though this has often been glossed as confusion about the +difference between absolute and relative paths, we don't believe this is true. +Very many languages use absolute paths for their import statement, but relative +paths internally to a module, without demonstrating the same confusion. We +believe the confusion has two causes, both addressed by this RFC. + +### Differences between crate root and other modules + +In the crate root, absolute and relative paths are the same today. In other +languages this tends not to be the case. Instead, a common system is to place +the root of the current package under some namespace; this RFC proposes to do +just that, using the new `crate` namespace. + +This will make absolute paths different from relative paths in every module, +including the crate root, so that users do not get mislead into believing that +`use` statements are relative, or that all paths are absolute. + +### Multiple keywords import things + +Another problem with the current system is that `extern crate` and `mod` bring +names into scope, but this is not immediately obvious. Users who have not +grasped this can be confused about how names come into scope, believing that +dependencies are inherently in scope in the root, for example (though it is +actually `extern crate` which brings them into scope). Some users have even +expressed confusion about how it seems to them that `extern crate` and `mod` +are "just" fancy import statements that they have to use in seemingly arbitrary +cases, not understanding that they are using them to construct the hierarchy +used by the `use` statements. + +We solve this problem by removing both the `extern crate` and `mod` statements +from the language. + +## Nonlocal reasoning + +Another frustration of the current module system - which affects advanced users +at least as much as newer users - is the way that visibility is highly +nonlocal. When an item is marked `pub` today, its very unclear if it is +actually `pub`. We make this much clearer by the combination of two choices in +this RFC: + +- All items that are `export` (equivalent of today's `pub`) must *actually* be + exposed in the external API, or the user receives an error. +- All modules are crate visible by default, so every visibility less than + `export` is inherently self-descriptive. + +[too-confusing]: https://withoutboats.github.io/blog/rust/2017/01/04/the-rust-module-system-is-too-confusing.html +[revisiting]: https://aturon.github.io/blog/2017/07/26/revisiting-rusts-modules/ +[learning-modules]: https://gist.github.com/aturon/2f10f19f084f39330cfe2ee028b2ea0c diff --git a/0000-modules/overview.md b/0000-modules/overview.md new file mode 100644 index 00000000000..d06a9d809ef --- /dev/null +++ b/0000-modules/overview.md @@ -0,0 +1,190 @@ +# Overview of the new system + +This is intended to be a "guide-level" overview of the new module system, to +give readers a sense of how things will work after this RFC is fully +implemented. + +This document is divided into two sections: one about structuring your project, +and the other about crafting your API. This distinction is intentional: more +users contribute to binary projects than to libraries, and it is especially +true that new users are more likely to start with binaries than libraries. +Binaries do not have significant public APIs. Libraries, in contrast, care that +their public APIs are well-structured & also stable across versions. + +This system is intended to be incremental - users contributing to binary +projects only need to worry about half of the system, whereas the second half +can be reserved to users contributing to libraries. The first section, about +internal project structure, applies to both binaries and libraries. The second +section applies only to users writing libraries. + +## Structuring your project + +### Adding an external dependency to your project + +To add a new dependency to a project managed with cargo, users need to edit +their `Cargo.toml`. This is the only thing they need to do to make that +dependency available to them: + +```toml +[dependencies] +serde = "1.0.0" +``` + +Once a user has done this, they will be able to use anything from that +dependency's public API in their project, using the `use` statement or absolute +paths. These paths look exactly like they look today: + +```rust +use std::collections::HashMap; +use serde::Serialize; +::std::iter::Iterator +::serde::de::DeserializeOwned +``` + +The major difference from today is that `extern crate` declarations are no +longer necessary. This change however is covered in [RFC #2088][extern-crate], +not directly by this RFC. + +### Adding a new file to your crate + +When a user wants to break their project into multiple files, they can do so by +creating a new `.rs` file in the directory that contains their crate root (or a +subdirectory thereof). For most users, this would be the `src` directory. + +This file will automatically be picked up to create a new **module** of the +user's crate. The precise mechanism is discussed in [another +section][loading-files] of the RFC. In brief, cargo will find the file and tell +rustc to treat it as a new module using an interface that is also available to +non-cargo users. Cargo users who do not want this behavior to be automatic can +instead specify their modules using a field in their `Cargo.toml`. + +The major difference from today is that a `mod` statement is no longer +necessary to get rustc to look up and load your files. The `mod` keyword still +exists to support inline modules (`mod foo { ... }`). + +### Making items public to the rest of your crate + +By default, all modules are public to the rest of the crate. However, all +items inside them - like types, functions, traits, etc - are private to that +module. To make them public to the rest of the crate, the can use the `pub` +keyword: + +```rust +// Private to this module +struct Foo; + +// Public to the crate +pub struct Bar; +``` + +Users can make it public to only part of your crate using the `pub(restricted)` +syntax that already exists. + +There are two significant differences from today: + +* All modules which are automatically loaded are public to the rest of the +crate, rather than taking a visibility modifier. +* The `pub` keyword means *public to this crate* - the equivalent of today's +`pub(crate)`. A new keyword is introduced to mean a part of the public API; +this is discussed in a subsequent section. + +### Importing items from other parts of your crate + +Once a user has a `pub` item in another module, they can import it using the +`use` statement, just like items from external dependencies. Paths to items in +modules of this crate work like this: + +* All these paths begin with the `crate` keyword, which means "this crate." +* All modules in the `src` directory are mounted in the root module, all +modules in other directories are mounted inside the module for that directory. +* Items are mounted in the module that defines them. + +So if a user has a structure like this: + +``` +// src/foo.rs +pub struct Bar; +``` + +They can access it with `use` and absolute paths like this: + +```rust +use crate::foo::Bar; +::crate::foo::Bar; +``` + +The major difference from today is that the `crate` keyword is used to prefix +items inside this crate, so that users can distinguish external dependencies +from modules in this crate by only looking at the import statement. + +## Crafting your API + +### Exporting things from your crate + +When a user writing a library wants to declare something exported from their +crate, they can use the `export` visibility modifier to declare it as an +exported item: + +```rust +// This is exported +export struct Foo; +``` + +Items marked `export` inside of files which have not been exported are not +visible outside of this crate, because the full path to that item is not +`exported`. + +If a user wants to make one of their file submodules a part of their API, they +can do so using the `export keyword (no `use`), followed by the name of the +module, in that module's parent: + +```rust +// In `src/lib.rs` Imagine that there is a `src/foo.rs` as well. +export foo; +``` + +(This is actually an instance of the re-export functionality described in the +next section.) + +The major difference from today is that `export` has been added as a keyword, +meaning the same thing that `pub` does today. + +### Re-exporting items to simplify your public API + +Sometimes users want to make items public using a path hierarchy that's +different from the true path hierarchy which makes up the crate. This can be to +simplify the API, or to maintain backwards compatibility. Users can do this by +using the `export` keyword with a relative path (just like all non-`use` +paths): + +```rust +export foo::Foo; +export bar::Bar; +``` + +This will create a new exported path to the item, even if the canonical path to +the item is not exported. The item itself needs to be exported, or this is an +error. + +This replaces the functionality of `pub use` - users can re-export with a +visibility modifier and a path without a `use` statement. + +## Deprecations + +Over time, this RFC proposes to deprecate and ultimately remove this syntax +that exists today: + +* `mod $ident;` - mod statements used to pick up new `.rs` files +* `extern crate $ident;` - extern crate statements +* `pub use` - re-exports using the `use` keyword +* `pub(crate)` - the `crate` visibility is no longer necessary + +All paths inside a module are relative to that module, just like today. +However, you can make them absolute using the `::` prefix. + +Paths prefixed `::` are from the absolute root, which is not (unlike today) the +root of this crate. To access the root of this crate, you need to prefix paths +`crate`. + +[extern-crate]: https://github.com/rust-lang/rfcs/pull/2088 +[loading-files]: /0000-modules/detailed-design/loading-files.md From 8f0b81a5596df1b240d9a42fb5b1b4f8e9878307 Mon Sep 17 00:00:00 2001 From: Without Boats Date: Mon, 14 Aug 2017 11:43:59 -0700 Subject: [PATCH 2/3] Fix links. --- 0000-modules/README.md | 12 ++++++------ 0000-modules/detailed-design/loading-files.md | 2 +- .../detailed-design/visibility-and-reexport.md | 2 +- 0000-modules/overview.md | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/0000-modules/README.md b/0000-modules/README.md index 93d84d10ed2..59a645dd103 100644 --- a/0000-modules/README.md +++ b/0000-modules/README.md @@ -26,9 +26,9 @@ sections: Each of the detailed design subsections contains its own description of drawbacks and alternatives. -[motivation]: 0000-modules/motivation.md -[overview]: 0000-modules/overview.md -[loading-files]: 000-modules/detailed-design/loading-files.md -[paths]: 000-modules/detailed-design/paths.md -[visibility]: 000-modules/detailed-design/visibility-and-reexport.md -[migration]: 000-modules/detailed-design/migration.md +[motivation]: motivation.md +[overview]: overview.md +[loading-files]: detailed-design/loading-files.md +[paths]: detailed-design/paths.md +[visibility]: detailed-design/visibility-and-reexport.md +[migration]: detailed-design/migration.md diff --git a/0000-modules/detailed-design/loading-files.md b/0000-modules/detailed-design/loading-files.md index 4774708da89..5e8d75309eb 100644 --- a/0000-modules/detailed-design/loading-files.md +++ b/0000-modules/detailed-design/loading-files.md @@ -191,4 +191,4 @@ making all files "inlined" into their directory and/or using the filename to determine the visibility of a module. We've decided not to steps that are this radical right now. -[migrations]: 0000-modules/detailed-design/migrations.md +[migrations]: detailed-design/migrations.md diff --git a/0000-modules/detailed-design/visibility-and-reexport.md b/0000-modules/detailed-design/visibility-and-reexport.md index de83a771dba..b0abc8f7193 100644 --- a/0000-modules/detailed-design/visibility-and-reexport.md +++ b/0000-modules/detailed-design/visibility-and-reexport.md @@ -171,4 +171,4 @@ appealing properties, such as grepability and clarity of naming. We could make the error on a non-exposed `export` item a warning instead of a hard error. -[migration]: 0000-modules/detailed-design/migration.md +[migration]: detailed-design/migration.md diff --git a/0000-modules/overview.md b/0000-modules/overview.md index d06a9d809ef..ea62db26e9f 100644 --- a/0000-modules/overview.md +++ b/0000-modules/overview.md @@ -187,4 +187,4 @@ root of this crate. To access the root of this crate, you need to prefix paths `crate`. [extern-crate]: https://github.com/rust-lang/rfcs/pull/2088 -[loading-files]: /0000-modules/detailed-design/loading-files.md +[loading-files]: detailed-design/loading-files.md From a2e1cb235bb5058332a540fb3687c66dcb634586 Mon Sep 17 00:00:00 2001 From: Without Boats Date: Mon, 14 Aug 2017 14:14:44 -0700 Subject: [PATCH 3/3] Fix some typos. --- 0000-modules/detailed-design/loading-files.md | 2 +- 0000-modules/detailed-design/visibility-and-reexport.md | 2 +- 0000-modules/overview.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/0000-modules/detailed-design/loading-files.md b/0000-modules/detailed-design/loading-files.md index 5e8d75309eb..e2ff2d75f70 100644 --- a/0000-modules/detailed-design/loading-files.md +++ b/0000-modules/detailed-design/loading-files.md @@ -191,4 +191,4 @@ making all files "inlined" into their directory and/or using the filename to determine the visibility of a module. We've decided not to steps that are this radical right now. -[migrations]: detailed-design/migrations.md +[migrations]: migrations.md diff --git a/0000-modules/detailed-design/visibility-and-reexport.md b/0000-modules/detailed-design/visibility-and-reexport.md index b0abc8f7193..62033e3fd4a 100644 --- a/0000-modules/detailed-design/visibility-and-reexport.md +++ b/0000-modules/detailed-design/visibility-and-reexport.md @@ -171,4 +171,4 @@ appealing properties, such as grepability and clarity of naming. We could make the error on a non-exposed `export` item a warning instead of a hard error. -[migration]: detailed-design/migration.md +[migration]: migration.md diff --git a/0000-modules/overview.md b/0000-modules/overview.md index ea62db26e9f..941b25b8a5c 100644 --- a/0000-modules/overview.md +++ b/0000-modules/overview.md @@ -135,7 +135,7 @@ visible outside of this crate, because the full path to that item is not `exported`. If a user wants to make one of their file submodules a part of their API, they -can do so using the `export keyword (no `use`), followed by the name of the +can do so using the `export` keyword (no `use`), followed by the name of the module, in that module's parent: ```rust