Skip to content

rustdoc: Add flag to use external source code pages #69167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

GuillaumeGomez
Copy link
Member

Fixes #67804

cc @jyn514
cc @ollie27

r? @kinnison

through an HTTP URL!), you can use this option:

```bash
$ rustdoc src/lib.rs -Z unstable-options --source-code-external-url 'https://somewhere.com'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's a good idea to use an example domain for the purpose of being used in examples, like https://example.com as per https://www.iana.org/domains/reserved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the difference with what I wrote?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

somewhere.com could be owned by anyone, and they could put any content there. example.com on the other hand is registered by the Internet Assigned Numbers Authority, and it's safe to use in examples.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't make sense to use it directly like this... Originally, I wanted to put http://a.a...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure where the misunderstanding is, but put simply, example.com is safe to use in examples, while any other random domain name is not (because anyone can grab those and host any content; example.com will always be safe). Sorry about the noise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't really understand your point... Like I said, if you use it as is, it'll just not do anything except creating links to "somewhere.com". Which isn't what any user want. If you're still not convinced, I propose you the following: open a PR changing the domain I'm pointing to in the example. I'll merge it without problem. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that sounds good to me if we can settle there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Then once this is merged, please open a PR and ping me on it so it can get merged quickly.

@rust-highfive
Copy link
Contributor

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-02-14T13:34:08.5336284Z ========================== Starting Command Output ===========================
2020-02-14T13:34:08.5353539Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/482fad44-62ae-49d1-9ed4-a8257dc496ff.sh
2020-02-14T13:34:08.7936640Z 
2020-02-14T13:34:08.8003351Z ##[section]Finishing: Disable git automatic line ending conversion
2020-02-14T13:34:08.8009929Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-02-14T13:34:08.8011640Z Task         : Get sources
2020-02-14T13:34:08.8011722Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-02-14T13:34:08.8011756Z Version      : 1.0.0
2020-02-14T13:34:08.8011789Z Author       : Microsoft
---
2020-02-14T13:34:11.5428076Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-02-14T13:34:11.5637841Z ##[command]git config gc.auto 0
2020-02-14T13:34:11.5706166Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-02-14T13:34:11.5761690Z ##[command]git config --get-all http.proxy
2020-02-14T13:34:11.5907696Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69167/merge:refs/remotes/pull/69167/merge
---
2020-02-14T14:32:53.7895233Z .................................................................................................... 1700/9637
2020-02-14T14:32:58.5595216Z .................................................................................................... 1800/9637
2020-02-14T14:33:10.0267719Z ..............................i..................................................................... 1900/9637
2020-02-14T14:33:17.4277244Z .................................................................................................... 2000/9637
2020-02-14T14:33:31.5331886Z ....................iiiii........................................................................... 2100/9637
2020-02-14T14:33:41.0036162Z .................................................................................................... 2300/9637
2020-02-14T14:33:43.3602721Z .................................................................................................... 2400/9637
2020-02-14T14:33:47.9862320Z .................................................................................................... 2500/9637
2020-02-14T14:34:08.6056435Z .................................................................................................... 2600/9637
---
2020-02-14T14:36:40.1433748Z ........................................................................i...............i........... 4900/9637
2020-02-14T14:36:47.6166188Z .................................................................................................... 5000/9637
2020-02-14T14:36:55.3782486Z .................................................................................................... 5100/9637
2020-02-14T14:36:59.9235655Z ..............i..................................................................................... 5200/9637
2020-02-14T14:37:10.7553444Z ........................................................................................ii.ii....... 5300/9637
2020-02-14T14:37:14.5778108Z .i...i.............................................................................................. 5400/9637
2020-02-14T14:37:24.5016095Z .................................................................................................... 5600/9637
2020-02-14T14:37:34.3064978Z .............................................................................i...................... 5700/9637
2020-02-14T14:37:41.5894780Z .................................................................................................... 5800/9637
2020-02-14T14:37:47.5978959Z ...........................................................................i........................ 5900/9637
2020-02-14T14:37:47.5978959Z ...........................................................................i........................ 5900/9637
2020-02-14T14:37:57.0826504Z .....................................................................ii...i..ii...........i......... 6000/9637
2020-02-14T14:38:17.8045422Z .................................................................................................... 6200/9637
2020-02-14T14:38:25.3075217Z .................................................................................................... 6300/9637
2020-02-14T14:38:32.9952403Z .................................................................................................i.. 6400/9637
2020-02-14T14:38:48.0398921Z ii.................................................................................................. 6500/9637
---
2020-02-14T14:40:49.5081726Z .................................................................................................... 7600/9637
2020-02-14T14:40:53.7282509Z .................................................................................................... 7700/9637
2020-02-14T14:40:59.6753857Z .................................................................................................... 7800/9637
2020-02-14T14:41:07.3397902Z .................................................................................................... 7900/9637
2020-02-14T14:41:15.9503231Z ............................................................................iiiiiii.i............... 8000/9637
2020-02-14T14:41:31.6225739Z ................i......i............................................................................ 8200/9637
2020-02-14T14:41:36.9866830Z .................................................................................................... 8300/9637
2020-02-14T14:41:49.8050733Z .................................................................................................... 8400/9637
2020-02-14T14:41:59.4124178Z .................................................................................................... 8500/9637
---
2020-02-14T14:44:21.1033932Z  finished in 7.115
2020-02-14T14:44:21.1237752Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:44:21.3241323Z 
2020-02-14T14:44:21.3242496Z running 178 tests
2020-02-14T14:44:24.1508872Z iiii......i...........ii..iiii...i....i...........i............i..i..................i....i......... 100/178
2020-02-14T14:44:26.4156500Z ...i.i.i...iii..iiiiiiiiiiiiiiii.......................iii............ii......
2020-02-14T14:44:26.4159703Z 
2020-02-14T14:44:26.4163848Z  finished in 5.292
2020-02-14T14:44:26.4381033Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:44:26.6101490Z 
---
2020-02-14T14:44:28.5087203Z  finished in 2.070
2020-02-14T14:44:28.5284366Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:44:28.6851055Z 
2020-02-14T14:44:28.6851237Z running 9 tests
2020-02-14T14:44:28.6853801Z iiiiiiiii
2020-02-14T14:44:28.6855126Z 
2020-02-14T14:44:28.6855310Z  finished in 0.156
2020-02-14T14:44:28.7030090Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:44:28.8867090Z 
---
2020-02-14T14:44:48.3962117Z  finished in 19.693
2020-02-14T14:44:48.4196304Z Check compiletest suite=debuginfo mode=debuginfo (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:44:48.6170432Z 
2020-02-14T14:44:48.6170987Z running 116 tests
2020-02-14T14:45:01.7207809Z iiiii..i.....i..i...i..i.i.i..i..i..ii....i.i....ii..........iiii..........i.....i..i.......ii.i.ii. 100/116
2020-02-14T14:45:03.5887892Z ....iiii.....ii.
2020-02-14T14:45:03.5888385Z 
2020-02-14T14:45:03.5890499Z  finished in 15.170
2020-02-14T14:45:03.5896267Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T14:45:03.5896588Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2020-02-14T14:58:05.0264261Z 
2020-02-14T14:58:05.0265052Z    Doc-tests core
2020-02-14T14:58:09.7235703Z 
2020-02-14T14:58:09.7236419Z running 2471 tests
2020-02-14T14:58:18.8347191Z ......iiiii......................................................................................... 100/2471
2020-02-14T14:58:27.6596465Z ..................................................................................ii................ 200/2471
2020-02-14T14:58:47.9168138Z .................i.................................................................................. 400/2471
2020-02-14T14:58:47.9168138Z .................i.................................................................................. 400/2471
2020-02-14T14:58:57.2707780Z ......................................................................i..i..................iiii.... 500/2471
2020-02-14T14:59:13.3746431Z .................................................................................................... 700/2471
2020-02-14T14:59:21.6547768Z .................................................................................................... 800/2471
2020-02-14T14:59:29.8729411Z .................................................................................................... 900/2471
2020-02-14T14:59:38.1196123Z .................................................................................................... 1000/2471
---
2020-02-14T15:03:09.2178570Z 
2020-02-14T15:03:09.2179526Z running 1009 tests
2020-02-14T15:03:26.4272314Z i................................................................................................... 100/1009
2020-02-14T15:03:36.2014355Z .................................................................................................... 200/1009
2020-02-14T15:03:42.9309436Z ..................iii......i......i...i......i...................................................... 300/1009
2020-02-14T15:03:47.6952026Z .................................................................................................... 400/1009
2020-02-14T15:03:54.1904910Z ............................................i..i.....................................ii............. 500/1009
2020-02-14T15:04:07.2310322Z .................................................................................................... 700/1009
2020-02-14T15:04:07.2310322Z .................................................................................................... 700/1009
2020-02-14T15:04:12.8749707Z ...................................iiii............................................................. 800/1009
2020-02-14T15:04:26.6921521Z .................................................................................................... 900/1009
2020-02-14T15:04:33.2257250Z .........................................................iiii....................................... 1000/1009
2020-02-14T15:04:33.5659363Z test result: ok. 989 passed; 0 failed; 20 ignored; 0 measured; 0 filtered out
2020-02-14T15:04:33.5659499Z 
2020-02-14T15:04:33.5746906Z  finished in 163.726
2020-02-14T15:04:33.5759471Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-02-14T15:23:52.9742307Z  finished in 40.543
2020-02-14T15:23:53.0021466Z Check compiletest suite=run-make-fulldeps mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-02-14T15:23:53.2292689Z 
2020-02-14T15:23:53.2292836Z running 208 tests
2020-02-14T15:24:24.5613042Z ......................i...ii.....................................................................i.. 100/208
2020-02-14T15:24:55.6619363Z ......................................iiiiii......i..............iii................................ 200/208
2020-02-14T15:24:59.0468024Z test result: ok. 193 passed; 0 failed; 15 ignored; 0 measured; 0 filtered out
2020-02-14T15:24:59.0468123Z 
2020-02-14T15:24:59.0473766Z  finished in 66.045
2020-02-14T15:24:59.0481894Z doc tests for: /checkout/src/doc/rustdoc/src/advanced-features.md
---
2020-02-14T15:25:00.9647257Z ---- /checkout/src/doc/rustdoc/src/unstable-features.md - Unstable_features::Unstable_command_line_arguments::__using_external_source_code (line 482) stdout ----
2020-02-14T15:25:00.9647326Z error: expected type, found `}`
2020-02-14T15:25:00.9647553Z  --> /checkout/src/doc/rustdoc/src/unstable-features.md:484:1
2020-02-14T15:25:00.9647598Z   |
2020-02-14T15:25:00.9647640Z 3 | https://somewhere.com/[crate name]
2020-02-14T15:25:00.9647861Z   |      - help: try using a semicolon: `;`
2020-02-14T15:25:00.9647944Z   | ^ expected type
2020-02-14T15:25:00.9647999Z   |
2020-02-14T15:25:00.9647999Z   |
2020-02-14T15:25:00.9648047Z   = note: `#![feature(type_ascription)]` lets you annotate an expression with a type: `<expr>: <type>`
2020-02-14T15:25:00.9648431Z   = note: see issue #23416 <***/issues/23416> for more information
2020-02-14T15:25:00.9648535Z error: aborting due to previous error
2020-02-14T15:25:00.9648562Z 
2020-02-14T15:25:00.9648758Z Couldn't compile the test.
2020-02-14T15:25:00.9648804Z 
---
2020-02-14T15:25:01.5650881Z   local time: Fri Feb 14 15:25:01 UTC 2020
2020-02-14T15:25:01.9252108Z   network time: Fri, 14 Feb 2020 15:25:01 GMT
2020-02-14T15:25:01.9257082Z == end clock drift check ==
2020-02-14T15:25:03.1532334Z 
2020-02-14T15:25:03.1626818Z ##[error]Bash exited with code '1'.
2020-02-14T15:25:03.1639512Z ##[section]Finishing: Run build
2020-02-14T15:25:03.1678431Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-02-14T15:25:03.1680639Z Task         : Get sources
2020-02-14T15:25:03.1680703Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-02-14T15:25:03.1680750Z Version      : 1.0.0
2020-02-14T15:25:03.1680790Z Author       : Microsoft
2020-02-14T15:25:03.1680790Z Author       : Microsoft
2020-02-14T15:25:03.1680835Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-02-14T15:25:03.1680900Z ==============================================================================
2020-02-14T15:25:03.5843314Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-02-14T15:25:03.5885475Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-02-14T15:25:03.6000962Z Cleaning up task key
2020-02-14T15:25:03.6001764Z Start cleaning up orphan processes.
2020-02-14T15:25:03.6111386Z Terminate orphan process: pid (7177) (python)
2020-02-14T15:25:03.6320779Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@GuillaumeGomez GuillaumeGomez force-pushed the source_code_external_url branch from 9dba772 to 368c777 Compare February 14, 2020 15:31
@GuillaumeGomez GuillaumeGomez force-pushed the source_code_external_url branch from 368c777 to 6f09ef6 Compare February 14, 2020 20:32
@GuillaumeGomez
Copy link
Member Author

GuillaumeGomez commented Feb 14, 2020

Updated!

Copy link
Member

@ollie27 ollie27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this restricted to full URLs? That doesn't seem to be what #67804 was requesting.

Is this intended to be used by docs.rs? If so it doesn't look like this will work:

  • rustdoc appends .html to the filenames but docs.rs doesn't
  • docs.rs doesn't support the line number fragments at the end of the URL
  • the paths rustdoc generates are relative to the crate root (typically lib.rs) but docs.rs uses the root of the source archive
  • rustdoc replaces .. in paths with up which doesn't work on docs.rs

Overall I think docs.rs would be better off using redirects. They will have to solve the above issues as well but at least redirects will work for existing docs and links coming from outside docs.rs.

@jyn514
Copy link
Member

jyn514 commented Feb 15, 2020

rustdoc appends .html to the filenames but docs.rs doesn't

Oh that's certainly a problem, I didn't realize that when I read through the code. Can this be changed?

docs.rs doesn't support the line number fragments at the end of the URL

We can add this later and it doesn't affect anything in the meantime. We would have the same problem with redirects anyway.

the paths rustdoc generates are relative to the crate root (typically lib.rs) but docs.rs uses the root of the source archive

I don't see why this is an issue? We can just pass --source-root .../:crate/source instead of /:crate alone.

rustdoc replaces .. in paths with up which doesn't work on docs.rs

I'm not sure what you mean by this, can you expand a little more?

Overall I think docs.rs would be better off using redirects. They will have to solve the above issues as well

We will have to use redirects even if this is merged because it will only affect new crates, not crates with existing documentation. The benefit of this flag is

  • it doesn't generate the source HTML files, saving CPU/IO/disk space
  • it avoids an unnecessary redirect when people click src, which could be helpful on high-latency connections

@jyn514
Copy link
Member

jyn514 commented Feb 15, 2020

Oops, I missed

Why is this restricted to full URLs?

I don't think this will affect us one way or another since we can hardcode https://docs.rs/ at the front. It would be nice to use relative paths though since we have long term dreams (too far off to call them goals 😆) of companies/other people being able to host their own version of the site.

@ollie27
Copy link
Member

ollie27 commented Feb 16, 2020

the paths rustdoc generates are relative to the crate root (typically lib.rs) but docs.rs uses the root of the source archive

I don't see why this is an issue? We can just pass --source-root .../:crate/source instead of /:crate alone.

Well docs.rs would have to pass something like https://docs.rs/crate/lazy_static/1.4.0/source/src/, noting the src at the end which won't be the same for every crate.

rustdoc replaces .. in paths with up which doesn't work on docs.rs

I'm not sure what you mean by this, can you expand a little more?

Say you have a crate foo which contains:

#[path = "../bar.rs"]
pub mod bar;

The source file for bar.rs that rustdoc will create is src/foo/up/bar.rs.html.

I've also noticed that this PR appends the crate name to the passed in URL which again doesn't match what docs.rs would need at all.

For this to work cross crate docs.rs will have to not only pass the location of the current crate's source but also the location for all dependent crates in a similar way to --extern-html-root-url.

Copy link
Contributor

@kinnison kinnison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of typos, and Ollie's concerns remain to consider.

@kinnison kinnison added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Feb 18, 2020
@GuillaumeGomez GuillaumeGomez force-pushed the source_code_external_url branch from 6f09ef6 to b5c573b Compare March 6, 2020 13:18
@GuillaumeGomez
Copy link
Member Author

I updated how I handle the path for the external source code. I think it answers all issues reported?

@@ -1550,6 +1553,21 @@ impl Context {
}
}

fn compute_path(path: String) -> String {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to use canonicalize() or a URL library instead of writing this from scratch? This does not handle symbolic links for example.

@jyn514
Copy link
Member

jyn514 commented Mar 6, 2020

For this to work cross crate docs.rs will have to not only pass the location of the current crate's source but also the location for all dependent crates

I'm a little confused by this point - I thought source links only showed up in the documentation page for that struct? As long as clicking on the external type takes you to the right place, I don't see why there would need to be special handling for source urls.

E.g. https://docs.rs/cranelift-faerie/0.59.0/cranelift_faerie/struct.FaerieProduct.html has a link to Artifact which takes you to a page in faerie which then has it's own src link that's separate from when cranelift-faerie was built.

@rust-highfive
Copy link
Contributor

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-03-09T08:30:14.6346656Z ========================== Starting Command Output ===========================
2020-03-09T08:30:14.6352507Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/50537a77-6f5a-4db2-bcb5-85e842f856e7.sh
2020-03-09T08:30:14.6353029Z 
2020-03-09T08:30:14.6357945Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-09T08:30:14.6379604Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-03-09T08:30:14.6383813Z Task         : Get sources
2020-03-09T08:30:14.6384140Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-09T08:30:14.6384455Z Version      : 1.0.0
2020-03-09T08:30:14.6384668Z Author       : Microsoft
---
2020-03-09T08:30:16.2322874Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-09T08:30:16.2330361Z ##[command]git config gc.auto 0
2020-03-09T08:30:16.2334766Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-09T08:30:16.2339024Z ##[command]git config --get-all http.proxy
2020-03-09T08:30:16.2347906Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69167/merge:refs/remotes/pull/69167/merge
---
2020-03-09T08:39:37.6954935Z     Checking rustdoc v0.0.0 (/checkout/src/librustdoc)
2020-03-09T08:39:37.9675252Z error: named argument never used
2020-03-09T08:39:37.9678097Z     --> src/librustdoc/html/render.rs:1629:29
2020-03-09T08:39:37.9679074Z      |
2020-03-09T08:39:37.9680032Z 1627 |                     "{root}/{path}",
2020-03-09T08:39:37.9682504Z 1628 |                     root = source_code_external_url,
2020-03-09T08:39:37.9683546Z 1629 |                     krate = krate,
2020-03-09T08:39:37.9685130Z      |                             ^^^^^ named argument never used
2020-03-09T08:39:37.9685802Z 
---
2020-03-09T08:39:41.0014982Z   local time: Mon Mar  9 08:39:40 UTC 2020
2020-03-09T08:39:41.2861693Z   network time: Mon, 09 Mar 2020 08:39:41 GMT
2020-03-09T08:39:41.2866543Z == end clock drift check ==
2020-03-09T08:39:41.7427270Z 
2020-03-09T08:39:41.7482563Z ##[error]Bash exited with code '1'.
2020-03-09T08:39:41.7497921Z ##[section]Finishing: Run build
2020-03-09T08:39:41.7553708Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-03-09T08:39:41.7559366Z Task         : Get sources
2020-03-09T08:39:41.7559756Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-09T08:39:41.7560115Z Version      : 1.0.0
2020-03-09T08:39:41.7560406Z Author       : Microsoft
2020-03-09T08:39:41.7560406Z Author       : Microsoft
2020-03-09T08:39:41.7560804Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-03-09T08:39:41.7561266Z ==============================================================================
2020-03-09T08:39:42.1192906Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-03-09T08:39:42.1242140Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/69167/merge to s
2020-03-09T08:39:42.1332882Z Cleaning up task key
2020-03-09T08:39:42.1334723Z Start cleaning up orphan processes.
2020-03-09T08:39:42.1539067Z Terminate orphan process: pid (3554) (python)
2020-03-09T08:39:42.1682949Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@ollie27
Copy link
Member

ollie27 commented Mar 10, 2020

For this to work cross crate docs.rs will have to not only pass the location of the current crate's source but also the location for all dependent crates

I'm a little confused by this point - I thought source links only showed up in the documentation page for that struct? As long as clicking on the external type takes you to the right place, I don't see why there would need to be special handling for source urls.

I was referring to cross crate inlining. For example on the page for std::vec::Vec the [src] link points to the source files for the alloc crate. For docs.rs rustdoc currently derives the src links for external crates from --extern-html-root-url but if we want to avoid redirects that would need to change.

@jyn514
Copy link
Member

jyn514 commented Mar 10, 2020

I was referring to cross crate inlining. For example on the page for std::vec::Vec the [src] link points to the source files for the alloc crate.

Is it possible to do something like that outside of std? It seems very strange to me to have the docs for an item not be in the crate it was defined in.

@GuillaumeGomez
Copy link
Member Author

We could allow to have formattable values in the parameter for the crate. For example:

--source-code-external-url path/{crate}

Or we could add another parameter to allow cross linking in such case (where the "{crate}" would still make sense).

In any case, having cross-crates linking is a must have if we want to keep the same level of quality for documentation generation.

What do you think?

@jyn514
Copy link
Member

jyn514 commented Mar 10, 2020

Can you give an example of how cross-crate source linking is used on docs.rs today (since we don't host std)? I'm still confused what this would look like for non-std crates.

None of these suggestions are very appealing unfortunately, we'd have to add a great deal more redirections to make the {crate} idea work since the prefixes are completely different.

@GuillaumeGomez
Copy link
Member Author

I thought we generated [src] links for reexports but apparently we don't. Therefore, there is no need for this {crate} thing. Anything else to sort out?

@jyn514
Copy link
Member

jyn514 commented Mar 15, 2020

I think the only remaining issue for docs.rs is #69167 (comment), especially /up. The bit about src/ varying we can probably work around by parsing Cargo.toml

@Elinvynia Elinvynia added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 10, 2020
@bors
Copy link
Collaborator

bors commented Jun 11, 2020

☔ The latest upstream changes (presumably #73235) made this pull request unmergeable. Please resolve the merge conflicts.

@Elinvynia Elinvynia added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 25, 2020
@crlf0710 crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 17, 2020
@Muirrum Muirrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 6, 2020
@Dylan-DPC-zz
Copy link

r? @jyn514

@jyn514
Copy link
Member

jyn514 commented Aug 19, 2020

Going through the concerns one at a time (and feel free to bring up any I missed):

Maybe it's a good idea to use an example domain for the purpose of being used in examples, like https://example.com as per https://www.iana.org/domains/reserved.

I think it would be better to fix this here instead of immediately making a follow-up PR.

Wouldn't it be better to use canonicalize() or a URL library instead of writing this from scratch? This does not handle symbolic links for example.

I still have this opinion but I'm willing to be convinced otherwise. It's secondary anyway, this needs design more than it needs implementation IMO.

rustdoc appends .html to the filenames but docs.rs doesn't

This still hasn't been addressed. Maybe it's enough to use path.trim_end_matches(".html")?

docs.rs doesn't support the line number fragments at the end of the URL

This can be fixed later on the docs.rs end, I don't think it should block the rustdoc implementation.

the paths rustdoc generates are relative to the crate root (typically lib.rs) but docs.rs uses the root of the source archive

As I mentioned above, docs.rs can add the crate root in --extern-source-url - we can find it from path = "..." in Cargo.toml, defaulting to src. I don't think it should block the rustdoc implementation.

rustdoc replaces .. in paths with up which doesn't work on docs.rs

Do we know why rustdoc does this? Could we use ../ instead?

For this to work cross crate docs.rs will have to not only pass the location of the current crate's source but also the location for all dependent crates

I don't see an easy way around this ... so I guess the implementation will have to allow passing --extern-source-url for multiple crates, not just the one being documented.

@Lonami
Copy link
Contributor

Lonami commented Aug 19, 2020

For this to work cross crate docs.rs will have to not only pass the location of the current crate's source but also the location for all dependent crates

I don't see an easy way around this

Would it be a safe assumption to make that, by default, external crates will also have their sources hosted there? If someone is hosting their own source, maybe they would go through the effort to host the rest of the dependencies too, it's hard to say (maybe need to ask the people who want this feature about this).

Definitely the cleanest option would be having and looking at per-crate metadata by default to see what external URL to use, and trusting the authors in that their external sources work (otherwise it's their problem), as opposed to using a command line flag.

@jyn514
Copy link
Member

jyn514 commented Aug 19, 2020

@Lonami sure, that's an assumption docs.rs can make. But rustdoc doesn't know the URL patterns docs.rs uses, and can't derive them from the crate names.

@jyn514
Copy link
Member

jyn514 commented Aug 19, 2020

For example, here's futures::join!: https://docs.rs/futures/0.3.5/futures/macro.join.html
And here's the source code: https://docs.rs/crate/futures-util/0.3.5/source/src/async_await/join_mod.rs
Notice how the extern source url in this case is https://docs.rs/crate/futures-util/0.3.5/source/, not https://docs.rs/futures/0.3.5/src. I guess we could use the {crate}/{version} format string @GuillaumeGomez was suggesting?

@crlf0710 crlf0710 marked this pull request as draft September 4, 2020 16:20
@jyn514
Copy link
Member

jyn514 commented Aug 16, 2021

@GuillaumeGomez are you planning to follow up on this?

@GuillaumeGomez
Copy link
Member Author

Yes, just need to update it.

@camelid
Copy link
Member

camelid commented Dec 10, 2021

triage: @GuillaumeGomez any updates?

@GuillaumeGomez
Copy link
Member Author

Completely forgot about it... Will update it as soon as possible.

@camelid
Copy link
Member

camelid commented Dec 11, 2021

Actually, I just remembered that there was still discussion in #67804 about whether this should be added.

@camelid camelid changed the title Source code external url rustdoc: Add flag to use external source code pages Dec 11, 2021
@jyn514
Copy link
Member

jyn514 commented Jan 4, 2022

r? @camelid (although I kinda lean towards just closing this)

@rust-highfive rust-highfive assigned camelid and unassigned jyn514 Jan 4, 2022
@camelid
Copy link
Member

camelid commented Jan 8, 2022

This PR has been sitting for so long and has accrued enough conflicts that I think it's probably best to just close it. We can continue discussion on the issue.

@camelid camelid closed this Jan 8, 2022
@GuillaumeGomez GuillaumeGomez deleted the source_code_external_url branch August 19, 2024 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a rustdoc flag to show the source code externally