-
Notifications
You must be signed in to change notification settings - Fork 283
Add crisp tools #1505
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add crisp tools #1505
Changes from 21 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
4aa7d01
split_ffi_entry_points: initial commit; comment-preserving TokenStrea…
spernsteiner cc34964
split_ffi: working implementation
spernsteiner b3345e3
split_ffi: fix warnings
spernsteiner dfdf319
split_ffi: fix rust-analyzer version in cargo.toml
spernsteiner ab34c14
split_ffi: refuse to rewrite files outside cargo dir
spernsteiner 155045d
split_ffi: refactor: move TokenStream rewriting into new rewrite_util…
spernsteiner 45cd826
rename rewrite_util -> rust_util::rewrite
spernsteiner c994b35
rust_util: add FileCollector
spernsteiner f740a77
tools/split_rust: initial commit
spernsteiner f88e410
rust_util: track module path in FileCollector
spernsteiner bcc86c6
split_rust: output code snippets instead of spans
spernsteiner 1091e04
refactor: move ItemSpanVisitor from split_rust into rust_util
spernsteiner 8e8456f
add tools/merge_rust/
spernsteiner c4effc6
tools/merge_rust: use clap for arguments
fw-immunant 7dc2fb8
tools/split_rust: use clap for arguments
fw-immunant 40f4a4d
tools/split_ffi_entry_points: use clap for arguments
fw-immunant a0a4e76
tools/split_ffi_entry_points: use "visit" syn feature
fw-immunant a2a241f
tools/rust_util: use "visit" syn feature
fw-immunant 265e101
tools/rust_util: do not include r# prefix in fs paths
fw-immunant 60776ca
cargo: exclude tools/ from Cargo workspace
fw-immunant b14877a
tools: add README
fw-immunant c689e5e
tools: add rust-toolchain.toml, fixing README build command with rustup
fw-immunant 44f79cc
tools: use match ergonomics
fw-immunant 6739500
tools: move doc comment above attributes
fw-immunant 84450a9
tools: rustfmt
fw-immunant File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,7 @@ exclude = [ | |
| "analysis/tests", | ||
| "examples", | ||
| "tests", | ||
| "tools", | ||
| ] | ||
|
|
||
| [workspace.package] | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # C2Rust CRISP tools | ||
|
|
||
| - `merge_rust` | ||
| - `split_rust` | ||
| - `split_ffi_entry_points` | ||
|
|
||
| ## Building | ||
|
|
||
| These tools rely on rust-analyzer's libraries, so they need a relatively recent version of Rust. Run `cargo build --release` in their respective directories to build them. | ||
|
|
||
| ## Running | ||
|
|
||
| `split_rust` and `merge_rust` expect the root source file of a Rust project as their first argument, e.g. `lib.rs` or `main.rs`. | ||
|
|
||
| - `merge_rust` modifies the specified codebase in-place. | ||
| - `split_rust` emits JSON on standard output. | ||
|
|
||
| `split_ffi_entry_points` expects a Rust project directory (the directory containing a `Cargo.toml` file) as its only argument. | ||
|
|
||
| It modifies that Rust project in-place. |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| [package] | ||
| name = "merge_rust" | ||
| version = "0.1.0" | ||
| edition = "2024" | ||
|
|
||
| [dependencies] | ||
| rust_util = { path = "../rust_util" } | ||
|
|
||
| syn = { version = "2.0.108", features = ["full", "visit"] } | ||
| proc-macro2 = { version = "1", features = ["span-locations"] } | ||
| clap = { version = "4.5", features = ["derive"] } | ||
| quote = "1" | ||
|
|
||
| serde_json = "1" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| use std::collections::HashMap; | ||
| use std::fs::{self, File}; | ||
| use std::path::PathBuf; | ||
| use rust_util::collect::FileCollector; | ||
| use rust_util::item_span::item_spans; | ||
| use serde_json; | ||
| use clap::Parser; | ||
|
|
||
| #[derive(Parser)] | ||
| /// Merge updated item definitions into a Rust codebase. | ||
| struct Args { | ||
| /// Root Rust source file to update (`lib.rs` or `main.rs`). | ||
| src_root_path: PathBuf, | ||
| /// JSON file containing mapping from Rust item paths to desired new contents. | ||
| new_snippets_file: PathBuf, | ||
| } | ||
|
|
||
| fn main() { | ||
| let args = Args::parse(); | ||
| let src_root_path = args.src_root_path; | ||
| let new_snippet_json_path = args.new_snippets_file; | ||
|
|
||
| let new_snippets_file = File::open(&new_snippet_json_path).unwrap(); | ||
| let new_snippets: HashMap<String, String> = | ||
| serde_json::from_reader(new_snippets_file).unwrap(); | ||
|
|
||
| let mut fc = FileCollector::default(); | ||
| fc.parse(&src_root_path, vec![], true).unwrap(); | ||
| for &(ref file_path, ref mod_path, ref ast) in &fc.files { | ||
| eprintln!("visit {:?}", file_path); | ||
| let old_src = fs::read_to_string(file_path).unwrap(); | ||
|
|
||
| let mut rewrites = Vec::new(); | ||
| for (item_path, lo, hi) in item_spans(mod_path.to_owned(), ast) { | ||
| let old_snippet = &old_src[lo .. hi]; | ||
kkysen marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| let item_path_str = item_path.join("::"); | ||
| if let Some(new_snippet) = new_snippets.get(&item_path_str) { | ||
| if new_snippet != old_snippet { | ||
| rewrites.push((lo, hi, new_snippet)); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if rewrites.len() == 0 { | ||
| continue; | ||
| } | ||
|
|
||
| rewrites.sort_by_key(|&(lo, _, _)| lo); | ||
| let mut new_src = String::with_capacity(old_src.len()); | ||
| let mut pos = 0; | ||
| for &(lo, hi, new_snippet) in &rewrites { | ||
| assert!(lo >= pos, "overlapping rewrites: previous rewrite ended at {}, \ | ||
| but current rewrite covers {} .. {}", pos, lo, hi); | ||
| new_src.push_str(&old_src[pos .. lo]); | ||
| new_src.push_str(new_snippet); | ||
| pos = hi; | ||
| } | ||
| new_src.push_str(&old_src[pos..]); | ||
|
|
||
| let tmp_path = file_path.with_extension(".new"); | ||
| fs::write(&tmp_path, &new_src).unwrap(); | ||
| fs::rename(&tmp_path, file_path).unwrap(); | ||
| eprintln!("applied {} rewrites to {:?}", rewrites.len(), file_path); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.