Scala language support #5869
Draft
Scala language support #5869
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Replace placeholder implementation with real Dotty compiler integration - Add ScalaCompilerBridge to interface with Scala 3 parser - Implement ScalaTreeVisitor to convert Scala AST to OpenRewrite LST - Handle expression wrapping and offset adjustment for bare literals - Support all literal types: int, hex, long, float, double, boolean, char, string, null - All 13 literal tests now passing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Committer: Jonathan Schneider <[email protected]>
- Added support for parsing binary expressions (arithmetic, comparison, logical, bitwise) - Handles infix notation like '1 + 2' and custom operators like '::' - 19/20 binary expression tests passing - Known issue: method call notation '1.+(2)' has remaining formatting issue All binary operators are properly mapped to J.Binary nodes with correct operator types.
- Added support for prefix unary operators: -, +, !, ~ - Maps Scala unary_- etc. to proper J.Unary nodes - 5/7 unary tests passing - Known issues: postfix operators and explicit unary method calls have formatting issues Co-Authored-By: Claude <[email protected]>
- Created comprehensive ParenthesesTest with 10 test cases - All parentheses tests passing (100% success) - Parentheses are preserved through Unknown nodes for now - Tests cover simple, nested, and complex expressions with parentheses Co-Authored-By: Claude <[email protected]>
- Variable declarations (val/var) now preserved as Unknown nodes - Handles all modifiers (private, protected, final, lazy, implicit) - Multi-line code detection to avoid wrapping declarations - All 12 variable declaration tests now passing (100%) - Overall test success rate: 75/90 (83%) 🤖 Generated with Claude Code(https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Mark completed infrastructure phases (1-5) - Add implementation progress section with current test results - Document completed LST elements: literals, identifiers, assignments, parentheses, variable declarations - Note in-progress work on binary/unary operations - Document key technical decisions and next steps 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Exclude postfix operators from expression wrapping to avoid parse errors - Handle unary operator method references in Select nodes to prevent duplication - Fixed postfixOperator test: "5\!" now parses correctly - Fixed prefixMethodCall test: "x.unary_-" no longer duplicates - All unary operation tests now passing (7/7) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Simplified compilation unit handling to preserve everything as Unknown nodes - Package and import handling deferred for proper implementation later - Fixed compilation unit tests except for comment handling - Test results improved from 92% to 95% passing (86/90 tests) - Remaining issues: comments in whitespace (2 tests), infixWithDot (1 test), parseWithPackage (1 test) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Updated current status with 86/90 tests passing - Added unary operations to completed list - Updated compilation units status to 7/9 tests passing - Listed remaining 4 failing tests with details - Updated key technical decisions with recent fixes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Changed from Space.build to Space.format in extractPrefix method - Space.format properly extracts comments from whitespace into Comment objects - All compilation unit tests now passing (9/9) - Test results improved from 95% to 97.7% passing (88/90 tests) - Only 2 tests remaining: infixWithDot and parseWithPackage 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
The issue was that the cursor wasn't being updated after processing package declarations, causing subsequent statements to include the package text in their prefix when parsed as J.Unknown nodes. Fixed by: - Updating ScalaASTConverter to use createPackageDeclaration method that properly tracks cursor position - Adding updateCursor method to ScalaTreeVisitor for manual cursor updates - Ensuring cursor is updated to end of package declaration before processing subsequent statements All 91 tests now passing. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Map Select nodes to J.FieldAccess for expressions like obj.field
- Map Apply nodes to J.MethodInvocation for method calls
- Support various method call patterns:
- Simple calls: println("Hello")
- Method calls with receivers: obj.method(args)
- Chained calls: str.toUpperCase().substring(1)
- No-arg method calls
- Preserve formatting with proper space extraction
- All 120 tests now passing (up from 106)
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <[email protected]>
- Map If nodes to J.If with proper then/else branch handling - Map WhileDo nodes to J.WhileLoop - Map Block nodes to J.Block for statement grouping - Add comprehensive tests for if, if-else, nested if, while loops - For comprehensions remain as Unknown (need S.ForComprehension) - All 127 tests now passing (up from 120) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add visitClassDef method to handle Scala class declarations - Map Scala TypeDef nodes to J.ClassDeclaration - Support class modifiers (private, protected, abstract, final) - Handle empty classes, classes with bodies, and nested classes - Extract modifiers with proper spacing preservation - Update test suite: 8/18 ClassDeclarationTest tests now passing Next steps: implement constructor parameters, type parameters, extends/implements 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Implement constructor parameter extraction from source - Override visitClassDeclaration in ScalaPrinter to handle Scala's primary constructor syntax - Fix double parentheses issue by not adding extra parentheses around primaryConstructor - Support single/multiple parameters, val/var parameters, and mixed access modifiers - 12/18 ClassDeclarationTest tests now passing (up from 8/18) - Overall test success rate: 96% (145/151 tests passing) Remaining class test failures: - Classes without parameters showing empty () - Type parameters - Extends clause - Case classes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…port - Fix space preservation before 'extends' keyword using JLeftPadded - Implement proper extends clause handling for simple and qualified types - Add support for 'with' clauses (Scala traits) mapped to implements - Handle multiple trait mixins with proper spacing - Add comprehensive test coverage for extends/implements scenarios - Tests cover: simple extends, extends with body, multiple with clauses, qualified extends, constructor parameters, and modifiers The key fix was using JLeftPadded correctly to include the space before 'extends' as the prefix of the extends clause, following the convention of preferring left padding and putting space on the outermost element. 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]>
- Fix empty constructor parameters being incorrectly added to classes without them - Preserve exact whitespace (spaces, tabs, newlines) in extends clauses - Ensure print idempotency by using parsed nodes as-is instead of rebuilding them - Add comprehensive test coverage for spacing edge cases The key fix was to preserve the original prefix spacing from parsed identifiers and field access nodes rather than trying to recreate the spacing. This ensures that double spaces, tabs, and other whitespace are preserved exactly as input. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Parse type parameters from Scala AST (first parameter list with TypeDef nodes) - Map TypeDef nodes to proper J.TypeParameter objects - Override visitContainer in ScalaPrinter to use square brackets [T] instead of angle brackets <T> - Override visitTypeParameter to support Scala-style bounds syntax (: instead of extends, with instead of &) - Update cursor management to properly handle type parameters before constructor parameters - Fix constructor parameter extraction to skip past type parameters when present This enables proper parsing and printing of generic Scala classes like: class Box[T](value: T) class Pair[A, B](first: A, second: B) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add SObject marker to distinguish objects from regular classes - Implement visitModuleDef in ScalaTreeVisitor to convert ModuleDef AST nodes - Update ScalaPrinter to properly print "object" keyword - Fix "case object" parsing by updating isSimpleExpression regex - Add comprehensive test suite for object declarations - 7 out of 8 tests passing (issue with multiple traits spacing remains) Following Kotlin's pattern, objects are represented as J.ClassDeclaration with an SObject marker, allowing reuse of existing infrastructure while distinguishing Scala's singleton objects from regular classes. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…Loop - Implement visitForDo and visitSimpleForEach in ScalaTreeVisitor - Map simple for comprehensions (for (n <- nums) body) to J.ForEachLoop - Handle pattern extraction, variable declaration, and body mapping - Add tests for simple for loops and for loops with block bodies - Currently limited to single-generator patterns without guards - Complex patterns and multiple generators still return Unknown nodes This provides the foundation for for loop support in Scala parsing, though full functionality awaits complete method declaration parsing. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Moved the S interface and S.CompilationUnit from Scala to Java to follow the established pattern used by Kotlin (K.java) and Groovy (G.java). This provides better cross-language compatibility and consistency with other JVM language implementations in OpenRewrite. Key changes: - Created S.java interface following K.java/G.java patterns - Implemented S.CompilationUnit with proper Lombok annotations - Updated ScalaParserVisitor to use the new Java-based S.CompilationUnit - Removed the old S.scala implementation - Documented design decision in Scala.md Benefits: - Consistent with other JVM language implementations - Avoids mixed Java/Scala compilation complexity - Better IDE support with Lombok annotations - Maintains cross-language compatibility with @nullable instead of Options All tests pass with the new implementation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
This commit brings significant improvements to the Scala parser implementation, raising the test passing rate from 97.0% to 97.9% (232/237 tests passing). Key fixes and improvements: - Fixed type variance annotations (+T, -T) by extracting variance symbols from source - Fixed trait printing issue where traits were printed as "classtrait" or "interface" - Fixed abstract class body preservation by improving hasExplicitBody detection - Implemented J.VariableDeclarations for val/var declarations - Added S.TuplePattern for tuple destructuring support - Improved cursor management throughout the parser Technical details: - Modified ScalaPrinter to handle traits as Interface kind correctly - Enhanced visitTypeParameter to preserve variance annotations in type parameters - Fixed class body detection to check for either body statements or braces - Added proper handling for lazy val declarations with ScalaLazyVal marker - Improved parentheses handling to avoid cursor position conflicts Known remaining issues (5 failing tests): - Object with multiple traits has extra spaces - Method call on field access has duplication - Type cast in if conditions needs special handling - Tuple assignment destructuring has AST span issues 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Fixed extra spaces between traits in object declarations - Updated visitContainer call to not include space before 'with' keyword - The implements container already includes the necessary spacing - Test passing rate improved to 98.3% (233/237 tests) Known remaining issues: - Method call on field access (System.out.println) has cursor management issue - Type cast in if conditions needs special handling - Tuple destructuring assignment has AST span issues - ExtendsImplementsTest has 2 failing tests to investigate 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…ex types - Fixed cursor management issue where cursor was incorrectly advanced twice when parsing variables with type annotations - Fixed space before equals sign being lost for variables with type annotations - Fixed complex types like List[Int] causing initializer to be lost - All 12 variable declaration tests now passing (100%) - Overall test success rate: 85% (273/323 tests passing) The key fix was to only advance the cursor past the variable name when no type annotation is present. When a type annotation exists, the cursor is already positioned correctly after parsing the type. Also updated ScalaPrinter to properly handle the space after the colon in type annotations by preserving it in the type expression's prefix rather than hardcoding it. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…s classes - Fixed argument handling in anonymous class constructors to properly handle non-Expression types - Added proper spacing between "new" keyword and class name in ScalaPrinter - Improved detection of constructor calls vs annotations in visitApply - All NewClassTest cases now passing (9/9 tests) The main issue was that constructor arguments in anonymous classes weren't always returning Expression types from visitTree, causing ClassCastException. Changed to use explicit type checking with fallback to J.Unknown for better error handling. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's changed?
Working out a Scala parser and recipe support. This could be a relatively long lived branch, while I use spare time to work out one language construct at a time. But the fundamentals of instantiating the compiler and establishing the mapping are in place and now it's about covering more syntax.