Skip to content

Commit 0cf4943

Browse files
committed
YARN-11886: Introduce Capacity Scheduler UI.
1 parent c745495 commit 0cf4943

File tree

309 files changed

+83005
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

309 files changed

+83005
-0
lines changed
Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# Apache Hadoop YARN Capacity Scheduler UI
2+
3+
A modern React-based web interface for managing and configuring the YARN Capacity Scheduler.
4+
5+
## Overview
6+
7+
The YARN Capacity Scheduler UI provides an intuitive graphical interface for:
8+
- **Queue Management**: Visual queue hierarchy with drag-and-drop organization
9+
- **Capacity Configuration**: Interactive capacity allocation and resource management
10+
- **Node Labels**: Partition management and node-to-label assignments
11+
- **Placement Rules**: Configure application placement policies
12+
- **Global Settings**: Manage scheduler-wide configuration parameters
13+
- **Configuration Validation**: Real-time validation with helpful error messages
14+
- **Change Staging**: Review and apply configuration changes in batches
15+
16+
## Technology Stack
17+
18+
- **Framework**: React 19 with TypeScript (strict mode)
19+
- **Build Tool**: Vite 6.4 with React Router v7 (SPA mode)
20+
- **UI Components**: shadcn/ui, Radix UI, Tailwind CSS
21+
- **State Management**: Zustand with Immer
22+
- **Visualization**: XYFlow (React Flow) for queue hierarchy
23+
- **Testing**: Vitest + React Testing Library
24+
25+
## Building
26+
27+
### Prerequisites
28+
29+
- **Maven**: 3.6.0 or later
30+
- **Node.js**: 22.16.0+ (automatically installed by Maven during build)
31+
- **Java**: JDK 8 or later
32+
33+
### Maven Build
34+
35+
Build the UI as part of the YARN build using the `yarn-ui` profile:
36+
37+
```bash
38+
# From hadoop-yarn directory
39+
cd hadoop-yarn-project/hadoop-yarn
40+
mvn clean package -Pyarn-ui
41+
42+
# From hadoop-yarn-capacity-scheduler-ui directory
43+
cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
44+
mvn clean package -Pyarn-ui
45+
```
46+
47+
This will:
48+
1. Install Node.js 22.16.0 and npm locally in `target/webapp/node/`
49+
2. Install npm dependencies
50+
3. Build the React application
51+
4. Package everything into a WAR file at `target/hadoop-yarn-capacity-scheduler-ui-*.war`
52+
53+
### Build Output
54+
55+
The build creates:
56+
- `target/webapp/build/client/` - Built React application (static files)
57+
- `target/hadoop-yarn-capacity-scheduler-ui-*.war` - Deployable WAR file
58+
59+
## Development
60+
61+
### Local Development Setup
62+
63+
```bash
64+
cd src/main/webapp
65+
66+
# Install dependencies
67+
npm install
68+
69+
# Start development server
70+
npm run dev
71+
```
72+
73+
The development server runs at `http://localhost:5173` with hot module replacement.
74+
75+
### Environment Variables
76+
77+
Create a `.env` file in `src/main/webapp/` based on `.env.example`:
78+
79+
```bash
80+
# Mock mode: "static" (use mock data), "cluster" (proxy to real cluster), "off" (no mocking)
81+
VITE_API_MOCK_MODE=static
82+
83+
# YARN ResourceManager URL (required when VITE_API_MOCK_MODE=cluster)
84+
VITE_CLUSTER_PROXY_TARGET=http://localhost:8088
85+
86+
# Development flags
87+
VITE_READONLY_MODE=false
88+
VITE_YARN_USER_NAME=admin
89+
```
90+
91+
### Available Scripts
92+
93+
```bash
94+
npm run dev # Start development server
95+
npm run build # Production build
96+
npm run start # Preview production build
97+
npm run test # Run tests
98+
npm run test:ui # Run tests with UI
99+
npm run test:coverage # Generate test coverage report
100+
npm run lint # Lint code
101+
npm run lint:fix # Fix linting issues
102+
npm run format # Format code with Prettier
103+
```
104+
105+
### Running Tests
106+
107+
```bash
108+
# Run all tests
109+
npm test
110+
111+
# Run tests in watch mode
112+
npm run test
113+
114+
# Run tests with coverage
115+
npm run test:coverage
116+
117+
# Run tests with UI
118+
npm run test:ui
119+
```
120+
121+
## Deployment
122+
123+
### WAR Deployment
124+
125+
The built WAR file can be deployed to a servlet container (Tomcat, Jetty, etc.) or integrated into the YARN ResourceManager web application.
126+
127+
### API Endpoints
128+
129+
The UI requires access to the following YARN ResourceManager REST API endpoints:
130+
131+
- `GET /ws/v1/cluster/scheduler` - Get scheduler configuration
132+
- `PUT /ws/v1/cluster/scheduler-conf` - Update scheduler configuration
133+
- `GET /ws/v1/cluster/scheduler-conf` - Get mutable configuration
134+
- `GET /ws/v1/cluster/nodes` - Get cluster nodes
135+
- `GET /conf` - Get Hadoop configuration
136+
137+
### Integration with ResourceManager
138+
139+
When deployed alongside the ResourceManager, the UI can access these endpoints directly. The `web.xml` includes SPA routing configuration to handle client-side routing.
140+
141+
## Architecture
142+
143+
### Directory Structure
144+
145+
```
146+
src/main/webapp/
147+
├── src/
148+
│ ├── app/ # React Router application setup
149+
│ ├── components/ # Reusable UI components
150+
│ ├── config/ # Scheduler configuration schemas
151+
│ ├── contexts/ # React contexts
152+
│ ├── features/ # Feature modules
153+
│ │ ├── global-settings/
154+
│ │ ├── node-labels/
155+
│ │ ├── placement-rules/
156+
│ │ ├── property-editor/
157+
│ │ ├── queue-management/
158+
│ │ ├── staged-changes/
159+
│ │ ├── template-config/
160+
│ │ └── validation/
161+
│ ├── hooks/ # Custom React hooks
162+
│ ├── lib/ # API client, utilities
163+
│ ├── stores/ # Zustand stores
164+
│ ├── types/ # TypeScript type definitions
165+
│ └── utils/ # Helper functions
166+
├── public/ # Static assets
167+
└── WEB-INF/ # Web application descriptor
168+
```
169+
170+
### State Management
171+
172+
The application uses Zustand for global state management with the following stores:
173+
174+
- **SchedulerStore**: Manages scheduler configuration and queue hierarchy
175+
- **NodeLabelStore**: Handles node labels and partitions
176+
- **StagedChangesStore**: Tracks pending configuration changes
177+
- **ValidationStore**: Manages validation state and error messages
178+
179+
## Configuration
180+
181+
### Build Profiles
182+
183+
- **yarn-ui**: Production build (default profile in this module)
184+
- **yarn-ui-dev**: Development build (includes mock data in WAR)
185+
186+
### Maven Properties
187+
188+
- `packagingType`: WAR when profile is active, POM otherwise
189+
- `webappDir`: Build directory (`${basedir}/target/webapp`)
190+
- `nodeExecutable`: Path to locally installed Node.js
191+
- `keepUIBuildCache`: Set to `true` to preserve node_modules between builds
192+
193+
## Browser Support
194+
195+
- Chrome/Edge: Latest 2 versions
196+
- Firefox: Latest 2 versions
197+
- Safari: Latest 2 versions
198+
199+
## Contributing
200+
201+
This module follows Apache Hadoop's contribution guidelines. All source files must include the Apache License header.
202+
203+
### Code Style
204+
205+
- TypeScript strict mode enabled
206+
- ESLint for code quality
207+
- Prettier for code formatting
208+
- Conventional commit messages
209+
210+
## License
211+
212+
Licensed under the Apache License, Version 2.0. See the LICENSE file in the Hadoop root directory.
213+
214+
## Related Documentation
215+
216+
- [YARN Capacity Scheduler Documentation](../hadoop-yarn-site/src/site/markdown/CapacityScheduler.md)
217+
- [YARN REST API Documentation](../hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md)
218+
- [Hadoop Main Documentation](https://hadoop.apache.org/docs/current/)

0 commit comments

Comments
 (0)