Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement/options-type-validations #612

Open
wants to merge 28 commits into
base: enhancement/optimized-options-and-logic
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fa50fda
Expanded exisitng envs validation logic into extensive validation mec…
PaulDalek Dec 10, 2024
8aa49a7
Added a new function to API called logZodIssues to log zod errors app…
PaulDalek Dec 10, 2024
935486e
Validate options.
PaulDalek Dec 10, 2024
4192bae
Added exhaustive testing of options validations.
PaulDalek Dec 10, 2024
046ce23
Corrected an import.
PaulDalek Dec 10, 2024
2468e10
Merge
PaulDalek Jan 5, 2025
14a1376
Built scripts.
PaulDalek Jan 5, 2025
5ad605c
Merged branch.
PaulDalek Jan 21, 2025
9116fd9
Added option to enable or disable validation of options.
PaulDalek Jan 21, 2025
ac21594
Updates of validators in the validation module.
PaulDalek Jan 21, 2025
a38fabd
Added two functions for validating data and some corrections.
PaulDalek Jan 21, 2025
579f9a6
Secured incoming data with the validation logic.
PaulDalek Jan 21, 2025
45cac84
The logZodIssues function small modifications.
PaulDalek Jan 21, 2025
ac9cfb8
Added tests for the validation and requestId parameters.
PaulDalek Jan 21, 2025
b564f2c
Updated README with the new info about validation option and new vali…
PaulDalek Jan 21, 2025
67e8093
Built scripts.
PaulDalek Jan 21, 2025
99de905
Imports corrections.
PaulDalek Jan 21, 2025
476d1cd
Merge.
PaulDalek Jan 22, 2025
1585c5f
Merge.
PaulDalek Jan 23, 2025
4366db6
Merge.
PaulDalek Jan 23, 2025
ee8a7ea
Improved options validation by moving it into the updateOptions funct…
PaulDalek Jan 23, 2025
003a7c9
Merge.
PaulDalek Feb 2, 2025
c22eeba
Small validation correction after merge.
PaulDalek Feb 3, 2025
0d6413c
Merge.
PaulDalek Feb 3, 2025
4200623
Merge.
PaulDalek Feb 14, 2025
d258e74
Small logs corrections.
PaulDalek Feb 14, 2025
5f348f3
Updated the CHANGELOG.md file.
PaulDalek Feb 14, 2025
051687c
Merge.
PaulDalek Feb 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ OTHER_LISTEN_TO_PROCESS_EXITS = true
OTHER_NO_LOGO = false
OTHER_HARD_RESET_PAGE = false
OTHER_BROWSER_SHELL_MODE = true
OTHER_VALIDATION = true

# DEBUG CONFIG
DEBUG_ENABLE = false
Expand Down
13 changes: 10 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ _Breaking Changes:_

_New Features:_

- Added a toggleable type validation for all options coming from various sources (environment variables, custom JSON, CLI arguments), providing a rich set of validators with both strict and loose validation. Replaced the `envs.js` module with an expanded validation logic module, `validation.js`.
- Added the `validateOption` function for validating a single option. It is used in the code to validate individual options (`svg`, `instr`, `resources`, `customCode`, `callback`, `globalOptions`, and `themeOptions`) loaded from a file.
- Added the `validateOptions` function for validating the full set of options. It is used in the code to validate options coming from functions that update global options, CLI arguments, configurations loaded via `--loadConfig`, and configurations created using the prompts functionality.
- Introduced redefined `getOptions` and `updateOptions` functions to retrieve and update the original global options or a copy of global options, allowing flexibility in export scenarios.
- Added a new option called `uploadLimit` to control the maximum size of a request's payload body.
- Added the possibility to return a Base64 version of the chart using any export method (not only through requests).
Expand All @@ -27,6 +30,7 @@ _Enhancements:_
- Adjusted the options loading sequence: `default config -> environment variables` at initialization, `custom JSON -> CLI arguments` when using `setCliOptions` (CLI exports only).
- The `getOptions` function can now return either a direct reference to the `globalOptions` or a copy (by setting the `getCopy` flag).
- The `updateOptions` function can now update and return either a direct reference to the `globalOptions` or a copy (by setting the `getCopy` flag).
- The `updateOptions` function now validates provided options (using `validateOptions` internally) before merging them into the global options.
- The `_mergeOptions` (renamed from the `mergeConfigOptions`) modifies the first object directly now and is used internally.
- Replaced the fixed `absoluteProps` array (previously in `./lib/schemas/config.js`) with dynamic generation via `_createAbsoluteProps` function.
- Enhanced the `isAllowedConfig` (renamed from the `isCorrectJSON`) and `_optionsStringify` functions to better handle stringified options in JSON.
Expand All @@ -49,16 +53,16 @@ _Enhancements:_
- Created `_handleSize` for handling the `height`, `width`, and `scale` options.
- Created `_checkDataSize` for handling the data size validation.
- Optimized `initExport`, utilizing `updateOptions` for global option updates.
- The `initExport` now have its options parameters set as optional, using global option values if not provided.
- The `initExport` now have its options parameters defaulted to an empty object, using global option values if none are provided.
- Updated exported API functions for module usage.
- Adjusted imports to get functions from corresponding modules rather than `index.js`.
- Server functions are now directly exported (rather than within a `server` object) as API functions.
- The `logger` API functions that modify options now update global options.
- Added following API functions: `getOptions`, `updateOptions`, `mapToNewOptions`, `enableConsoleLogging`.
- Exposed `getOptions`, `updateOptions`, `mapToNewOptions`, `enableConsoleLogging`, `validateOption`, `validateOptions`, and `logZodIssues` as API functions.
- Small corrections of the `_attachProcessExitListeners` (renamed from the `attachProcessExitListeners`).
- Refactored logic for initial configuration, startup, and HTTP/HTTPS server management.
- Optimized `startServer`, utilizing `updateOptions` for global option updates.
- The `startServer` now have its options parameters set as optional, using global option values if not provided.
- The `startServer` now have its options parameters defaulted to an empty object, using global option values if none are provided.
- Optimized logic and statistics display in `health.js` router.
- Optimized logic and corrected the url (from `version/change` to `version_change`) in the `versionChange.js` router.
- Refactored `exportHandler` (to `requestExport`) by moving some logic to the `validaion` middleware and refactoring the rest.
Expand Down Expand Up @@ -89,6 +93,7 @@ _Enhancements:_
- Renamed the `get` function to `getBrowser`, the `create` function to `createBrowser`, and the `close` function to `closeBrowser`.
- Renamed `triggerExport` to `createChart`, optimizing the options passed and processed in the `highcharts.js` module.
- Improved the module's overall logic, optimizing logging functions and the initialization process.
- Added the `logZodIssues` function for displaying correctly formatted validation errors.
- Added `pathToLog` to the module's logging options to remember the full path to the log file.
- Added the `enableConsoleLogging` function.
- Removed `listen` function and `listeners` array from the module's logging options.
Expand Down Expand Up @@ -118,12 +123,14 @@ _Enhancements:_
- Enhanced all files with improved JSDoc tags, descriptions, and comments, adding extra tags such as `@overview`, `@async`, and `@function`.
- Corrected function descriptions, parameter types, return values, and documented errors.
- Fixed all tests, samples, and scenario runners.
- Added unit tests for validating each option from every source (CLI, config, environment variables).
- Created, renamed, or removed various tests, samples, and scenario runners.
- Removed separate test runner scripts.
- Made a minor correction in the `build` script.
- Updated package versions.
- Corrected the description of options prioritization order in the `Configuration` section.
- Added explanations of overall option handling, management, and processing, along with descriptions for each export method (`Options Handling` section and subsections).
- Added a description of options validation in the `Options Validation` section.
- Added, updated, corrected, or redefined descriptions and values of options in the following sections: `Default JSON Config`, `Environment Variables`, `Custom JSON Config`, `Command Line Arguments`, `HTTP Server POST Arguments`.
- Fixed an incorrect version change endpoint description in the `Switching Highcharts Version at Runtime` section.
- Corrected example and added description of the `Node.js Module` section.
Expand Down
37 changes: 33 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ The `singleExport()`, `batchExport()`, and `startExport()` functions must be pro

Essentially, all options can be configured through `.env`, the CLI, and prompts, with one exception: the `HIGHCHARTS_ADMIN_TOKEN`, which is only available as an environment variable.

## Options Validation

By default, options validation is enabled, and it is recommended to keep it enabled to ensure that the provided options are correctly checked, validated, and parsed, allowing the exporting process to function without issues. However, it is possible to disable validation (by setting the `validation` option to **false**) if you are confident in the accuracy of the data you provide. Additionaly, when used as a Node.js module, each API function that updates global options with the provided data also offers the ability to validate the data.

## Default JSON Config

The JSON below represents the default configuration stored in the `./lib/schemas/config.js` file. If no `.env` file is found (more details on the file and environment variables below), these options will be used. The configuration is not recommended to be modified directly, as it can typically be managed through other sources.
Expand Down Expand Up @@ -296,7 +300,8 @@ _Available default JSON config:_
"listenToProcessExits": true,
"noLogo": false,
"hardResetPage": false,
"browserShellMode": true
"browserShellMode": true,
"validation": true
},
"debug": {
"enable": false,
Expand Down Expand Up @@ -428,6 +433,7 @@ _Available environment variables:_
- `OTHER_NO_LOGO`: Skip printing the logo on a startup. Will be replaced by a simple text (defaults to `false`).
- `OTHER_HARD_RESET_PAGE`: Determines whether the page's content should be reset from scratch, including Highcharts scripts (defaults to `false`).
- `OTHER_BROWSER_SHELL_MODE`: Decides whether to enable older but much more performant _shell_ mode for the browser (defaults to `true`).
- `OTHER_VALIDATION`: Decides whether or not to enable validation of options types (defaults to `true`).

### Debugging Config

Expand Down Expand Up @@ -563,6 +569,7 @@ _Available CLI arguments:_
- `--noLogo`: Skip printing the logo on a startup. Will be replaced by a simple text (defaults to `false`).
- `--hardResetPage`: Determines whether the page's content should be reset from scratch, including Highcharts scripts (defaults to `false`).
- `--browserShellMode`: Decides whether to enable older but much more performant _shell_ mode for the browser (defaults to `true`).
- `--validation`: Decides whether or not to enable validation of options types (defaults to `true`).

### Debugging Config

Expand Down Expand Up @@ -716,9 +723,9 @@ This package supports both CommonJS and ES modules.

**highcharts-export-server module**

- `async function startServer(serverOptions)`: Starts an HTTP and/or HTTPS server based on the provided configuration. The `serverOptions` object contains server-related properties (refer to the `server` section in the `./lib/schemas/config.js` file for details).
- `async function startServer(serverOptions = {})`: Starts an HTTP and/or HTTPS server based on the provided configuration. The `serverOptions` object contains server-related properties (refer to the `server` section in the `./lib/schemas/config.js` file for details).

- `@param {Object} serverOptions` - The configuration object containing `server` options. This object may include a partial or complete set of the `server` options. If the options are partial, missing values will default to the current global configuration.
- `@param {Object} [serverOptions={}]` - The configuration object containing `server` options. This object may include a partial or complete set of the `server` options. If the options are partial, missing values will default to the current global configuration. The default value is an empty object.

- `@returns {Promise<void>}` A Promise that resolves when the server is either not enabled or no valid Express app is found, signaling the end of the function's execution.

Expand Down Expand Up @@ -763,10 +770,11 @@ This package supports both CommonJS and ES modules.

- `@returns {Object}` A copy of the global options object, or a reference to the global options object.

- `function updateOptions(newOptions, getCopy = false)`: Updates and returns the global options object or a copy of the global options object, based on the `getCopy` flag.
- `function updateOptions(newOptions, getCopy = false, strictCheck = true)`: Updates and returns the global options object or a copy of the global options object, based on the `getCopy` flag. The `newOptions` object can be strictly validated depending on the `strictCheck` flag.

- `@param {Object} newOptions` - An object containing the new options to be merged into the global options.
- `@param {boolean} [getCopy=false]` - Determines whether to merge the new options into a copy of the global options object (`true`) or directly into the global options object (`false`). The default value is `false`.
- `@param {boolean} [strictCheck=true]` - Determines if stricter validation should be applied. The default value is `true`.

- `@returns {Object}` The updated options object, either the modified global options or a modified copy, based on the value of `getCopy`.

Expand All @@ -776,6 +784,21 @@ This package supports both CommonJS and ES modules.

- `@returns {Object}` A new object containing options structured according to the mapping defined in the `nestedProps` object or an empty object if the provided `oldOptions` is not a correct object.

- `function validateOption(name, configOption, strictCheck = true)`: Validates a specified option using the corresponding validator from the configuration object. Returns the original option if the validation is disabled globally.

- `@param {string} name` - The name of the option to validate.
- `@param {any} configOption` - The value of the option to validate.
- `@param {boolean} [strictCheck=true]` - Determines if stricter validation should be applied. The default value is `true`.

- `@returns {any}` The parsed and validated value of the option.

- `function validateOptions(configOptions, strictCheck = true)`: Validates the provided configuration options for the exporting process. Returns the original option if the validation is disabled globally.

- `@param {Object} configOptions` - The configuration options to be validated.
- `@param {boolean} [strictCheck=true]` - Determines if stricter validation should be applied. The default value is `true`.

- `@returns {Object}` The parsed and validated configuration options object.

- `async function initExport(initOptions = {})`: Initializes the export process. Tasks such as configuring logging, checking the cache and sources, and initializing the resource pool occur during this stage.

This function must be called before attempting to export charts or set up a server.
Expand Down Expand Up @@ -831,6 +854,12 @@ This package supports both CommonJS and ES modules.

- `@returns {void}` Exits the function execution if attempting to log at a level higher than allowed.

- `function logZodIssues(newLevel, issues, customMessage)`: Logs an error message related to Zod validation issues. Optionally, a custom message can be provided.

- `@param {number} newLevel` - The log level.
- `@param {Error[]} issues` - An array of Zod validation issues.
- `@param {string} customMessage` - An optional custom message to be included in the log alongside the error.

- `function setLogLevel(level)`: Sets the log level to the specified value. Log levels are (`0` = no logging, `1` = error, `2` = warning, `3` = notice, `4` = verbose, or `5` = benchmark).

- `@param {number} level` - The log level to be set.
Expand Down
30 changes: 27 additions & 3 deletions lib/chart.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ See LICENSE file in root for details.

import { readFileSync, writeFileSync } from 'fs';

import { isAllowedConfig, updateOptions } from './config.js';
import { isAllowedConfig, updateOptions, validateOption } from './config.js';
import { log, logWithStack } from './logger.js';
import { getPoolStats, killPool, postWork } from './pool.js';
import { sanitize } from './sanitize.js';
Expand Down Expand Up @@ -285,10 +285,10 @@ export async function startExport(imageOptions, endCallback) {
// Check the file's extension
if (exportOptions.infile.endsWith('.svg')) {
// Set to the `svg` option
exportOptions.svg = fileContent;
exportOptions.svg = validateOption('svg', fileContent);
} else if (exportOptions.infile.endsWith('.json')) {
// Set to the `instr` option
exportOptions.instr = fileContent;
exportOptions.instr = validateOption('instr', fileContent);
} else {
throw new ExportError(
'[chart] Incorrect value of the `infile` option.',
Expand Down Expand Up @@ -716,6 +716,12 @@ function _handleCustomLogic(customLogicOptions) {
customLogicOptions.allowFileResources,
true
);

// Validate option
customLogicOptions.resources = validateOption(
'resources',
customLogicOptions.resources
);
} catch (error) {
log(2, '[chart] The `resources` cannot be loaded.');

Expand All @@ -730,6 +736,12 @@ function _handleCustomLogic(customLogicOptions) {
customLogicOptions.customCode,
customLogicOptions.allowFileResources
);

// Validate the option
customLogicOptions.customCode = validateOption(
'customCode',
customLogicOptions.customCode
);
} catch (error) {
logWithStack(2, error, '[chart] The `customCode` cannot be loaded.');

Expand All @@ -745,6 +757,12 @@ function _handleCustomLogic(customLogicOptions) {
customLogicOptions.allowFileResources,
true
);

// Validate the option
customLogicOptions.callback = validateOption(
'callback',
customLogicOptions.callback
);
} catch (error) {
logWithStack(2, error, '[chart] The `callback` cannot be loaded.');

Expand Down Expand Up @@ -956,6 +974,12 @@ function _handleGlobalAndTheme(exportOptions, customLogicOptions) {
allowCodeExecution
);
}

// Validate the option
exportOptions[optionsName] = validateOption(
optionsName,
exportOptions[optionsName]
);
}
} catch (error) {
logWithStack(
Expand Down
Loading
Loading