Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement/optimized-options-and-logic #611

Open
wants to merge 104 commits into
base: enhancement/configs-improvements
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
c924678
Added functionalities for checking version and usage (--v and --help …
PaulDalek Dec 10, 2024
2c21ec2
Added separate HTTP errors and made some corrections.
PaulDalek Dec 10, 2024
cd9fcce
Revision and overhaul of the default config options.
PaulDalek Dec 10, 2024
39e8b5b
Optimization of the browser module.
PaulDalek Dec 10, 2024
0ad7f57
Optimization of the cache module.
PaulDalek Dec 10, 2024
c5b14e1
Modification, corrections and separation exporting logic into smaller…
PaulDalek Dec 10, 2024
9262453
Complete optimization the way options from all sources are set, get a…
PaulDalek Dec 10, 2024
bf9e711
Added missing envs for the validation.
PaulDalek Dec 10, 2024
8b92d05
Optimization of the export module.
PaulDalek Dec 10, 2024
eae2efb
Optimization of the fetch module.
PaulDalek Dec 10, 2024
82cc719
Optimization of the highcharts module.
PaulDalek Dec 10, 2024
87eaf5c
Optimization of the index module.
PaulDalek Dec 10, 2024
b2e34d4
Optimization of the logger module.
PaulDalek Dec 10, 2024
13af747
Removed the poolConfig in favor of the global pool options, created a…
PaulDalek Dec 10, 2024
d96247e
Moved the logic of the prompt functionality into a separated module.
PaulDalek Dec 10, 2024
2e30083
Optimization of the resourceRelease module.
PaulDalek Dec 10, 2024
973306e
Optimization of the sanitize module.
PaulDalek Dec 10, 2024
6cb2e6f
Renaming and optimization of the timer module.
PaulDalek Dec 10, 2024
5d52239
Extended, corrected, moved existing and introduced a few new utility …
PaulDalek Dec 10, 2024
557f65d
Corrected the HTTP error handler and moved it to the middlewares folder.
PaulDalek Dec 10, 2024
c767c04
Separated and moved some of the export handler logic into a new valid…
PaulDalek Dec 10, 2024
4260c1a
Optimization of the health route.
PaulDalek Dec 10, 2024
62e537d
Corrected missing route setting functionality with the optimization o…
PaulDalek Dec 10, 2024
f99e077
Corrected the route to the /version_change/ with the optimization of …
PaulDalek Dec 10, 2024
e35f62e
Optimization of the rateLimiting module.
PaulDalek Dec 10, 2024
5b6ce79
Corrections and updates of the server module logic.
PaulDalek Dec 10, 2024
8982d90
Updated .env.sample and README files.
PaulDalek Dec 10, 2024
b5cedf7
Updated versions of packages.
PaulDalek Dec 10, 2024
8542ba0
Samples, scenarios, tests, and other smaller fixes and additions.
PaulDalek Dec 10, 2024
2f36161
Small corrections.
PaulDalek Dec 10, 2024
015af52
Optimization of the cli module.
PaulDalek Jan 5, 2025
9c8035f
Optimization of the browser module, with the newPage and _setPageEven…
PaulDalek Jan 5, 2025
f859e59
Optimization of the cache module.
PaulDalek Jan 5, 2025
8690d3c
Main refactorization of the way how the export is prepared.
PaulDalek Jan 5, 2025
7b8849a
Optimization and refactorization of the config module, with some new …
PaulDalek Jan 5, 2025
b3fc896
Optimization of the envs module.
PaulDalek Jan 5, 2025
117e840
Optimization of the export module, with better distinction between sv…
PaulDalek Jan 5, 2025
fa7bedd
Optimization of the fetch module.
PaulDalek Jan 5, 2025
f372cfe
Optimization of the highcharts module, with a simpler logic.
PaulDalek Jan 5, 2025
7b304f8
Modified new options merging logic, new API functions, along with JSD…
PaulDalek Jan 5, 2025
56edd6e
Optimization of the logger module.
PaulDalek Jan 5, 2025
40bc66b
Optimization of the pool module, with factory functions corrections.
PaulDalek Jan 5, 2025
3ce9051
The prompt module file saving correction.
PaulDalek Jan 5, 2025
da5233a
Optimization of the resourceRelease module.
PaulDalek Jan 5, 2025
2fd4d71
Optimization of the sanitize module.
PaulDalek Jan 5, 2025
27deb26
Optimization of the timer module.
PaulDalek Jan 5, 2025
1730105
New fixOutfile, getAbsolutePath, getBase64 utils and moved out isCorr…
PaulDalek Jan 5, 2025
b54aa98
Template corrections.
PaulDalek Jan 5, 2025
ddf81a7
Moved mapToNewConfig, printUsage, _cycleCategories functions and opti…
PaulDalek Jan 5, 2025
036dfcb
Removed unnecessary errors in separated files.
PaulDalek Jan 5, 2025
d061c60
Corrections in validator middleware and export route.
PaulDalek Jan 5, 2025
9760007
Optimized server related logic, middlewares and routes.
PaulDalek Jan 5, 2025
1152374
Other, smaller corrections.
PaulDalek Jan 5, 2025
06623ae
Scenarios and samples corrections and modifications.
PaulDalek Jan 5, 2025
855e5be
Unit tests corrections and modifications.
PaulDalek Jan 5, 2025
5c59e26
Documentation corrections, updates and extends.
PaulDalek Jan 5, 2025
1423166
Updated package.json and package-lock.json.
PaulDalek Jan 5, 2025
0e1cd18
Corrected parameters usage in the API functions for the CLI export sc…
PaulDalek Jan 21, 2025
a56f4a0
Modified exported API functions.
PaulDalek Jan 21, 2025
7cef88c
Try catched the checkAndUpdateCache function.
PaulDalek Jan 21, 2025
8270d01
Optimized main functions for exporting process.
PaulDalek Jan 21, 2025
55963e4
Optimized the puppeteerExport function, in the export module.
PaulDalek Jan 21, 2025
05c793a
Optimized the createChart function, in the highcharts module.
PaulDalek Jan 21, 2025
e54188d
Optimized the pool module.
PaulDalek Jan 21, 2025
2acf13f
Enhanced logger functions.
PaulDalek Jan 21, 2025
546ae5f
Corrected saving and loading the config created by the prompt functio…
PaulDalek Jan 21, 2025
4b2230c
Corrected types of the config based options.
PaulDalek Jan 21, 2025
f99b57d
Improved the options processing logic of the config module.
PaulDalek Jan 21, 2025
b98f9e0
Enhanced updates of the options in the API functions of the server mo…
PaulDalek Jan 21, 2025
fec665f
Optimized the rateLimitingMiddleware function, in the rateLimiting mo…
PaulDalek Jan 21, 2025
a91e72f
Optimized the validation middleware.
PaulDalek Jan 21, 2025
e30b014
Got rid of the HttpError and moved the setStatus function to the Expo…
PaulDalek Jan 21, 2025
8df1c58
Other, smaller corrections.
PaulDalek Jan 21, 2025
8c7c3fc
Samples and scenarios runners corrections.
PaulDalek Jan 21, 2025
14219f2
Packages updates.
PaulDalek Jan 21, 2025
c513383
Updated API functions descriptions.
PaulDalek Jan 21, 2025
b663fe7
Built scripts.
PaulDalek Jan 21, 2025
b2a9743
Simplified and optimized options processing logic.
PaulDalek Jan 22, 2025
19846ed
Samples, scenarios and tests corrections.
PaulDalek Jan 22, 2025
6c3c10d
Extended README with an updated options handling and processing infor…
PaulDalek Jan 22, 2025
c7c8a83
Built scripts.
PaulDalek Jan 22, 2025
289e5e6
Small optimizations.
PaulDalek Jan 22, 2025
42ceaf7
Updated paths handling and other small corrections.
PaulDalek Jan 23, 2025
49e7f8a
Corrections in the README file.
PaulDalek Jan 23, 2025
82eb3e7
Minor visual corrections.
PaulDalek Jan 24, 2025
78aa15a
Added a check for the data size before sending it to a page for expor…
PaulDalek Jan 24, 2025
1d8dba3
Optimization by moving and modifing the config-related logic.
PaulDalek Feb 2, 2025
c67c322
The schemas/config.js file contains the default configuration object …
PaulDalek Feb 2, 2025
018a102
Optimized and simplified the cache.js module logic.
PaulDalek Feb 2, 2025
f3f79e2
Optimized and simplified the chart.js module logic.
PaulDalek Feb 2, 2025
906d564
Corrections and moved some function to the chart.js module.
PaulDalek Feb 2, 2025
f09dd55
Moving some logic from the puppeteerExport into a new, _getChartSize …
PaulDalek Feb 2, 2025
9955adf
Fixed the problem with not loading images from URLs.
PaulDalek Feb 2, 2025
2da70b8
Fixed the ui.enabled options usage.
PaulDalek Feb 2, 2025
81f133d
Removed the setStatus function.
PaulDalek Feb 2, 2025
809f6a2
Optimized, reduced and moved info-related functions into a separate m…
PaulDalek Feb 2, 2025
071525a
Smaller optimizations, corrections and descriptions updates.
PaulDalek Feb 2, 2025
9ea61e3
Removed unnecessary tests.
PaulDalek Feb 2, 2025
bac641b
Updated package.json.
PaulDalek Feb 2, 2025
cce01a6
Built scripts.
PaulDalek Feb 2, 2025
8fdaa55
Merge.
PaulDalek Feb 3, 2025
55f83d3
Merge.
PaulDalek Feb 10, 2025
e354d57
Updated the CHANGELOG.md file and bumped the version.
PaulDalek Feb 14, 2025
220fefd
Cosmetic corrections.
PaulDalek Feb 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 30 additions & 7 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -1,29 +1,52 @@
# PUPPETEER CONFIG
PUPPETEER_ARGS =

# HIGHCHARTS CONFIG
HIGHCHARTS_VERSION = latest
HIGHCHARTS_CDN_URL = https://code.highcharts.com/
HIGHCHARTS_CDN_URL = https://code.highcharts.com
HIGHCHARTS_FORCE_FETCH = false
HIGHCHARTS_CACHE_PATH = .cache
HIGHCHARTS_ADMIN_TOKEN =
HIGHCHARTS_CORE_SCRIPTS =
HIGHCHARTS_MODULE_SCRIPTS =
HIGHCHARTS_INDICATOR_SCRIPTS =
HIGHCHARTS_FORCE_FETCH = false
HIGHCHARTS_CACHE_PATH =
HIGHCHARTS_ADMIN_TOKEN =
HIGHCHARTS_CUSTOM_SCRIPTS =

# EXPORT CONFIG
EXPORT_INFILE =
EXPORT_INSTR =
EXPORT_OPTIONS =
EXPORT_SVG =
EXPORT_BATCH =
EXPORT_OUTFILE =
EXPORT_TYPE = png
EXPORT_CONSTR = chart
EXPORT_B64 = false
EXPORT_NO_DOWNLOAD = false
EXPORT_HEIGHT =
EXPORT_WIDTH =
EXPORT_SCALE =
EXPORT_DEFAULT_HEIGHT = 400
EXPORT_DEFAULT_WIDTH = 600
EXPORT_DEFAULT_SCALE = 1
EXPORT_GLOBAL_OPTIONS =
EXPORT_THEME_OPTIONS =
EXPORT_RASTERIZATION_TIMEOUT = 1500

# CUSTOM LOGIC CONFIG
CUSTOM_LOGIC_ALLOW_CODE_EXECUTION = false
CUSTOM_LOGIC_ALLOW_FILE_RESOURCES = false
CUSTOM_LOGIC_CUSTOM_CODE =
CUSTOM_LOGIC_CALLBACK =
CUSTOM_LOGIC_RESOURCES =
CUSTOM_LOGIC_LOAD_CONFIG =
CUSTOM_LOGIC_CREATE_CONFIG =

# SERVER CONFIG
SERVER_ENABLE = false
SERVER_HOST = 0.0.0.0
SERVER_PORT = 7801
SERVER_UPLOAD_LIMIT = 3
SERVER_BENCHMARKING = false

# SERVER PROXY CONFIG
Expand Down Expand Up @@ -61,12 +84,12 @@ POOL_BENCHMARKING = false
# LOGGING CONFIG
LOGGING_LEVEL = 4
LOGGING_FILE = highcharts-export-server.log
LOGGING_DEST = log/
LOGGING_DEST = log
LOGGING_TO_CONSOLE = true
LOGGING_TO_FILE = true

# UI CONFIG
UI_ENABLE = true
UI_ENABLE = false
UI_ROUTE = /

# OTHER CONFIG
Expand All @@ -78,7 +101,7 @@ OTHER_BROWSER_SHELL_MODE = true

# DEBUG CONFIG
DEBUG_ENABLE = false
DEBUG_HEADLESS = true
DEBUG_HEADLESS = false
DEBUG_DEVTOOLS = false
DEBUG_LISTEN_TO_CONSOLE = false
DEBUG_DUMPIO = false
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@

<!-- Please describe how to reproduce the issue. -->
<!-- If you can, please include a curl fetch that demonstrates the issue, e.g. -->
<!-- curl -H "Content-Type: application/json" -X POST -d '{"infile":{ <CHART OPTIONS HERE> }' 127.0.0.1:7801 -o mychart.png -->
<!-- curl -H "Content-Type: application/json" -X POST -d '{"options":{ <CHART OPTIONS HERE> }' 127.0.0.1:7801 -o mychart.png -->
161 changes: 152 additions & 9 deletions CHANGELOG.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Highcharts Export Server

Copyright (c) 2016-2024, Highsoft
Copyright (c) 2016-2025, Highsoft

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
Expand Down
507 changes: 332 additions & 175 deletions README.md

Large diffs are not rendered by default.

132 changes: 85 additions & 47 deletions bin/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

Highcharts Export Server

Copyright (c) 2016-2024, Highsoft
Copyright (c) 2016-2025, Highsoft

Licenced under the MIT licence.

Expand All @@ -13,99 +13,137 @@ See LICENSE file in root for details.

*******************************************************************************/

import main from '../lib/index.js';
/**
* @overview This module serves as the entry point for the Highcharts Export
* Server. It provides functionality for starting the server, performing batch
* or single chart exports, managing manual configuration options through
* the prompts functionality, and updating options with values from the passed
* CLI arguments. It supports both CLI and server-based usage, enabling
* integration for generating Highcharts exports in various environments.
*/

import { batchExport, singleExport } from '../lib/chart.js';
import { setCliOptions } from '../lib/config.js';
import { initExport } from '../lib/index.js';
import { printInfo, printUsage } from '../lib/info.js';
import { log, logWithStack } from '../lib/logger.js';
import { manualConfig } from '../lib/prompt.js';
import { shutdownCleanUp } from '../lib/resourceRelease.js';

import { startServer } from '../lib/server/server.js';

import ExportError from '../lib/errors/ExportError.js';

/**
* The primary function to initiate the server or perform the direct export.
* The primary function to initiate the server or perform the direct CLI export.
* Logs an error if it occurs during the execution and gracefully shuts down
* the process.
*
* @async
* @function start
*
* @throws {ExportError} Throws an ExportError if no valid options are provided.
* @throws {Error} Throws an Error if an unexpected error occurs during
* execution.
* @returns {Promise<void>} A Promise that resolves when the function execution
* ends after displaying the logo with license and version information, showing
* the CLI usage details, or triggering manual configuration using the prompts
* functionality.
*
* @throws {ExportError} Throws an `ExportError` if no valid options
* are provided.
*/
const start = async () => {
async function start() {
try {
// Get the CLI arguments
const args = process.argv;

// Print the usage information if no arguments supplied
// If no arguments are supplied
if (args.length <= 2) {
main.log(
// Print logo with version and license information
printInfo();
log(
2,
'[cli] The number of provided arguments is too small. Please refer to the section below.'
'[cli] The number of provided arguments is too small. Please refer to the help section (use --h or --help command).'
);
return main.printUsage();
return;
}

// Display version and license information if requested
if (['-v', '--v'].some((a) => args.includes(a))) {
// Print logo with version and license information
printInfo();
return;
}

// Display help information if requested
if (['-h', '--h', '-help', '--help'].some((a) => args.includes(a))) {
// Print CLI usage information
printUsage();
return;
}

// Set the options, keeping the priority order of setting values:
// 1. Options from the lib/schemas/config.js file
// 2. Options from a custom JSON file (loaded by the --loadConfig argument)
// 3. Options from the environment variables (the .env file)
// 4. Options from the CLI
const options = main.setOptions(null, args);
// Set the options from CLI, keeping the priority order of setting values
const options = setCliOptions(args);

// If all options correctly parsed
// If all options are correctly parsed
if (options) {
// Print initial logo or text
main.printLogo(options.other.noLogo);
// Print initial logo or text with the version information
printInfo(options.other.noLogo, true);

// In this case we want to prepare config manually
if (options.customLogic.createConfig) {
return main.manualConfig(options.customLogic.createConfig);
manualConfig(
options.customLogic.createConfig,
options.customLogic.allowCodeExecution
);
return;
}

// Start server
if (options.server.enable) {
// Init the export mechanism for the server configuration
await main.initExport(options);
await initExport();

// Run the server
await main.startServer(options.server);
await startServer();
} else {
// Perform batch exports
if (options.export.batch) {
// If not set explicitly, use default option for batch exports
if (!args.includes('--minWorkers', '--maxWorkers')) {
options.pool = {
...options.pool,
minWorkers: 2,
maxWorkers: 25
};
}

// Init a pool for the batch exports
await main.initExport(options);
// Init the export mechanism for batch exports
await initExport();

// Start batch exports
await main.batchExport(options);
await batchExport({
export: options.export,
customLogic: options.customLogic
});
} else {
// No need for multiple workers in case of a single CLI export
options.pool = {
...options.pool,
minWorkers: 1,
maxWorkers: 1
};
options.pool.minWorkers = 1;
options.pool.maxWorkers = 1;

// Init a pool for one export
await main.initExport(options);
// Init the export mechanism for a single export
await initExport();

// Start a single export
await main.singleExport(options);
await singleExport({
export: options.export,
customLogic: options.customLogic
});
}
}
} else {
throw new ExportError(
'[cli] No valid options provided. Please check your input and try again.'
'[cli] No valid options found. Please check your input and try again.',
400
);
}
} catch (error) {
// Log the error with stack
main.logWithStack(1, error);
logWithStack(1, error);

// Gracefully shut down the process
await main.shutdownCleanUp(1);
await shutdownCleanUp(1);
}
};
}

// Start the Export Server process
start();
Loading
Loading