Configuration file propsoal

[tigercosmos]

This proposal suggests having a configuration file for turning on/off features in Modmesh.
Currently, we may use Python script to set up some configs during runtime, such as setting modmesh.view.app.pycon.python_redirect = False, but it's inconvenient to do it again and again when we are doing some testing.

The steps should be:

1. have a new file config.{format} in json or ymal format.
2. have an in-memory config object class Config
3. when Modmesh starts, it loads the config file to Config and Modmesh initializes everything according to Config
4. when Python scripts modify config later during runtime, just change Config

There should be no overhead for Modmesh with this proposed change. This change may also benefit the later web server feature since we can expect some configurations may be required.

[tigercosmos]

cc @yungyuc, what do you think?

[yungyuc]

Thanks @tigercosmos . Remarks:

1. The configuration data processing engine should be implemented in C++. Configuration data can become very large and it is likely to require a C++ implementation. The initial proof-of-concept may use Python.
2. There needs to be comprehensive Python API wrapped for the config object.
3. The configuration objects needs serialization and deserialization. It would be good to be able to partially serialized (and deserialized).
4. For testing and further prototyping, the Python API should be made to construct the config object from scratch and manage each piece in it.

[tai271828]

@yungyuc I have one question: why the configuration data can become very large? I am wondering what you meant by "large" here?

[yungyuc]

Imagine the data object contains boundary conditions of over 100 logic surfaces in a simulation, and there are 100 such simulations to be managed in a row. And also include material specification. Material database may also use, although a separate design is not impossible.

The simulation and UI core will also need a lot of parameters. It is easy to have tens of thousand of parameters in a production system.

[yungyuc]

Converted this to a discussion.

[tigercosmos]

sure

[yungyuc]

@tigercosmos Before filing a contentless PR like #166 , perhaps it will be a good idea to use this discussion thread to provide some thoughts. It would also be good to use prototype / wip PR to facilitate discussions. But I think we need to provide points in any of the posts and PRs.

[tigercosmos]

@yungyuc about the format, I choose to use JSON just because of my favorite and few experience with RapidJson. Do you suggest me to use ymal (RapidYaml)?

[yungyuc]

Either json or yaml is fine. Considering that you like to connect to web frontend, perhaps json is good.

But I hesitate to include a third-party at the early stage. The configuration system should be scriptable through Python. What prevents you from using https://docs.python.org/3/library/json.html ?

[tigercosmos]

the Config object should be a C++ object? then I "guess" it's a little inefficient to use Python to read the JSON, and to let C++ to read PyObject?
at the same time, according to our architecture, Python is always a binding API, and we may always want to put any implementation into C++?

[yungyuc]

The Config object should be in C++. Most of the reading should happen in C++, especially in the tight loops. Python should be used to populate the Config object.