Skip to content

Auto fragmentation of mass data files between load nodes

Compare
Choose a tag to compare
@marcosborges marcosborges released this 06 Nov 21:22
· 12 commits to master since this release
cfbbc57

Split data between nodes

The implementation for dividing the data mass file between the load executing nodes aims to uncomplicate this existing friction in the execution of distributed load tests;

Is very simple to activate this option.

Just set the split_data_mass_between_nodes variable by activating the feature and informing your mass data files to be distributed.

See the example below...

Example

module "loadtest" {

    source  = "marcosborges/loadtest-distribuited/aws"
    
    name = "nome-da-implantacao"
    executor = "jmeter"
    loadtest_dir_source = "../plan/"
    nodes_size = 2
    
    loadtest_entrypoint = "jmeter -n -t jmeter/basic-with-data.jmx  -R \"{NODES_IPS}\" -l /loadtest/logs -e -o /var/www/html/jmeter -Dnashorn.args=--no-deprecation-warning -Dserver.rmi.ssl.disable=true -LDEBUG "

    split_data_mass_between_nodes = {
        enable = true
        data_mass_filenames = [
            "data/users.csv"
        ]
    }

    subnet_id = data.aws_subnet.current.id
}

Behind the scene:

  1. sends all data mass files to the leader.

split

  1. After submission, the files are divided by the leader into fragments by us for each nodes.

split-result

  1. The last action is to send each fragment to its respective node.

split-result

Splitting the files uses the linux split command and splits the main file into 1 fragment for each node.

More info: Split doc