Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto fragmentation of mass data files between load nodes #12

Merged
merged 11 commits into from
Nov 6, 2021

Conversation

marcosborges
Copy link
Owner

Split data between nodes

The implementation for dividing the data mass file between the load executing nodes aims to uncomplicate this existing friction in the execution of distributed load tests;

Is very simple to activate this option.

Just set the split_data_mass_between_nodes variable by activating the feature and informing your mass data files to be distributed.

See the example below...

Example

module "loadtest" {

    source  = "marcosborges/loadtest-distribuited/aws"
    
    name = "nome-da-implantacao"
    executor = "jmeter"
    loadtest_dir_source = "../plan/"
    nodes_size = 2
    
    loadtest_entrypoint = "jmeter -n -t jmeter/basic-with-data.jmx  -R \"{NODES_IPS}\" -l /loadtest/logs -e -o /var/www/html/jmeter -Dnashorn.args=--no-deprecation-warning -Dserver.rmi.ssl.disable=true -LDEBUG "

    split_data_mass_between_nodes = {
        enable = true
        data_mass_filenames = [
            "data/users.csv"
        ]
    }

    subnet_id = data.aws_subnet.current.id
}

Behind the scene:

  1. sends all data mass files to the leader.

split

  1. After submission, the files are divided by the leader into fragments by us for each nodes.

split-result

  1. The last action is to send each fragment to its respective node.

split-result

Splitting the files uses the linux split command and splits the main file into 1 fragment for each node.

More info: Split doc


@marcosborges marcosborges merged commit cfbbc57 into master Nov 6, 2021
@marcosborges marcosborges deleted the feat/spliter branch November 7, 2021 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant