hadding TTrees >100 GB

[MatthiesC]

By default, ROOT's hadd command will interrupt once the size of the output TTree generated from all the input files is larger than 100 GB. Unfornutately, sframe_batch.py -a/-f does not warn the user if this actually happens.

So far, the user can manually hadd affected output files with the following recipe that is adapted from [1] and sets the TTree limit from 100 GB to 1 TB. Create the following file startup.C:

#include "TTree.h"

int startup() {
  TTree::SetMaxTreeSize( 1000000000000LL ); // 1 TB
  return 0;
}

namespace {
  static int i = startup();
}

Compile it with:

root -b -l -q startup.C+

Use hadd like so:

LD_PRELOAD=startup_C.so hadd output.root input_*.root

This could be implemented into the hadd routines (-a/-f) of sframe_batch.py somehow.

NB: While hadd does not generate TTrees > 100 GB by default, there apparently is no further action needed when reading TTrees > 100 GB (e.g. when reading such a large TTree from a preselection directory while running a subsequent selection step).

[1] https://root-forum.cern.ch/t/root-6-04-14-hadd-100gb-and-rootlogon/24581/2