Add control over the number of processes involved in the particles output. The best values for a config/cluster can be written in a bashrc or a bath job (controlled by env variables):
const size_t MinBytesPerProcess = env_utils::GetValue<size_t>("BFPS_PO_MIN_BYTES", 32 * 1024 * 1024); // Default 32MB
const size_t ChunkBytes = env_utils::GetValue<size_t>("BFPS_PO_CHUNK_BYTES", 8 * 1024 * 1024); // Default 8MB
const int MaxProcessesInvolved = std::min(nb_processes, env_utils::GetValue<int>("BFPS_PO_MAX_PROCESSES", 128));
- The
MinBytesPerProcess
ensure that a process involved write at least this amount of data. - The
ChunkBytes
ensure that the size to write is of the formMinBytesPerProcess + k * ChunkBytes
- MaxProcessesInvolved is a hard limit to the number of processes involved in the output