bagua.bagua_define

Module Contents

class bagua.bagua_define.DistributedAlgorithm

Bases: enum.Enum

An enum-like class of available distributed algorithms: allreduce, sg-allreduce, quantize and decentralize.

The values of this class are lowercase strings, e.g., "allreduce". They can be accessed as attributes, e.g., DistributedAlgorithm.GradientAllReduce.

This class can be directly called to parse the string, e.g., DistributedAlgorithm(algor_str).

GradientAllReduce = allreduce

ScatterGatherAllReduce = sg-allreduce

Decentralize = decentralize

QuantizeAllReduce = quantize