bagua.torch_api.algorithms.bytegrad

Module Contents

class bagua.torch_api.algorithms.bytegrad.ByteGradAlgorithm(average=True)

Bases: bagua.torch_api.algorithms.Algorithm

Create an instance of the ByteGrad algorithm.

Parameters

average (bool) – If True, the gradients on each worker are averaged. Otherwise, they are summed.