bagua

Bagua is a communication library developed by Kuaishou Technology and DS3 Lab for deep learning.

See tutorials for Bagua’s rationale and benchmark.

Subpackages

• bagua.torch_api
  • bagua.torch_api.algorithms
    • bagua.torch_api.algorithms.async_model_average
    • bagua.torch_api.algorithms.base
    • bagua.torch_api.algorithms.bytegrad
    • bagua.torch_api.algorithms.decentralized
    • bagua.torch_api.algorithms.gradient_allreduce
    • bagua.torch_api.algorithms.q_adam
  • bagua.torch_api.checkpoint
    • bagua.torch_api.checkpoint.checkpointing
  • bagua.torch_api.contrib
    • bagua.torch_api.contrib.fuse
    • bagua.torch_api.contrib.utils
    • bagua.torch_api.contrib.cache_loader
    • bagua.torch_api.contrib.cached_dataset
    • bagua.torch_api.contrib.load_balancing_data_loader
    • bagua.torch_api.contrib.sync_batchnorm
  • bagua.torch_api.data_parallel
    • bagua.torch_api.data_parallel.bagua_distributed
    • bagua.torch_api.data_parallel.distributed
    • bagua.torch_api.data_parallel.functional
  • bagua.torch_api.model_parallel
    • bagua.torch_api.model_parallel.moe
  • bagua.torch_api.bucket
  • bagua.torch_api.communication
  • bagua.torch_api.distributed
  • bagua.torch_api.env
  • bagua.torch_api.tensor