FederatedScope

FederatedScope uses specialized configuration files to manage different aspects of a Federated Learning task. The available configuration modules are:

• config.py: Environment and execution settings.
• cfg_data.py: Data and dataset configurations.
• cfg_model.py: Model architecture configurations.
• cfg_fl_algo.py: Federated learning algorithm settings.
• cfg_training.py: Training process configurations.
• cfg_fl_setting.py: Federated learning setup/topology settings.
• cfg_evaluation.py: Evaluation metrics and procedures.
• cfg_asyn.py: Asynchronous training strategies.
• cfg_differential_privacy.py: Differential privacy settings.
• cfg_hpo.py: Hyperparameter optimization (Auto-tuning) components.
• cfg_attack.py: Security attack configurations.

The federatedscope/core/trainers module follows a hierarchical structure. Users can extend or use different types of trainers depending on their needs, ranging from general-purpose torch trainers to user-defined implementations.

Key components in the hierarchy include:

• BaseTrainer: The foundation for all trainers.
• Trainer: A specialized implementation inheriting from BaseTrainer.
• GeneralTorchTrainer and GeneralTFTrainer: Standard implementations for PyTorch and TensorFlow-based local learning.
• UserDefineTrainer: A base for implementing custom local learning logic.

federatedscope/core
├── trainers
│   ├── BaseTrainer
│   │   ├── Trainer
│   │   │   ├── GeneralTorchTrainer
│   │   │   ├── GeneralTFTrainer
│   │   │   ├── Context
│   │   │   ├── ...
│   │   ├── UserDefineTrainer
│   │   ├── ...

FederatedScope provides a specialized module for Federated Matrix Factorization tasks. The module is organized into four primary components:

• federatedscope.mf.dataset: Handles dataset definitions and management for MF tasks.
• federatedscope.mf.model: Contains the model architectures used for matrix factorization.
• federatedscope.mf.dataloader: Manages data loading processes specifically optimized for MF.
• federatedscope.mf.trainer: Provides the training logic and orchestration for federated MF experiments.

FederatedScope provides several baseline configurations for Vertical Federated Learning (VFL) using tree-based models like GBDT, Random Forest (RF), and XGBoost. These baselines demonstrate different model types, protection methods, and datasets.

Key configurations include:

• Model Types: 'feature_gathering' and 'label_scattering'.
• Algorithms: 'gbdt', 'rf', and 'xgb'.
• Protection Methods: None, 'he' (Homomorphic Encryption), 'dp' (Differential Privacy), and 'op_boost'.
• Evaluation Protection: None or 'he'.

Refer to the specific YAML configuration files for detailed parameter settings.

The federatedscope.nlp package provides specialized modules for Federated Natural Language Processing (NLP) tasks. It is organized into four primary functional areas:

• federatedscope.nlp.dataset: Handles NLP-specific dataset loading and processing.
• federatedscope.nlp.dataloader: Manages data loading pipelines tailored for NLP sequences and tokens.
• federatedscope.nlp.model: Contains model architectures designed for federated NLP (e.g., Transformers, RNNs).
• federatedscope.nlp.trainer: Provides training logic and orchestration specifically for NLP tasks in a federated setting.

The FL-Attacker documentation provides a curated list of research papers regarding attacks in Federated Learning (FL). You can find surveys covering general threats and specific attack vectors like privacy and backdoor attacks.

Key survey papers include:

• A Survey on Gradient Inversion: Attacks, Defenses and Future Directions (2022)
• Threats to Federated Learning: A Survey (2020)

FederatedScope provides a specialized set of modules for Computer Vision tasks in a federated learning setting. The CV module is organized into four main functional areas:

• federatedscope.cv.dataset: Contains dataset implementations, including leaf and leaf_cv variants.
• federatedscope.cv.dataloader: Handles data loading logic specifically optimized for CV tasks.
• federatedscope.cv.model: Provides model architectures suitable for federated computer vision.
• federatedscope.cv.trainer: Contains the training logic and orchestration for CV-specific federated learning tasks.

FederatedScope supports several advanced Federated Learning (FL) research areas:

• Personalized Federated Learning: Handles non-IID data and heterogeneous system resources using client-specific architectures and configurations.
• Federated Hyperparameter Optimization (HPO): Supports techniques like low-fidelity HPO to reduce communication costs.
• Privacy Attacker: Provides algorithms to verify the privacy protection strength of FL systems.
• Graph Federated Learning: Enables learning global models from isolated sub-graph data.
• Recommendation: Privacy-preserving recommender systems.
• Differential Privacy: Economical and flexible privacy protection techniques.

Detailed materials and paper lists for these topics are available in the materials/paper_list/ directory of the repository.

For feature-gathering models, you can protect the feature_order using two methods:

1. Differential Privacy (DP)

Adds noise to the feature order.

• protect_method: 'dp'
• protect_args: [{'bucket_num': b, 'epsilon': e}].
• Smaller bucket_num and epsilon increase privacy but may reduce model utility.

2. OpBoost (Global or Adjusting)

• protect_method: 'op_boost'
• global: Maps data to integers $[lb, ub]$ via affine transformation and re-maps with probability $p$ based on $\epsilon$.
• adjust: Maps data to $[lb, ub]$, partitions into $pb$ buckets, and uses a two-step random selection process involving $\epsilon_{prt}$ and $\epsilon_{ner}$.

vertical:
  protect_object: 'feature_order'
  protect_method: 'dp'
  protect_args: [{'bucket_num': 50, 'epsilon': 3}]