advsecurenet.evaluation package

---

• advsecurenet.evaluation.evaluators package
  • advsecurenet.evaluation.evaluators.attack_success_rate_evaluator module
    • AttackSuccessRateEvaluator
      • AttackSuccessRateEvaluator.get_results()
      • AttackSuccessRateEvaluator.reset()
      • AttackSuccessRateEvaluator.update()
  • advsecurenet.evaluation.evaluators.perturbation_distance_evaluator module
    • PerturbationDistanceEvaluator
      • PerturbationDistanceEvaluator.calculate_l0_distance()
      • PerturbationDistanceEvaluator.calculate_l2_distance()
      • PerturbationDistanceEvaluator.calculate_l_inf_distance()
      • PerturbationDistanceEvaluator.calculate_perturbation_distances()
      • PerturbationDistanceEvaluator.get_perturbation_distance()
      • PerturbationDistanceEvaluator.get_results()
      • PerturbationDistanceEvaluator.reset()
      • PerturbationDistanceEvaluator.update()
  • advsecurenet.evaluation.evaluators.perturbation_effectiveness_evaluator module
    • PerturbationEffectivenessEvaluator
      • PerturbationEffectivenessEvaluator.calculate_perturbation_effectiveness_score()
      • PerturbationEffectivenessEvaluator.get_results()
      • PerturbationEffectivenessEvaluator.reset()
      • PerturbationEffectivenessEvaluator.update()
  • advsecurenet.evaluation.evaluators.robustness_gap_evaluator module
    • RobustnessGapEvaluator
      • RobustnessGapEvaluator.get_results()
      • RobustnessGapEvaluator.reset()
      • RobustnessGapEvaluator.update()
  • advsecurenet.evaluation.evaluators.similarity_evaluator module
    • SimilarityEvaluator
      • SimilarityEvaluator.calculate_psnr()
      • SimilarityEvaluator.calculate_similarity_scores()
      • SimilarityEvaluator.calculate_ssim()
      • SimilarityEvaluator.get_psnr()
      • SimilarityEvaluator.get_results()
      • SimilarityEvaluator.get_ssim()
      • SimilarityEvaluator.reset()
      • SimilarityEvaluator.update()
      • SimilarityEvaluator.update_psnr()
      • SimilarityEvaluator.update_ssim()
  • advsecurenet.evaluation.evaluators.transferability_evaluator module
    • TransferabilityEvaluator
      • TransferabilityEvaluator.transferability_data
      • TransferabilityEvaluator.get_results()
      • TransferabilityEvaluator.reset()
      • TransferabilityEvaluator.update()

---

advsecurenet.evaluation.adversarial_evaluator module

class advsecurenet.evaluation.adversarial_evaluator.AdversarialEvaluator(evaluators: list[str] | None = None, **kwargs)

Bases: BaseEvaluator

Composite evaluator that can be used to evaluate multiple metrics at once.

Parameters:

• evaluators (Optional[list[str]], optional) – List of evaluators to use. If None, all evaluators will be used. Defaults to None. .

• **kwargs – Arbitrary keyword arguments for the evaluators.

Note

It’s possible to provide a list of target models to evaluate the transferability of the adversarial examples. It’s also possible to provide a distance metric to evaluate the perturbation effectiveness of the adversarial examples. Possible distance metrics are: - L0 - L2 - Linf Default distance metric is L0.

get_results() → dict

Calculates the results for the streaming session.

reset()

Resets the evaluator for a new streaming session.

update(model: BaseModel, original_images: Tensor, true_labels: Tensor, adversarial_images: Tensor, is_targeted: bool = False, target_labels: Tensor | None = None) → None

Updates the evaluator with new data for streaming mode. :param model: The model to evaluate. :type model: BaseModel :param original_images: The original images. :type original_images: torch.Tensor :param true_labels: The true labels of the original images. :type true_labels: torch.Tensor :param adversarial_images: The adversarial images. :type adversarial_images: torch.Tensor :param is_targeted: Whether the attack is targeted or not. Defaults to False. :type is_targeted: bool, optional :param target_labels: The target labels for the targeted attack. Defaults to None. :type target_labels: Optional[torch.Tensor], optional

advsecurenet.evaluation.base_evaluator module

class advsecurenet.evaluation.base_evaluator.BaseEvaluator(*args, **kwargs)

Bases: ABC

Base class for all evaluators.

abstract get_results()

Calculates the results for the streaming session.

abstract reset()

Resets the evaluator for a new streaming session.

save_results_to_csv(evaluation_results: dict, experiment_info: dict | None = None, path: str | None = None, file_name: str | None = None) → None

Saves the evaluation results to a CSV file in a structured format.

Parameters:

• evaluation_results (dict) – The evaluation results.

• experiment_info (dict, optional) – The experiment info.

• path (str, optional) – The path where the file will be saved.

• file_name (str, optional) – The name of the CSV file.

abstract update(model: BaseModel, original_images: Tensor, true_labels: Tensor, adversarial_images: Tensor, is_targeted: bool | None = False, target_labels: Tensor | None = None) → None

Updates the evaluator with new data for streaming mode.

Parameters:

• model (BaseModel) – The model being evaluated.

• original_images (torch.Tensor) – The original images.

• original_labels (torch.Tensor) – The true labels for the original images.

• adversarial_images (torch.Tensor) – The adversarial images.

• is_targeted (bool, optional) – Whether the attack is targeted.

• target_labels (Optional[torch.Tensor], optional) – Target labels for the adversarial images if the attack is targeted.

advsecurenet.evaluation.tester module

class advsecurenet.evaluation.tester.Tester(config: TestConfig)

Bases: object

Base tester module for testing a model.

Parameters:

config (TestConfig) – The configuration for testing.

test() → tuple[float, float]

Tests the model on the given test_loader. Prints the average loss and accuracy.

Returns:

A tuple containing the average loss and accuracy.

Return type:

tuple

---