gnm.evaluation

`evaluation`

Network evaluation criteria for comparing generated networks to real networks.

This subpackage provides classes for quantifying the similarity or dissimilarity between synthetic (generated) networks and real (target) networks. These evaluation criteria can be used to:

Assess the quality of generated networks.
Guide optimization processes to find optimal model parameters.
Compare different generative models parameters.

The module includes:

Base classes for defining novel evaluation criteria.
Criteria based on the Kolmogorov-Smirnov test comparing network property distributions.
Correlation-based criteria comparing spatial patterns of network properties.
Composite criteria combining multiple evaluation metrics.

Both binary (unweighted) and weighted network evaluations are supported.

Evaluation

`gnm.evaluation.EvaluationCriterion(device=None)`

Bases: ABC

Abstract base class for network evaluation criteria.

This class provides a framework for defining various criteria to evaluate the similarity between a synthetic (generated) network and a real (target) network. Each criterion computes a dissimilarity measure between the two networks based on specific network properties.

Subclasses must implement:

_pre_call to perform validation checks on input matrices.
_evaluate to compute the actual dissimilarity measure.

See Also

evaluation.BinaryEvaluationCriterion: Base class for binary network evaluations.
evaluation.WeightedEvaluationCriterion: Base class for weighted network evaluations.
fitting.perform_evalutions: Function to evaluate networks using an evaluation criterion.
fitting.optimise_evaluation: Function to optimise model parameters using an evaluation.

Parameters:

Name	Type	Description	Default
`device`	`device`	PyTorch device to use for computations. If None, uses CUDA if available, otherwise CPU.	`None`

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, device: torch.device = None):
    r"""
    Args:
        device:
            PyTorch device to use for computations. If None, uses CUDA if available,
            otherwise CPU.
    """
    self.device = DEVICE if device is None else device

`str()`

Return a string representation of the criterion.

Source code in src/gnm/evaluation/evaluation_base.py

def __str__(self) -> str:
    r"""Return a string representation of the criterion."""
    return self.__class__.__name__

`call(synthetic_matrices, real_matrices)`

Compute the dissimilarity between synthetic and real networks.

This method validates the input matrices and then calls the _evaluate method to compute the actual dissimilarity measure.

Parameters:

Name	Type	Description	Default
`synthetic_matrices`	`Float[Tensor, 'num_synthetic_networks num_nodes num_nodes']`	Batch of adjacency/weight matrices of the synthetic networks with shape [num_synthetic_networks, num_nodes, num_nodes]	required
`real_matrices`	`Float[Tensor, 'num_real_networks num_nodes num_nodes']`	Adjacency/weight matrices of the real networks with shape [num_real_networks, num_nodes, num_nodes]	required

Returns:

Type	Description
`Float[Tensor, 'num_synthetic_networks num_real_networks']`	Tensor of dissimilarity values with shape [num_synthetic_networks, num_real_networks],
`Float[Tensor, 'num_synthetic_networks num_real_networks']`	where higher values indicate greater dissimilarity

Source code in src/gnm/evaluation/evaluation_base.py

@jaxtyped(typechecker=typechecked)
def __call__(
    self,
    synthetic_matrices: Float[
        torch.Tensor, "num_synthetic_networks num_nodes num_nodes"
    ],
    real_matrices: Float[torch.Tensor, "num_real_networks num_nodes num_nodes"],
) -> Float[torch.Tensor, "num_synthetic_networks num_real_networks"]:
    r"""Compute the dissimilarity between synthetic and real networks.

    This method validates the input matrices and then calls the _evaluate method
    to compute the actual dissimilarity measure.

    Args:
        synthetic_matrices:
            Batch of adjacency/weight matrices of the synthetic networks with shape
            [num_synthetic_networks, num_nodes, num_nodes]
        real_matrices:
            Adjacency/weight matrices of the real networks with shape
            [num_real_networks, num_nodes, num_nodes]

    Returns:
        Tensor of dissimilarity values with shape [num_synthetic_networks, num_real_networks],
        where higher values indicate greater dissimilarity
    """
    self._pre_call(synthetic_matrices)
    self._pre_call(real_matrices)
    return self._evaluate(synthetic_matrices, real_matrices).to(DEVICE)

`_pre_call(matrices)` `abstractmethod`

Perform validation checks on input matrices.

This abstract method should be implemented by subclasses to ensure that input matrices meet the required criteria (e.g., binary values, symmetry).

Parameters:

Name	Type	Description	Default
`matrices`	`Float[Tensor, 'num_network num_nodes num_nodes']`	Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]	required

Source code in src/gnm/evaluation/evaluation_base.py

@abstractmethod
@jaxtyped(typechecker=typechecked)
def _pre_call(
    self, matrices: Float[torch.Tensor, "num_network num_nodes num_nodes"]
):
    r"""Perform validation checks on input matrices.

    This abstract method should be implemented by subclasses to ensure that input
    matrices meet the required criteria (*e.g.*, binary values, symmetry).

    Args:
        matrices:
            Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]
    """
    pass

`_evaluate(synthetic_matrices, real_matrices)` `abstractmethod`

Compute the actual dissimilarity measure between networks.

This abstract method should be implemented by subclasses to define how to calculate the dissimilarity between synthetic and real networks.

Parameters:

Name	Type	Description	Default
`synthetic_matrices`	`Float[Tensor, 'num_synthetic_networks num_nodes num_nodes']`	Batch of adjacency/weight matrices of the synthetic networks with shape [num_synthetic_networks, num_nodes, num_nodes]	required
`real_matrices`	`Float[Tensor, 'num_real_networks num_nodes num_nodes']`	Adjacency/weight matrices of the real networks with shape [num_real_networks, num_nodes, num_nodes]	required

Returns:

Type	Description
`Float[Tensor, 'num_synthetic_networks num_real_networks']`	Tensor of dissimilarity values with shape [num_synthetic_networks, num_real_networks]

Source code in src/gnm/evaluation/evaluation_base.py

@abstractmethod
@jaxtyped(typechecker=typechecked)
def _evaluate(
    self,
    synthetic_matrices: Float[
        torch.Tensor, "num_synthetic_networks num_nodes num_nodes"
    ],
    real_matrices: Float[torch.Tensor, "num_real_networks num_nodes num_nodes"],
) -> Float[torch.Tensor, "num_synthetic_networks num_real_networks"]:
    r"""Compute the actual dissimilarity measure between networks.

    This abstract method should be implemented by subclasses to define how
    to calculate the dissimilarity between synthetic and real networks.

    Args:
        synthetic_matrices:
            Batch of adjacency/weight matrices of the synthetic networks with shape
            [num_synthetic_networks, num_nodes, num_nodes]
        real_matrices:
            Adjacency/weight matrices of the real networks with shape
            [num_real_networks, num_nodes, num_nodes]

    Returns:
        Tensor of dissimilarity values with shape [num_synthetic_networks, num_real_networks]
    """
    pass

Criterion types

`gnm.evaluation.BinaryEvaluationCriterion()`

Bases: EvaluationCriterion, ABC

Base class for evaluation criteria specialized for binary networks.

This class extends EvaluationCriterion to specifically handle binary (unweighted) networks. It implements validation checks to ensure input matrices contain only binary values (0 or 1), are symmetric, and contain no self-connections.

Subclasses must implement the _evaluate method to compute the actual dissimilarity measure.

See Also

evaluation.EvaluationCriterion: Parent abstract base class.
evaluation.WeightedEvaluationCriterion: Base class for weighted network evaluations.

The initialisation method sets the accepts attribute to 'binary' to indicate that this criterion works with binary networks.

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self):
    r"""The initialisation method sets the accepts attribute to 'binary' to indicate that this criterion
    works with binary networks.
    """
    self.accepts = "binary"

`_pre_call(matrices)`

Perform validation checks on binary matrices.

Validates that the input matrices contain only binary values (0 or 1), are symmetric, and have no self-connections.

Parameters:

Name	Type	Description	Default
`matrices`	`Float[Tensor, 'num_networks num_nodes num_nodes']`	Binary adjacency matrices with shape [num_networks, num_nodes, num_nodes]	required

Source code in src/gnm/evaluation/evaluation_base.py

@jaxtyped(typechecker=typechecked)
def _pre_call(
    self, matrices: Float[torch.Tensor, "num_networks num_nodes num_nodes"]
):
    r"""Perform validation checks on binary matrices.

    Validates that the input matrices contain only binary values (0 or 1),
    are symmetric, and have no self-connections.

    Args:
        matrices:
            Binary adjacency matrices with shape [num_networks, num_nodes, num_nodes]
    """
    binary_checks(matrices)

`gnm.evaluation.WeightedEvaluationCriterion()`

Bases: EvaluationCriterion, ABC

Base class for evaluation criteria specialized for weighted networks.

This class extends EvaluationCriterion to specifically handle weighted networks. It implements validation checks to ensure input matrices contain only non-negative values which are symmetric and contain no self-connections.

Subclasses must implement the _evaluate to compute the actual dissimilarity measure.

See Also

evaluation.EvaluationCriterion: Parent abstract base class.
evaluation.BinaryEvaluationCriterion: Base class for binary network evaluations.

The initialisation method sets the accepts attribute to 'weighted' to indicate that this criterion works with weighted networks.

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self):
    r"""The initialisation method sets the accepts attribute to 'weighted' to indicate that this criterion
    works with weighted networks.
    """
    self.accepts = "weighted"

`_pre_call(matrices)`

Perform validation checks on weighted matrices.

Validates that the input matrices contain only non-negative values, are symmetric, and have no self-connections.

Parameters:

Name	Type	Description	Default
`matrices`	`Float[Tensor, 'num_networks num_nodes num_nodes']`	Weighted adjacency matrices with shape [num_networks, num_nodes, num_nodes]	required

Source code in src/gnm/evaluation/evaluation_base.py

@jaxtyped(typechecker=typechecked)
def _pre_call(
    self, matrices: Float[torch.Tensor, "num_networks num_nodes num_nodes"]
):
    r"""Perform validation checks on weighted matrices.

    Validates that the input matrices contain only non-negative values,
    are symmetric, and have no self-connections.

    Args:
        matrices:
            Weighted adjacency matrices with shape [num_networks, num_nodes, num_nodes]
    """
    weighted_checks(matrices)

`gnm.evaluation.KSCriterion(device=None)`

Bases: EvaluationCriterion, ABC

Base class for Kolmogorov-Smirnov (KS) distance based network evaluation.

This class implements network comparison using the KS test statistic between distributions of network properties (e.g., degree distribution, clustering coefficients). The KS statistic measures the maximum difference between two cumulative distribution functions, providing a measure of how different two distributions are.

Subclasses must implement the _get_graph_statistics method to define the specific network property to use in the KS test.

See Also

evaluation.DegreeKS: KS test on degree distributions
evaluation.ClusteringKS: KS test on clustering coefficient distributions
evaluation.BetweennessKS: KS test on betweenness centrality distributions
utils.ks_statistic: Function to compute KS statistics, used internally to this class.

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, device: torch.device = None):
    r"""
    Args:
        device:
            PyTorch device to use for computations. If None, uses CUDA if available,
            otherwise CPU.
    """
    self.device = DEVICE if device is None else device

`_get_graph_statistics(matrices)` `abstractmethod`

Compute network properties for KS comparison.

This abstract method should be implemented by subclasses to define which network property to extract for comparison (e.g., degrees, clustering coefficients).

Parameters:

Name	Type	Description	Default
`matrices`	`Float[Tensor, 'num_networks num_nodes num_nodes']`	Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]	required

Returns:

Type	Description
`Float[Tensor, 'num_networks _']`	Network property values with shape [num_networks, num_values]

Source code in src/gnm/evaluation/evaluation_base.py

@abstractmethod
@jaxtyped(typechecker=typechecked)
def _get_graph_statistics(
    self, matrices: Float[torch.Tensor, "num_networks num_nodes num_nodes"]
) -> Float[torch.Tensor, "num_networks _"]:
    r"""Compute network properties for KS comparison.

    This abstract method should be implemented by subclasses to define which
    network property to extract for comparison (*e.g.*, degrees, clustering coefficients).

    Args:
        matrices:
            Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]

    Returns:
        Network property values with shape [num_networks, num_values]
    """
    pass

`gnm.evaluation.CorrelationCriterion(smoothing_matrix)`

Bases: EvaluationCriterion, ABC

Base class for correlation-based network evaluation criteria.

This class implements network comparison using correlation coefficients between spatial patterns of network properties (e.g., node degree, clustering coefficients). Higher correlation indicates greater similarity in the spatial organisation of network properties.

Subclasses must implement the _get_graph_statistics method to define the specific network property to use in the correlation calculation.

Parameters:

Name	Type	Description	Default
`smoothing_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix used to spatially smooth the network property values, which can help account for registration errors or spatial uncertainty in brain networks.	required

See Also

evaluation.DegreeCorrelation: Correlation of node degree patterns
evaluation.ClusteringCorrelation: Correlation of clustering coefficient patterns
evaluation.BetweennessCorrelation: Correlation of betweenness centrality patterns

Parameters:

Name	Type	Description	Default
`smoothing_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix used to spatially smooth the network property values with shape [num_nodes, num_nodes]. This can help account for registration errors or spatial uncertainty in brain networks.	required

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, smoothing_matrix: Float[torch.Tensor, "num_nodes num_nodes"]):
    r"""
    Args:
        smoothing_matrix:
            Matrix used to spatially smooth the network property values with shape
            [num_nodes, num_nodes]. This can help account for registration errors
            or spatial uncertainty in brain networks.
    """
    self.smoothing_matrix = smoothing_matrix

`_get_graph_statistics(matrices)` `abstractmethod`

Compute network properties for correlation comparison.

This abstract method should be implemented by subclasses to define which network property to extract for comparison (e.g., degrees, clustering coefficients).

Parameters:

Name	Type	Description	Default
`matrices`	`Float[Tensor, 'num_networks num_nodes num_nodes']`	Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]	required

Returns:

Type	Description
`Float[Tensor, 'num_networks num_nodes']`	Network property values with shape [num_networks, num_nodes]

Source code in src/gnm/evaluation/evaluation_base.py

@abstractmethod
@jaxtyped(typechecker=typechecked)
def _get_graph_statistics(
    self, matrices: Float[torch.Tensor, "num_networks num_nodes num_nodes"]
) -> Float[torch.Tensor, "num_networks num_nodes"]:
    r"""Compute network properties for correlation comparison.

    This abstract method should be implemented by subclasses to define which
    network property to extract for comparison (*e.g.*, degrees, clustering coefficients).

    Args:
        matrices:
            Adjacency/weight matrices with shape [num_networks, num_nodes, num_nodes]

    Returns:
        Network property values with shape [num_networks, num_nodes]
    """
    pass

Composite Criteria

`gnm.evaluation.CompositeCriterion(criteria)`

Bases: EvaluationCriterion, ABC

Base class for composite evaluation criteria combining multiple metrics.

This class allows combining multiple evaluation criteria into a single composite criterion. Subclasses define how to combine the individual criteria results (e.g., maximum, mean, weighted sum).

Parameters:

Name	Type	Description	Default
`criteria`	`list[EvaluationCriterion]`	List of EvaluationCriterion objects to combine	required

Notes

All criteria in the list must accept the same type of network (binary or weighted).

See Also

evaluation.MaxCriteria: Takes the maximum value across all criteria
evaluation.MeanCriteria: Takes the mean value across all criteria
evaluation.WeightedSumCriteria: Takes a weighted sum of all criteria

Parameters:

Name	Type	Description	Default
`criteria`	`list[EvaluationCriterion]`	List of EvaluationCriterion objects to combine. All criteria must accept the same type of network (binary or weighted).	required

Raises:

Type	Description
`AssertionError`	If no criteria are provided or if they accept different network type

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, criteria: list[EvaluationCriterion]):
    r"""
    Args:
        criteria:
            List of EvaluationCriterion objects to combine. All criteria must
            accept the same type of network (binary or weighted).

    Raises:
        AssertionError: If no criteria are provided or if they accept different network type
    """
    assert len(criteria) > 0, "Must provide at least one criterion"
    self.criteria = criteria
    self.accepts = self.criteria[0].accepts
    assert all(
        criterion.accepts == self.accepts for criterion in self.criteria
    ), "All criteria must accept the same type of network"

`gnm.evaluation.MaxCriteria(criteria)`

Bases: CompositeCriterion

Combine multiple evaluation criteria by taking their maximum value.

This class enables the evaluation of networks using multiple criteria simultaneously, where the overall dissimilarity is determined by the worst-performing (maximum) criterion. This approach ensures that the synthetic network must match the real network well across all specified properties.

Examples:

>>> import torch
>>> from gnm.evaluation import DegreeKS, ClusteringKS, MaxCriteria
>>> from gnm.defaults import get_binary_network
>>> from gnm.utils import get_control
>>> # Create individual criteria
>>> degree_ks = DegreeKS()
>>> clustering_ks = ClusteringKS()
>>> # Combine them using MaxCriteria
>>> max_criterion = MaxCriteria([degree_ks, clustering_ks])
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate degree distribution dissimilarity
>>> dissimilarity = max_criterion(random_network, real_network)

See Also

evaluation.CompositeCriterion: Base class for combining criteria, from which this class inherits.
evaluation.MeanCriteria: For averaging multiple criteria instead of taking the maximum.

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, criteria: list[EvaluationCriterion]):
    r"""
    Args:
        criteria:
            List of EvaluationCriterion objects to combine. All criteria must
            accept the same type of network (binary or weighted).

    Raises:
        AssertionError: If no criteria are provided or if they accept different network type
    """
    assert len(criteria) > 0, "Must provide at least one criterion"
    self.criteria = criteria
    self.accepts = self.criteria[0].accepts
    assert all(
        criterion.accepts == self.accepts for criterion in self.criteria
    ), "All criteria must accept the same type of network"

`gnm.evaluation.MeanCriteria(criteria)`

Bases: CompositeCriterion

Combine multiple evaluation criteria by taking their mean value.

This class enables the evaluation of networks using multiple criteria simultaneously, where the overall dissimilarity is determined by the average value of all criteria. This approach balances different aspects of network similarity without letting a single criterion dominate.

Examples:

>>> import torch
>>> from gnm.evaluation import DegreeKS, ClusteringKS, MeanCriteria
>>> from gnm.defaults import get_binary_network
>>> from gnm.utils import get_control
>>> # Create individual criteria
>>> degree_ks = DegreeKS()
>>> clustering_ks = ClusteringKS()
>>> # Combine them using MeanCriteria
>>> max_criterion = MeanCriteria([degree_ks, clustering_ks])
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate degree distribution dissimilarity
>>> dissimilarity = max_criterion(random_network, real_network)

See Also

evaluation.CompositeCriterion: Base class for combining criteria, from which this class inherits.
evaluation.MaxCriteria: For maximum-based combination.
evaluation.WeightedSumCriteria: For weighted combinations of criteria.

Source code in src/gnm/evaluation/evaluation_base.py

def __init__(self, criteria: list[EvaluationCriterion]):
    r"""
    Args:
        criteria:
            List of EvaluationCriterion objects to combine. All criteria must
            accept the same type of network (binary or weighted).

    Raises:
        AssertionError: If no criteria are provided or if they accept different network type
    """
    assert len(criteria) > 0, "Must provide at least one criterion"
    self.criteria = criteria
    self.accepts = self.criteria[0].accepts
    assert all(
        criterion.accepts == self.accepts for criterion in self.criteria
    ), "All criteria must accept the same type of network"

`gnm.evaluation.WeightedSumCriteria(criteria, weights)`

Bases: CompositeCriterion

Combine multiple evaluation criteria by taking their weighted sum.

This class enables the evaluation of networks using multiple criteria with different importance weights assigned to each criterion. By adjusting the weights, users can emphasize certain network properties over others in the evaluation.

Examples:

>>> import torch
>>> from gnm.evaluation import DegreeKS, ClusteringKS, WeightedSumCriteria
>>> from gnm.defaults import get_binary_network
>>> from gnm.utils import get_control
>>> # Create individual criteria
>>> degree_ks = DegreeKS()
>>> clustering_ks = ClusteringKS()
>>> # Combine them using WeightedSumCriteria (more weight on degree)
>>> weighted_criterion = WeightedSumCriteria(
...     [degree_ks, clustering_ks],
...     weights=[0.7, 0.3]
... )
>>> # Calculate dissimilarity between networks
>>> real_network = get_binary_network()  # Example real network
>>> random_network = get_control(real_network)  # Example random network
>>> dissimilarity = weighted_criterion(random_network, real_network)

See Also

evaluation.CompositeCriterion: Base class for combining criteria, from which this class inherits.
evaluation.MaxCriteria: For maximum-based combination.
evaluation.MeanCriteria: For averaging multiple criteria.

Parameters:

Name	Type	Description	Default
`criteria`	`list[EvaluationCriterion]`	List of evaluation criteria to combine.	required
`weights`	`list[float]`	List of weights for each criterion. Should have the same length as criteria.	required

Source code in src/gnm/evaluation/composite_criteria.py

def __init__(self, criteria: list[EvaluationCriterion], weights: list[float]):
    r"""
    Args:
        criteria:
            List of evaluation criteria to combine.
        weights:
            List of weights for each criterion. Should have the same length as criteria.
    """
    self.weights = weights
    super().__init__(criteria)

Binary KS Criteria

`gnm.evaluation.DegreeKS()`

Bases: KSCriterion, BinaryEvaluationCriterion

Compare degree distributions between binary networks using KS statistic.

This criterion measures the dissimilarity between the degree distributions of synthetic and real networks using the Kolmogorov-Smirnov test. The degree of a node is the number of connections it has to other nodes.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network
>>> from gnm.evaluation import DegreeKS
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate degree distribution dissimilarity
>>> criterion = DegreeKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.
evaluation.KSCriterion: Base class for KS criteria, from which this class inherits.

Source code in src/gnm/evaluation/binary_ks_criteria.py

def __init__(self):
    KSCriterion.__init__(self)
    BinaryEvaluationCriterion.__init__(self)

`gnm.evaluation.ClusteringKS()`

Bases: KSCriterion, BinaryEvaluationCriterion

Compare clustering coefficient distributions between binary networks using KS statistic.

This criterion measures the dissimilarity between the clustering coefficient distributions of synthetic and real networks using the Kolmogorov-Smirnov test. The clustering coefficient measures the degree to which nodes in a graph tend to cluster together.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network
>>> from gnm.evaluation import ClusteringKS
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate clustering coefficient distribution dissimilarity
>>> criterion = ClusteringKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.
evaluation.KSCriterion: Base class for KS criteria, from which this class inherits.
utils.binary_clustering_coefficients: Function to compute clustering coefficients for binary networks, which this class uses.

Source code in src/gnm/evaluation/binary_ks_criteria.py

def __init__(self):
    KSCriterion.__init__(self)
    BinaryEvaluationCriterion.__init__(self)

`gnm.evaluation.BetweennessKS()`

Bases: KSCriterion, BinaryEvaluationCriterion

Compare betweenness centrality distributions between binary networks using KS statistic.

This criterion measures the dissimilarity between the betweenness centrality distributions of synthetic and real networks using the Kolmogorov-Smirnov test. Betweenness centrality quantifies the number of times a node acts as a bridge along the shortest path between two other nodes.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network
>>> from gnm.evaluation import BetweennessKS
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate betweenness centrality distribution dissimilarity
>>> criterion = BetweennessKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.
evaluation.KSCriterion: Base class for KS criteria, from which this class inherits.
utils.binary_betweenness_centrality: Function used to calculate betweenness centrality.

Source code in src/gnm/evaluation/binary_ks_criteria.py

def __init__(self):
    KSCriterion.__init__(self)
    BinaryEvaluationCriterion.__init__(self)

`gnm.evaluation.EdgeLengthKS(distance_matrix)`

Bases: BinaryEvaluationCriterion

Compare edge length distributions between binary networks using KS statistic.

This criterion measures the dissimilarity between the edge length distributions of synthetic and real networks using the Kolmogorov-Smirnov test. Edge length is determined by a distance matrix that defines the spatial distance between nodes.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network, get_distance_matrix
>>> from gnm.evaluation import EdgeLengthKS
>>> from gnm.utils import get_control
>>> # Load a default binary network and distance matrix
>>> real_network = get_binary_network()
>>> distance_matrix = get_distance_matrix()
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate edge length distribution dissimilarity
>>> criterion = EdgeLengthKS(distance_matrix)
>>> dissimilarity = criterion(random_network, real_network)

See Also

evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`distance_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix defining the spatial distance between each pair of nodes with shape [num_nodes, num_nodes]	required

Source code in src/gnm/evaluation/binary_ks_criteria.py

def __init__(self, distance_matrix: Float[torch.Tensor, "num_nodes num_nodes"]):
    r"""
    Args:
        distance_matrix:
            Matrix defining the spatial distance between each pair of nodes
            with shape [num_nodes, num_nodes]
    """
    BinaryEvaluationCriterion.__init__(self)
    self.distance_matrix = distance_matrix

Weighted KS Criteria

`gnm.evaluation.WeightedNodeStrengthKS(normalise=True)`

Bases: KSCriterion, WeightedEvaluationCriterion

Compare node strength distributions between weighted networks using KS statistic.

This criterion measures the dissimilarity between the node strength distributions of synthetic and real weighted networks using the Kolmogorov-Smirnov test. Node strength is the weighted equivalent of node degree - it is the sum of the weights of all edges connected to a node.

Examples:

>>> import torch
>>> from gnm.defaults import get_weighted_network
>>> from gnm.evaluation import WeightedNodeStrengthKS
>>> from gnm.utils import get_control
>>> # Load a default weighted network
>>> real_network = get_weighted_network()
>>> # Create a random network with the same weight distribution
>>> random_network = get_control(real_network)
>>> # Calculate node strength distribution dissimilarity
>>> criterion = WeightedNodeStrengthKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

utils.node_strengths: The function used to calculate node strengths.
evaluation_base.KSCriterion: The base class for KS criteria, from which this class inherits.
evaluation_base.WeightedEvaluationCriterion: The base class for weighted criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`normalise`	`Optional[bool]`	If True, normalise the weights of each network by its maximum weight before computing node strengths. This can be useful when comparing networks with different weight scales. Defaults to True.	`True`

Source code in src/gnm/evaluation/weighted_ks_criteria.py

@jaxtyped(typechecker=typechecked)
def __init__(self, normalise: Optional[bool] = True):
    r"""
    Args:
        normalise:
            If True, normalise the weights of each network by its maximum weight
            before computing node strengths. This can be useful when comparing
            networks with different weight scales. Defaults to True.
    """
    KSCriterion.__init__(self)
    WeightedEvaluationCriterion.__init__(self)
    self.normalise = normalise

`gnm.evaluation.WeightedBetweennessKS(normalise=True)`

Bases: KSCriterion, WeightedEvaluationCriterion

Compare weighted betweenness centrality distributions using KS statistic.

This criterion measures the dissimilarity between the weighted betweenness centrality distributions of synthetic and real networks using the Kolmogorov-Smirnov test.

We compute betweenness centrality using Brandes algorithm. Betweenness centrality for a node \(u\) in a weighted network is:

\[ c_B(u) = \sum_{v,w} \\frac{\sigma(v,w|u)}{\sigma(v,w)}, \]

where \(\sigma(v,w)\) is the number of shortest paths from \(v\) to \(w\), and \(\sigma(v,w|u)\) is the number of those that pass through \(u\). For weighted networks, path lengths are computed using the edge weights as distance.

Examples:

>>> import torch
>>> from gnm.defaults import get_weighted_network
>>> from gnm.evaluation import WeightedBetweennessKS
>>> from gnm.utils import get_control
>>> # Load a default weighted network
>>> real_network = get_weighted_network()
>>> # Create a random network with the same weight distribution
>>> random_network = get_control(real_network)
>>> # Calculate weighted betweenness centrality distribution dissimilarity
>>> criterion = WeightedBetweennessKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

evaluation_base.KSCriterion: The base class for KS criteria, from which this class inherits.
evaluation_base.WeightedEvaluationCriterion: The base class for weighted criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`normalise`	`Optional[bool]`	If True, normalise the weights of the network by the maximum weight in the network. Defaults to True.	`True`

Source code in src/gnm/evaluation/weighted_ks_criteria.py

@jaxtyped(typechecker=typechecked)
def __init__(self, normalise: Optional[bool] = True):
    """
    Args:
        normalise: If True, normalise the weights of the network by the maximum weight in the network. Defaults to True.
    """
    KSCriterion.__init__(self)
    WeightedEvaluationCriterion.__init__(self)
    self.normalise = normalise

`gnm.evaluation.WeightedClusteringKS()`

Bases: KSCriterion, WeightedEvaluationCriterion

Compare weighted clustering coefficient distributions using KS statistic.

This criterion measures the dissimilarity between the weighted clustering coefficient distributions of synthetic and real networks using the Kolmogorov-Smirnov test.

Implements the Onnela et al. (2005) definition of weighted clustering, which uses the geometric mean of triangle weights. For each node \(u\), the clustering coefficient is:

\[ c(u) = \\frac{1}{k_u(k_u-1)} \sum_{v,w} (\hat{w}_{uv} \\times \hat{w}_{uw} \\times \hat{w}_{vw})^{1/3}, \]

where \(k_u\) is the node strength of node \(u\), and \(\hat{w}_{uv}\) is the weight of the edge between nodes \(u\) and \(v\), after normalising by dividing by the maximum weight in the network.

Examples:

>>> import torch
>>> from gnm.defaults import get_weighted_network
>>> from gnm.evaluation import WeightedClusteringKS
>>> from gnm.utils import get_control
>>> # Load a default weighted network
>>> real_network = get_weighted_network()
>>> # Create a random network with the same weight distribution
>>> random_network = get_control(real_network)
>>> # Calculate weighted clustering coefficient distribution dissimilarity
>>> criterion = WeightedClusteringKS()
>>> dissimilarity = criterion(random_network, real_network)

See Also

utils.weighted_clustering_coefficients: The function used to calculate weighted clustering
evaluation_base.KSCriterion: The base class for KS criteria, from which this class inherits.
evaluation_base.WeightedEvaluationCriterion: The base class for weighted criteria, from which this class inherits.

Source code in src/gnm/evaluation/weighted_ks_criteria.py

def __init__(self):
    KSCriterion.__init__(self)
    WeightedEvaluationCriterion.__init__(self)

Binary Correlation Criteria

`gnm.evaluation.DegreeCorrelation(smoothing_matrix)`

Bases: CorrelationCriterion, BinaryEvaluationCriterion

Compare spatial patterns of node degrees between binary networks using correlation.

This criterion measures the similarity between the spatial patterns of node degrees in synthetic and real networks using Pearson correlation. Unlike KS statistics which compare distributions, correlation criteria assess whether the same nodes have similar relative values in both networks.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network, get_distance_matrix
>>> from gnm.evaluation import DegreeCorrelation
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create smoothing matrix (here using identity for example)
>>> num_nodes = real_network.shape[-1]
>>> smoothing_matrix = torch.eye(num_nodes)
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate degree spatial correlation
>>> criterion = DegreeCorrelation(smoothing_matrix)
>>> dissimilarity = criterion(random_network, real_network)

See Also

utils.node_strengths: The function used to calculate node degrees
evaluation.CorrelationCriterion: The base class for correlation criteria, from which this class inherits.
evaluation.BinaryEvaluationCriterion: The base class for binary evaluation criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`smoothing_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix used to spatially smooth the degree values, which can help account for registration errors or spatial uncertainty in brain networks. Shape [num_nodes, num_nodes].	required

Source code in src/gnm/evaluation/binary_corr_criteria.py

def __init__(self, smoothing_matrix: Float[torch.Tensor, "num_nodes num_nodes"]):
    r"""
    Args:
        smoothing_matrix:
            Matrix used to spatially smooth the degree values, which can
            help account for registration errors or spatial uncertainty in brain networks.
            Shape [num_nodes, num_nodes].
    """
    CorrelationCriterion.__init__(self, smoothing_matrix)
    BinaryEvaluationCriterion.__init__(self)

`gnm.evaluation.ClusteringCorrelation(smoothing_matrix)`

Bases: CorrelationCriterion, BinaryEvaluationCriterion

Compare spatial patterns of clustering coefficients using correlation.

This criterion measures the similarity between the spatial patterns of clustering coefficients in synthetic and real networks using Pearson correlation. The clustering coefficient measures the degree to which nodes in a graph tend to cluster together.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network, get_distance_matrix
>>> from gnm.evaluation import ClusteringCorrelation
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create smoothing matrix (here using identity for example)
>>> num_nodes = real_network.shape[-1]
>>> smoothing_matrix = torch.eye(num_nodes)
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate clustering coefficient spatial correlation
>>> criterion = ClusteringCorrelation(smoothing_matrix)
>>> dissimilarity = criterion(random_network, real_network)

See Also

utils.binary_clustering_coefficients: Function used to calculate clustering coefficients
evaluation.CorrelationCriterion: Base class for correlation criteria, from which this class inherits.
evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`smoothing_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix used to spatially smooth the clustering coefficient values, which can help account for registration errors or spatial uncertainty in brain networks. Shape [num_nodes, num_nodes].	required

Source code in src/gnm/evaluation/binary_corr_criteria.py

def __init__(self, smoothing_matrix: Float[torch.Tensor, "num_nodes num_nodes"]):
    r"""
    Args:
        smoothing_matrix:
            Matrix used to spatially smooth the clustering coefficient values, which can
            help account for registration errors or spatial uncertainty in brain networks.
            Shape [num_nodes, num_nodes].
    """
    CorrelationCriterion.__init__(self, smoothing_matrix)
    BinaryEvaluationCriterion.__init__(self)

`gnm.evaluation.BetweennessCorrelation(smoothing_matrix)`

Bases: CorrelationCriterion, BinaryEvaluationCriterion

Compare spatial patterns of betweenness centrality using correlation.

This criterion measures the similarity between the spatial patterns of betweenness centrality in synthetic and real networks using Pearson correlation. Betweenness centrality quantifies the number of times a node acts as a bridge along the shortest path between two other nodes.

Examples:

>>> import torch
>>> from gnm.defaults import get_binary_network, get_distance_matrix
>>> from gnm.evaluation import BetweennessCorrelation
>>> from gnm.utils import get_control
>>> # Load a default binary network
>>> real_network = get_binary_network()
>>> # Create smoothing matrix (here using identity for example)
>>> num_nodes = real_network.shape[-1]
>>> smoothing_matrix = torch.eye(num_nodes)
>>> # Create a random network
>>> random_network = get_control(real_network)
>>> # Calculate betweenness centrality spatial correlation
>>> criterion = BetweennessCorrelation(smoothing_matrix)
>>> dissimilarity = criterion(random_network, real_network)

See Also

utils.binary_betweenness_centrality: Function used to calculate betweenness centrality
evaluation.CorrelationCriterion: Base class for correlation criteria, from which this class inherits.
evaluation.BinaryEvaluationCriterion: Base class for binary evaluation criteria, from which this class inherits.

Parameters:

Name	Type	Description	Default
`smoothing_matrix`	`Float[Tensor, 'num_nodes num_nodes']`	Matrix used to spatially smooth the betweenness centrality values, which can help account for registration errors or spatial uncertainty in brain networks. Shape [num_nodes, num_nodes].	required

Source code in src/gnm/evaluation/binary_corr_criteria.py

def __init__(self, smoothing_matrix: Float[torch.Tensor, "num_nodes num_nodes"]):
    r"""
    Args:
        smoothing_matrix:
            Matrix used to spatially smooth the betweenness centrality values, which can
            help account for registration errors or spatial uncertainty in brain networks.
            Shape [num_nodes, num_nodes].
    """

    CorrelationCriterion.__init__(self, smoothing_matrix)
    BinaryEvaluationCriterion.__init__(self)

gnm.evaluation

evaluation

Evaluation

gnm.evaluation.EvaluationCriterion(device=None)

__str__()

__call__(synthetic_matrices, real_matrices)

_pre_call(matrices) abstractmethod

_evaluate(synthetic_matrices, real_matrices) abstractmethod

Criterion types

gnm.evaluation.BinaryEvaluationCriterion()

_pre_call(matrices)

gnm.evaluation.WeightedEvaluationCriterion()

_pre_call(matrices)

gnm.evaluation.KSCriterion(device=None)

_get_graph_statistics(matrices) abstractmethod

gnm.evaluation.CorrelationCriterion(smoothing_matrix)

_get_graph_statistics(matrices) abstractmethod

Composite Criteria

gnm.evaluation.CompositeCriterion(criteria)

gnm.evaluation.MaxCriteria(criteria)

gnm.evaluation.MeanCriteria(criteria)

gnm.evaluation.WeightedSumCriteria(criteria, weights)

Binary KS Criteria

gnm.evaluation.DegreeKS()

gnm.evaluation.ClusteringKS()

gnm.evaluation.BetweennessKS()

gnm.evaluation.EdgeLengthKS(distance_matrix)

Weighted KS Criteria

gnm.evaluation.WeightedNodeStrengthKS(normalise=True)

gnm.evaluation.WeightedBetweennessKS(normalise=True)

gnm.evaluation.WeightedClusteringKS()

Binary Correlation Criteria

gnm.evaluation.DegreeCorrelation(smoothing_matrix)

gnm.evaluation.ClusteringCorrelation(smoothing_matrix)

gnm.evaluation.BetweennessCorrelation(smoothing_matrix)

`evaluation`

`gnm.evaluation.EvaluationCriterion(device=None)`

`str()`

`call(synthetic_matrices, real_matrices)`

`_pre_call(matrices)` `abstractmethod`

`_evaluate(synthetic_matrices, real_matrices)` `abstractmethod`

`gnm.evaluation.BinaryEvaluationCriterion()`

`_pre_call(matrices)`

`gnm.evaluation.WeightedEvaluationCriterion()`

`_pre_call(matrices)`

`gnm.evaluation.KSCriterion(device=None)`

`_get_graph_statistics(matrices)` `abstractmethod`

`gnm.evaluation.CorrelationCriterion(smoothing_matrix)`

`_get_graph_statistics(matrices)` `abstractmethod`

`gnm.evaluation.CompositeCriterion(criteria)`

`gnm.evaluation.MaxCriteria(criteria)`

`gnm.evaluation.MeanCriteria(criteria)`

`gnm.evaluation.WeightedSumCriteria(criteria, weights)`

`gnm.evaluation.DegreeKS()`

`gnm.evaluation.ClusteringKS()`

`gnm.evaluation.BetweennessKS()`

`gnm.evaluation.EdgeLengthKS(distance_matrix)`

`gnm.evaluation.WeightedNodeStrengthKS(normalise=True)`

`gnm.evaluation.WeightedBetweennessKS(normalise=True)`

`gnm.evaluation.WeightedClusteringKS()`

`gnm.evaluation.DegreeCorrelation(smoothing_matrix)`

`gnm.evaluation.ClusteringCorrelation(smoothing_matrix)`

`gnm.evaluation.BetweennessCorrelation(smoothing_matrix)`