Performing sweeps¶

In this example notebook, we use the gnm package to perform a sweep over generative rules and parameter values.

In [1]:

import os
notebook_dir = os.getcwd() 

# able to run example script from within Jupyter notebook in package
import sys
loc = os.path.abspath(os.path.join(notebook_dir, '..', '..', 'src'))
sys.path.append(loc)

# other basic imports
import time
import torch
from gnm import *
from gnm import defaults, utils, evaluation, fitting, generative_rules, weight_criteria

import os notebook_dir = os.getcwd() # able to run example script from within Jupyter notebook in package import sys loc = os.path.abspath(os.path.join(notebook_dir, '..', '..', 'src')) sys.path.append(loc) # other basic imports import time import torch from gnm import * from gnm import defaults, utils, evaluation, fitting, generative_rules, weight_criteria

In [ ]:

# device is the GPU if available, otherwise the CPU - GPU is much faster but ensure you have 
# GPU available in your PC/HPC

DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# device is the GPU if available, otherwise the CPU - GPU is much faster but ensure you have # GPU available in your PC/HPC DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

Loading in data¶

We'll start by loading in some data for our sweep. In particular, we'll need:

1. A distance matrix
2. A binary network to compare our networks to

We'll get these from the defaults sub-module, which has a built in distance matrix and consensus network.

In [ ]:

distance_matrix = defaults.get_distance_matrix(device=DEVICE)
binary_consensus_network = defaults.get_binary_network(device=DEVICE)

distance_matrix = defaults.get_distance_matrix(device=DEVICE) binary_consensus_network = defaults.get_binary_network(device=DEVICE)

We'll run the models until they have the same number of connections as the real binary consensus network

In [15]:

num_connections = int( binary_consensus_network.sum().item() / 2 )
print(f"The binary consensus network contains {num_connections} connections.")

num_connections = int( binary_consensus_network.sum().item() / 2 ) print(f"The binary consensus network contains {num_connections} connections.")

The binary consensus network contains 400 connections.

Defining our sweep¶

The next step is to define the parameters we want to sweep over. Here, we'll sweep over a range of values for $\eta$ and $\gamma$, while keeping the generative rule fixed (using the Matching Index) and the weight optimisation criterion fixed (using the distance weighted communicability).

For each set of parameters, we'll generate 100 networks using the model with that set of parameters. This means setting the number of simulations to 100.

In [ ]:

eta_values = torch.linspace(-5, -1, 3)
gamma_values = torch.linspace(-0.5, 0.5, 3)

binary_sweep_parameters = fitting.BinarySweepParameters(
    eta = eta_values,
    gamma = gamma_values,
    lambdah = torch.Tensor([0.0]),
    distance_relationship_type = ["powerlaw"],
    preferential_relationship_type = ["powerlaw"],
    heterochronicity_relationship_type = ["powerlaw"],
    generative_rule = [generative_rules.MatchingIndex()],
    num_iterations = [num_connections],
)

weighted_sweep_parameters = fitting.WeightedSweepParameters(
    alpha = [0.01],
    optimisation_criterion = [weight_criteria.DistanceWeightedCommunicability(distance_matrix=distance_matrix) ],
)   

num_simulations = 100

sweep_config = fitting.SweepConfig(
    binary_sweep_parameters = binary_sweep_parameters,
    weighted_sweep_parameters = weighted_sweep_parameters,
    num_simulations = num_simulations,
    distance_matrix = [distance_matrix]    
)

eta_values = torch.linspace(-5, -1, 3) gamma_values = torch.linspace(-0.5, 0.5, 3) binary_sweep_parameters = fitting.BinarySweepParameters( eta = eta_values, gamma = gamma_values, lambdah = torch.Tensor([0.0]), distance_relationship_type = ["powerlaw"], preferential_relationship_type = ["powerlaw"], heterochronicity_relationship_type = ["powerlaw"], generative_rule = [generative_rules.MatchingIndex()], num_iterations = [num_connections], ) weighted_sweep_parameters = fitting.WeightedSweepParameters( alpha = [0.01], optimisation_criterion = [weight_criteria.DistanceWeightedCommunicability(distance_matrix=distance_matrix) ], ) num_simulations = 100 sweep_config = fitting.SweepConfig( binary_sweep_parameters = binary_sweep_parameters, weighted_sweep_parameters = weighted_sweep_parameters, num_simulations = num_simulations, distance_matrix = [distance_matrix] )

Creating our evaluations¶

We want to evaluate how good the fit of our models is the real binary consensus network. For our evaluation criteria, we'll use the maximum of the KS statistics across clustering coefficient, degree, and edge length distributions.

In [6]:

criteria = [ evaluation.ClusteringKS(), evaluation.DegreeKS(), evaluation.EdgeLengthKS(distance_matrix) ]
energy = evaluation.MaxCriteria( criteria )
binary_evaluations = [energy]

criteria = [ evaluation.ClusteringKS(), evaluation.DegreeKS(), evaluation.EdgeLengthKS(distance_matrix) ] energy = evaluation.MaxCriteria( criteria ) binary_evaluations = [energy]

We also want to evaluate the fit of the weighted networks. We'll give a couple of evaluations for the weighted networks as well.

In [7]:

weighted_evaluations = [ evaluation.WeightedNodeStrengthKS(normalise=True), evaluation.WeightedClusteringKS() ]

weighted_evaluations = [ evaluation.WeightedNodeStrengthKS(normalise=True), evaluation.WeightedClusteringKS() ]

Performing the sweep¶

In [ ]:

start_time = time.perf_counter()

experiments = fitting.perform_sweep(sweep_config=sweep_config, 
                                binary_evaluations=binary_evaluations, 
                                real_binary_matrices=binary_consensus_network,
                                weighted_evaluations=weighted_evaluations,
                                save_model = False,
                                save_run_history = False,
)

end_time = time.perf_counter()

start_time = time.perf_counter() experiments = fitting.perform_sweep(sweep_config=sweep_config, binary_evaluations=binary_evaluations, real_binary_matrices=binary_consensus_network, weighted_evaluations=weighted_evaluations, save_model = False, save_run_history = False, ) end_time = time.perf_counter()

Let's look at the efficiency of the sweep.

In [ ]:

print(f"Sweep took {end_time - start_time:0.3f} seconds.")

total_simulations = num_simulations * len(eta_values) * len(gamma_values)

print(f"Total number of simulations: {total_simulations}")

print(f"Average time per simulation: {(end_time - start_time) / total_simulations:0.3f} seconds.")

print(f"Sweep took {end_time - start_time:0.3f} seconds.") total_simulations = num_simulations * len(eta_values) * len(gamma_values) print(f"Total number of simulations: {total_simulations}") print(f"Average time per simulation: {(end_time - start_time) / total_simulations:0.3f} seconds.")

Analysing the results¶

In [ ]:

optimal_experiments, optimal_energies = fitting.optimise_evaluation(
    experiments=experiments,
    criterion=energy,
)

optimal_experiment = optimal_experiments[0]
optimal_energy = optimal_energies[0]

optimal_experiments, optimal_energies = fitting.optimise_evaluation( experiments=experiments, criterion=energy, ) optimal_experiment = optimal_experiments[0] optimal_energy = optimal_energies[0]

In [ ]:

print(f"Optimal energy: {optimal_energy:0.3f}")
print(f"Optimal value of eta: {optimal_experiment.run_config.binary_parameters.eta:0.2f}")
print(f"Optimal value of gamma: {optimal_experiment.run_config.binary_parameters.gamma:0.2f}")

print(f"Optimal energy: {optimal_energy:0.3f}") print(f"Optimal value of eta: {optimal_experiment.run_config.binary_parameters.eta:0.2f}") print(f"Optimal value of gamma: {optimal_experiment.run_config.binary_parameters.gamma:0.2f}")