Backend

Base RNN Object

Classes:

RNN(params)

The base recurrent neural network class.
class psychrnn.backend.rnn.RNN(params)[source]

Bases: ABC

The base recurrent neural network class.

Note

The base RNN class is not itself a functioning RNN. forward_pass must be implemented to define a functioning RNN.

Parameters

  • params (dict) – The RNN parameters. Use your tasks’s get_task_params() function to start building this dictionary. Optionally use a different network’s get_weights() function to initialize the network with preexisting weights.

  • Dictionary Keys:

    • name (str) – Unique name used to determine variable scope. Having different variable scopes allows multiple distinct models to be instantiated in the same TensorFlow environment. See TensorFlow’s variable_scope for more details.

    • N_in (int) – The number of network inputs.

    • N_rec (int) – The number of recurrent units in the network.

    • N_out (int) – The number of network outputs.

    • N_steps (int): The number of simulation timesteps in a trial.

    • dt (float) – The simulation timestep.

    • tau (float) – The intrinsic time constant of neural state decay.

    • N_batch (int) – The number of trials per training update.

    • rec_noise (float, optional) – How much recurrent noise to add each time the new state of the network is calculated. Default: 0.0.

    • load_weights_path (str, optional) – When given a path, loads weights from file in that path. Default: None

    • initializer (WeightInitializer or child object, optional) – Initializer to use for the network. Default: WeightInitializer (params) if params includes W_rec or load_weights_path as a key, GaussianSpectralRadius (params) otherwise.

    • W_in_train (bool, optional) – True if input weights, W_in, are trainable. Default: True

    • W_rec_train (bool, optional) – True if recurrent weights, W_rec, are trainable. Default: True

    • W_out_train (bool, optional) – True if output weights, W_out, are trainable. Default: True

    • b_rec_train (bool, optional) – True if recurrent bias, b_rec, is trainable. Default: True

    • b_out_train (bool, optional) – True if output bias, b_out, is trainable. Default: True

    • init_state_train (bool, optional) – True if the inital state for the network, init_state, is trainable. Default: True

    • loss_function (str, optional) – Which loss function to use. See psychrnn.backend.loss_functions.LossFunction for details. Defaults to "mean_squared_error".

    Other Dictionary Keys

    • Any dictionary keys used by the regularizer will be passed onwards to psychrnn.backend.regularizations.Regularizer. See Regularizer for key names and details.

    • Any dictionary keys used for the loss function will be passed onwards to psychrnn.backend.loss_functions.LossFunction. See LossFunction for key names and details.

    • If initializer is not set, any dictionary keys used by the initializer will be pased onwards to WeightInitializer if load_weights_path is set or W_rec is passed in. Otherwise all keys will be passed to GaussianSpectralRadius

    • If initializer is not set and load_weights_path is not set, the dictionary entries returned previously by get_weights() can be passed in to initialize the network. See WeightInitializer for a list and explanation of possible parameters. At a minimum, W_rec must be included as a key to make use of this option.

    • If initializer is not set and load_weights_path is not set, the following keys can be used to set biological connectivity constraints:

      • input_connectivity (ndarray(dtype=float, shape=( N_rec, N_in )), optional) – Connectivity mask for the input layer. 1 where connected, 0 where unconnected. Default: np.ones((N_rec, N_in)).

      • rec_connectivity (ndarray(dtype=float, shape=( N_rec, N_rec )), optional) – Connectivity mask for the recurrent layer. 1 where connected, 0 where unconnected. Default: np.ones((N_rec, N_rec)).

      • output_connectivity (ndarray(dtype=float, shape=( N_out, N_rec )), optional) – Connectivity mask for the output layer. 1 where connected, 0 where unconnected. Default: np.ones((N_out, N_rec)).

      • autapses (bool, optional) – If False, self connections are not allowed in N_rec, and diagonal of rec_connectivity will be set to 0. Default: True.

      • dale_ratio (float, optional) – Dale’s ratio, used to construct Dale_rec and Dale_out. 0 <= dale_ratio <=1 if dale_ratio should be used. dale_ratio * N_rec recurrent units will be excitatory, the rest will be inhibitory. Default: None

      • transfer_function (function, optional) – Transfer function to use for the network. Default: tf.nn.relu.

    Inferred Parameters:

    • alpha (float) – The number of unit time constants per simulation timestep.

Methods:

build()

Build the TensorFlow network and start a TensorFlow session.

destruct()

Close the TensorFlow session and reset the global default graph.

forward_pass()

Run the RNN on a batch of task inputs.

get_effective_W_in()

Get the input weights used in the network, after masking by connectivity and dale_ratio.

get_effective_W_out()

Get the output weights used in the network, after masking by connectivity, and dale_ratio.

get_effective_W_rec()

Get the recurrent weights used in the network, after masking by connectivity and dale_ratio.

get_weights()

Get weights used in the network.

save(save_path)

Save the weights returned by get_weights() to save_path

test(trial_batch)

Test the network on a certain task input.

train(trial_batch_generator[, train_params])

Train the network.

train_curric(train_params)

Wrapper function for training with curriculum to streamline curriculum learning.
build()[source]

Build the TensorFlow network and start a TensorFlow session.

destruct()[source]

Close the TensorFlow session and reset the global default graph.

abstract forward_pass()[source]

Run the RNN on a batch of task inputs.

Note

This is an abstract function that must be defined in a child class.

Returns

  • predictions (ndarray(dtype=float, shape=(N_batch, N_steps, N_out ))) – Network output on inputs found in self.x within the tf network.

  • states (ndarray(dtype=float, shape=(N_batch, N_steps, N_rec ))) – State variable values over the course of the trials found in self.x within the tf network.

Return type

tuple

get_effective_W_in()[source]

Get the input weights used in the network, after masking by connectivity and dale_ratio.

Returns

tf.Tensor(dtype=float, shape=(N_rec, N_in ))

get_effective_W_out()[source]

Get the output weights used in the network, after masking by connectivity, and dale_ratio.

Returns

tf.Tensor(dtype=float, shape=(N_out, N_rec ))

get_effective_W_rec()[source]

Get the recurrent weights used in the network, after masking by connectivity and dale_ratio.

Returns

tf.Tensor(dtype=float, shape=(N_rec, N_rec ))

get_weights()[source]

Get weights used in the network.

Allows for rebuilding or tweaking different weights to do experiments / analyses.

Returns

Dictionary of rnn weights including the following keys:

Dictionary Keys

  • init_state (ndarray(dtype=float, shape=(1, :attr:`N_rec` *))) – Initial state of the network’s recurrent units.

  • W_in (ndarray(dtype=float, shape=(:attr:`N_rec`. :attr:`N_in` *))) – Input weights.

  • W_rec (ndarray(dtype=float, shape=(:attr:`N_rec`, :attr:`N_rec` *))) – Recurrent weights.

  • W_out (ndarray(dtype=float, shape=(:attr:`N_out`, :attr:`N_rec` *))) – Output weights.

  • b_rec (ndarray(dtype=float, shape=(:attr:`N_rec`, *))) – Recurrent bias.

  • b_out (ndarray(dtype=float, shape=(:attr:`N_out`, *))) – Output bias.

  • Dale_rec (ndarray(dtype=float, shape=(:attr:`N_rec`, :attr:`N_rec`))*) – Diagonal matrix with ones and negative ones on the diagonal. If dale_ratio is not None, indicates whether a recurrent unit is excitatory(1) or inhibitory(-1).

  • Dale_out (ndarray(dtype=float, shape=(:attr:`N_rec`, :attr:`N_rec`))*) – Diagonal matrix with ones and zeroes on the diagonal. If dale_ratio is not None, indicates whether a recurrent unit is excitatory(1) or inhibitory(0). Inhibitory neurons do not contribute to the output.

  • input_connectivity (ndarray(dtype=float, shape=(:attr:`N_rec`, :attr:`N_in`))*) – Connectivity mask for the input layer. 1 where connected, 0 where unconnected.

  • rec_connectivity (ndarray(dtype=float, shape=(:attr:`N_rec`, :attr:`N_rec`))*) – Connectivity mask for the recurrent layer. 1 where connected, 0 where unconnected.

  • output_connectivity (ndarray(dtype=float, shape=(:attr:`N_out`, :attr:`N_rec`))*) – Connectivity mask for the output layer. 1 where connected, 0 where unconnected.

  • dale_ratio (float) – Dale’s ratio, used to construct Dale_rec and Dale_out. Either None if dale’s law was not applied, or 0 <= dale_ratio <=1 if dale_ratio was applied.

  • transfer_function (function) – Transfer function to use for the network.

Note

Keys returned may be different / include other keys depending on the implementation of RNN used. A different set of keys will be included e.g. if the LSTM implementation is used. The set of keys above is accurate and meaningful for the Basic and BasicScan implementations.

Return type

dict

save(save_path)[source]

Save the weights returned by get_weights() to save_path

Parameters

save_path (str) – Path for where to save the network weights.

test(trial_batch)[source]

Test the network on a certain task input.

Parameters

trial_batch ((ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task stimulus to run the network on. Stimulus from psychrnn.tasks.task.Task.get_trial_batch(), or from next(psychrnn.tasks.task.Task.batch_generator() ).

Returns

  • outputs (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Output time series of the network for each trial in the batch.

  • states (ndarray(dtype=float, shape =(N_batch, N_steps, N_rec ))) – Activity of recurrent units during each trial.

Return type

tuple

train(trial_batch_generator, train_params={})[source]

Train the network.

Parameters

  • trial_batch_generator (Task object or Generator[tuple, None, None]) – the task to train on, or the task to train on’s batch_generator. If a task is passed in, task.:func:batch_generator () will be called to get the generator for the task to train on.

  • train_params (dict, optional) – Dictionary of training parameters containing the following possible keys:

    Dictionary Keys

    • learning_rate (float, optional) – Sets learning rate if use default optimizer Default: .001

    • training_iters (int, optional) – Number of iterations to train for Default: 50000.

    • loss_epoch (int, optional) – Compute and record loss every ‘loss_epoch’ epochs. Default: 10.

    • verbosity (bool, optional) – If true, prints information as training progresses. Default: True.

    • save_weights_path (str, optional) – Where to save the model after training. Default: None

    • save_training_weights_epoch (int, optional) – Save training weights every ‘save_training_weights_epoch’ epochs. Weights only actually saved if training_weights_path is set. Default: 100.

    • training_weights_path (str, optional) – What directory to save training weights into as training progresses. Default: None.

    • curriculum (~psychrnn.backend.curriculum.Curriculum object, optional) – Curriculum to train on. If a curriculum object is provided, it overrides the trial_batch_generator argument. Default: None.

    • optimizer (tf.compat.v1.train.Optimizer object, optional) – What optimizer to use to compute gradients. Default: tf.train.AdamOptimizer (learning_rate=:data:train_params`[‘learning_rate’] ).

    • clip_grads (bool, optional) – If true, clip gradients by norm 1. Default: True

    • fixed_weights (dict, optional) – By default all weights are allowed to train unless fixed_weights or W_rec_train, W_in_train, or W_out_train are set. Default: None. Dictionary of weights to fix (not allow to train) with the following optional keys:

      Fixed Weights Dictionary Keys (in case of Basic and BasicScan implementations)

      • W_in (ndarray(dtype=bool, shape=(:attr:`N_rec`. :attr:`N_in` *)), optional) – True for input weights that should be fixed during training.

      • W_rec (ndarray(dtype=bool, shape=(:attr:`N_rec`, :attr:`N_rec` *)), optional) – True for recurrent weights that should be fixed during training.

      • W_out (ndarray(dtype=bool, shape=(:attr:`N_out`, :attr:`N_rec` *)), optional) – True for output weights that should be fixed during training.

      Note

      In general, any key in the dictionary output by get_weights() can have a key in the fixed_weights matrix, however fixed_weights will only meaningfully apply to trainable matrices.

    • performance_cutoff (float) – If performance_measure is not None, training stops as soon as performance_measure surpases the performance_cutoff. Default: None.

    • performance_measure (function) – Function to calculate the performance of the network using custom criteria. Default: None.

      Arguments

      • trial_batch (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))): Task stimuli for N_batch trials.

      • trial_y (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))): Target output for the network on N_batch trials given the trial_batch.

      • output_mask (ndarray(dtype=bool, shape =(N_batch, N_steps, N_out ))): Output mask for N_batch trials. True when the network should aim to match the target output, False when the target output can be ignored.

      • output (ndarray(dtype=bool, shape =(N_batch, N_steps, N_out ))): Output to compute the accuracy of. output as returned by psychrnn.backend.rnn.RNN.test().

      • epoch (int): Current training epoch (e.g. perhaps the performance_measure is calculated differently early on vs late in training)

      • losses (list of float): List of losses from the beginning of training until the current epoch.

      • verbosity (bool): Passed in from train_params.

      Returns

      float

      Performance, greater when the performance is better.

Returns

  • losses (list of float) – List of losses, computed every loss_epoch epochs during training.

  • training_time (float) – Time spent training.

  • initialization_time (float) – Time spent initializing the network and preparing to train.

Return type

tuple

train_curric(train_params)[source]

Wrapper function for training with curriculum to streamline curriculum learning.

Parameters

train_params (dict, optional) – See train() for details.

Returns

See train() for details.

Return type

tuple

Implemented RNN Models

Basic (Vanilla) RNNs

Classes:

Basic(params)

The basic continuous time recurrent neural network model.

BasicScan(params)

The basic continuous time recurrent neural network model implemented with tf.scan .
class psychrnn.backend.models.basic.Basic(params)[source]

Bases: RNN

The basic continuous time recurrent neural network model.

Basic implementation of psychrnn.backend.rnn.RNN with a simple RNN, enabling biological constraints.

Parameters

params (dict) – See psychrnn.backend.rnn.RNN for details.

Methods:

forward_pass()

Run the RNN on a batch of task inputs.

output_timestep(state)

Returns the output node activity for a given timestep.

recurrent_timestep(rnn_in, state)

Recurrent time step.
forward_pass()[source]

Run the RNN on a batch of task inputs.

Iterates over timesteps, running the recurrent_timestep() and output_timestep()

Implements psychrnn.backend.rnn.RNN.forward_pass().

Returns

  • predictions (tf.Tensor(N_batch, N_steps, N_out ))) – Network output on inputs found in self.x within the tf network.

  • states (tf.Tensor(N_batch, N_steps, N_rec ))) – State variable values over the course of the trials found in self.x within the tf network.

Return type

tuple

output_timestep(state)[source]

Returns the output node activity for a given timestep.

Parameters

state (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – State of network at a given timepoint for each trial in the batch.

Returns

Output of the network at a given timepoint for each trial in the batch.

Return type

output (tf.Tensor(dtype=float, shape=( N_batch , N_out )))

recurrent_timestep(rnn_in, state)[source]

Recurrent time step.

Given input and previous state, outputs the next state of the network.

Parameters

  • rnn_in (tf.Tensor(dtype=float, shape=(?, N_in ))) – Input to the rnn at a certain time point.

  • state (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – State of network at previous time point.

Returns

New state of the network.

Return type

new_state (tf.Tensor(dtype=float, shape=( N_batch , N_rec )))

class psychrnn.backend.models.basic.BasicScan(params)[source]

Bases: Basic

The basic continuous time recurrent neural network model implemented with tf.scan .

Produces the same results as Basic, with possible differences in execution time.

Parameters

params (dict) – See psychrnn.backend.rnn.RNN for details.

Methods:

forward_pass()

Run the RNN on a batch of task inputs.

output_timestep(dummy, state)

Wrapper function for psychrnn.backend.models.basic.Basic.output_timestep().

recurrent_timestep(state, rnn_in)

Wrapper function for psychrnn.backend.models.basic.Basic.recurrent_timestep().
forward_pass()[source]

Run the RNN on a batch of task inputs.

Iterates over timesteps, running the recurrent_timestep() and output_timestep()

Implements psychrnn.backend.rnn.RNN.forward_pass().

Returns

  • predictions (tf.Tensor(N_batch, N_steps, N_out ))) – Network output on inputs found in self.x within the tf network.

  • states (tf.Tensor(N_batch, N_steps, N_rec ))) – State variable values over the course of the trials found in self.x within the tf network.

Return type

tuple

output_timestep(dummy, state)[source]

Wrapper function for psychrnn.backend.models.basic.Basic.output_timestep().

Includes additional dummy argument to facilitate tf.scan.

Parameters

  • dummy – Dummy variable provided by tf.scan. Not actually used by the function.

  • state (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – State of network at a given timepoint for each trial in the batch.

Returns

Output of the network at a given timepoint for each trial in the batch.

Return type

output (tf.Tensor(dtype=float, shape=( N_batch , N_out )))

recurrent_timestep(state, rnn_in)[source]

Wrapper function for psychrnn.backend.models.basic.Basic.recurrent_timestep().

Parameters

  • state (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – State of network at previous time point.

  • rnn_in (tf.Tensor(dtype=float, shape=(?, N_in ))) – Input to the rnn at a certain time point.

Returns

New state of the network.

Return type

new_state (tf.Tensor(dtype=float, shape=( N_batch , N_rec )))

LSTM

Classes:

LSTM(params)

LSTM (Long Short Term Memory) recurrent network model
class psychrnn.backend.models.lstm.LSTM(params)[source]

Bases: RNN

LSTM (Long Short Term Memory) recurrent network model

LSTM implementation of psychrnn.backend.rnn.RNN. Because LSTM is structured differently from the basic RNN, biological constraints such as dale’s, autapses, and connectivity are not enabled.

Parameters

params (dict) – See psychrnn.backend.rnn.RNN for details.

Methods:

forward_pass()

Run the LSTM on a batch of task inputs.

output_timestep(hidden)

Returns the output node activity for a given timestep.

recurrent_timestep(rnn_in, hidden, cell)

Recurrent time step.
forward_pass()[source]

Run the LSTM on a batch of task inputs.

Iterates over timesteps, running the recurrent_timestep() and output_timestep()

Implements psychrnn.backend.rnn.RNN.forward_pass().

Returns

  • predictions (tf.Tensor(N_batch, N_steps, N_out ))) – Network output on inputs found in self.x within the tf network.

  • hidden (tf.Tensor(N_batch, N_steps, N_rec ))) – Hidden unit values over the course of the trials found in self.x within the tf network.

Return type

tuple

output_timestep(hidden)[source]

Returns the output node activity for a given timestep.

Parameters

hidden (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – Hidden units of network at a given timepoint for each trial in the batch.

Returns

Output of the network at a given timepoint for each trial in the batch.

Return type

output (tf.Tensor(dtype=float, shape=( N_batch , N_out )))

recurrent_timestep(rnn_in, hidden, cell)[source]

Recurrent time step.

Given input and previous state, outputs the next state of the network.

Parameters

  • rnn_in (tf.Tensor(dtype=float, shape=(?, N_in ))) – Input to the rnn at a certain time point.

  • hidden (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – Hidden units state of network at previous time point.

  • cell (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – Cell state of the network at previous time point.

Returns

  • new_hidden (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – New hidden unit state of the network.

  • new_cell (tf.Tensor(dtype=float, shape=( N_batch , N_rec ))) – New cell state of the network.

Return type

tuple

Backend Modules

Initializations

Classes:

AlphaIdentity(**kwargs)

Generate recurrent weights w(i,i) = alpha, w(i,j) = 0 where i \neq j.

GaussianSpectralRadius(**kwargs)

Generate random gaussian weights with specified spectral radius.

WeightInitializer(**kwargs)

Base Weight Initialization class.
class psychrnn.backend.initializations.AlphaIdentity(**kwargs)[source]

Bases: WeightInitializer

Generate recurrent weights w(i,i) = alpha, w(i,j) = 0 where i \neq j.

If Dale is set, balances the alpha excitatory and inhibitory weights using balance_dale_ratio(), so w(i,i) will not be exactly equal to alpha.

Keyword Arguments

alpha (float) – The value of alpha to set w(i,i) to in W_rec.

Other Keyword Args:

See WeightInitializer for details.

class psychrnn.backend.initializations.GaussianSpectralRadius(**kwargs)[source]

Bases: WeightInitializer

Generate random gaussian weights with specified spectral radius.

If Dale is set, balances the random gaussian weights between excitatory and inhibitory using balance_dale_ratio() before applying the specified spectral radius.

Keyword Arguments

spec_rad (float, optional) – The spectral radius to initialize W_rec with. Default: 1.1.

Other Keyword Args:

See WeightInitializer for details.

class psychrnn.backend.initializations.WeightInitializer(**kwargs)[source]

Bases: object

Base Weight Initialization class.

Initializes biological constraints and network weights, optionally loading weights from a file or from passed in arrays.

Keyword Arguments

  • N_in (int) – The number of network inputs.

  • N_rec (int) – The number of recurrent units in the network.

  • N_out (int) – The number of network outputs.

  • load_weights_path (str, optional) – Path to load weights from using np.load. Weights saved at that path should be in the form saved out by psychrnn.backend.rnn.RNN.save() Default: None.

  • input_connectivity (ndarray(dtype=float, shape=(N_rec, N_in)), optional) – Connectivity mask for the input layer. 1 where connected, 0 where unconnected. Default: np.ones((N_rec, N_in)).

  • rec_connectivity (ndarray(dtype=float, shape=(N_rec, N_rec)), optional) – Connectivity mask for the recurrent layer. 1 where connected, 0 where unconnected. Default: np.ones((N_rec, N_rec)).

  • output_connectivity (ndarray(dtype=float, shape=(N_out, N_rec)), optional) – Connectivity mask for the output layer. 1 where connected, 0 where unconnected. Default: np.ones((N_out, N_rec)).

  • autapses (bool, optional) – If False, self connections are not allowed in N_rec, and diagonal of rec_connectivity will be set to 0. Default: True.

  • dale_ratio (float, optional) – Dale’s ratio, used to construct Dale_rec and Dale_out. 0 <= dale_ratio <=1 if dale_ratio should be used. dale_ratio * N_rec recurrent units will be excitatory, the rest will be inhibitory. Default: None

  • which_rand_init (str, optional) – Which random initialization to use for W_in and W_out. Will also be used for W_rec if which_rand_W_rec_init is not passed in. Options: 'const_unif', 'const_gauss', 'glorot_unif', 'glorot_gauss'. Default: 'glorot_gauss'.

  • which_rand_W_rec_init (str, optional) – Which random initialization to use for W_rec. Options: 'const_unif', 'const_gauss', 'glorot_unif', 'glorot_gauss'. Default: which_rand_init.

  • init_minval (float, optional) – Used by const_unif_init() as minval if 'const_unif' is passed in for which_rand_init or which_rand_W_rec_init. Default: -.1.

  • init_maxval (float, optional) – Used by const_unif_init() as maxval if 'const_unif' is passed in for which_rand_init or which_rand_W_rec_init. Default: .1.

  • W_in (ndarray(dtype=float, shape=(N_rec, N_in )), optional) – Input weights. Default: Initialized using the function indicated by which_rand_init

  • W_rec (ndarray(dtype=float, shape=(N_rec, N_rec )), optional) – Recurrent weights. Default: Initialized using the function indicated by which_rand_W_rec_init

  • W_out (ndarray(dtype=float, shape=(N_out, N_rec )), optional) – Output weights. Defualt: Initialized using the function indicated by which_rand_init

  • b_rec (ndarray(dtype=float, shape=(N_rec, )), optional) – Recurrent bias. Default: np.zeros(N_rec)

  • b_out (ndarray(dtype=float, shape=(N_out, )), optional) – Output bias. Default: np.zeros(N_out)

  • Dale_rec (ndarray(dtype=float, shape=(N_rec, N_rec)), optional) – Diagonal matrix with ones and negative ones on the diagonal. If dale_ratio is not None, indicates whether a recurrent unit is excitatory(1) or inhibitory(-1). Default: constructed based on dale_ratio

  • Dale_out (ndarray(dtype=float, shape=(N_rec, N_rec)), optional) – Diagonal matrix with ones and zeroes on the diagonal. If dale_ratio is not None, indicates whether a recurrent unit is excitatory(1) or inhibitory(0). Inhibitory neurons do not contribute to the output. Default: constructed based on dale_ratio

  • init_state (ndarray(dtype=float, shape=(1, N_rec )), optional) – Initial state of the network’s recurrent units. Default: .1 + .01 * np.random.randn(N_rec ).

initializations

Dictionary containing entries for input_connectivity, rec_connectivity, output_connectivity,:data:transfer_function, dale_ratio, Dale_rec, Dale_out, W_in, W_rec, W_out, b_rec, b_out, and init_state.

Type

dict

Methods:

balance_dale_ratio()

If dale_ratio is not None, balances initializations['W_rec'] 's excitatory and inhibitory weights so the network will train.

const_gauss_init(connectivity)

Initialize ndarray of shape connectivity with values from a normal distribution.

const_unif_init(connectivity)

Initialize ndarray of shape connectivity with values uniform distribution with minimum init_minval and maximum init_maxval as set in WeightInitializer.

get(tensor_name)

Get tensor_name from initializations as a Tensor.

get_dale_ratio()

Returns the dale_ratio.

get_rand_init_func(which_rand_init)

Maps initialization function names (strings) to generating functions.

get_transfer_function()

Returns the transfer function.

glorot_gauss_init(connectivity)

Initialize ndarray of shape connectivity with values from a glorot normal distribution.

glorot_unif_init(connectivity)

Initialize ndarray of shape connectivity with values from a glorot uniform distribution.

save(save_path)

Save initializations to save_path.
balance_dale_ratio()[source]

If dale_ratio is not None, balances initializations['W_rec'] ‘s excitatory and inhibitory weights so the network will train.

const_gauss_init(connectivity)[source]

Initialize ndarray of shape connectivity with values from a normal distribution.

Parameters

connectivity (ndarray) – 1 where connected, 0 where unconnected.

Returns

ndarray(dtype=float, shape=connectivity.shape)

const_unif_init(connectivity)[source]

Initialize ndarray of shape connectivity with values uniform distribution with minimum init_minval and maximum init_maxval as set in WeightInitializer.

Parameters

connectivity (ndarray) – 1 where connected, 0 where unconnected.

Returns

ndarray(dtype=float, shape=connectivity.shape)

get(tensor_name)[source]

Get tensor_name from initializations as a Tensor.

Parameters

tensor_name (str) – The name of the tensor to get. See initializations for options.

Returns

Tensor object

get_dale_ratio()[source]

Returns the dale_ratio.

0 \leq dale\_ratio \leq 1 if dale_ratio should be used, dale_ratio = None otherwise. dale_ratio * N_rec recurrent units will be excitatory, the rest will be inhibitory.

Returns

Dale ratio, None if no dale ratio is set.

Return type

float

get_rand_init_func(which_rand_init)[source]

Maps initialization function names (strings) to generating functions.

Parameters

which_rand_init (str) – Maps to [which_rand_init]_init. Options are 'const_unif', 'const_gauss', 'glorot_unif', 'glorot_gauss'.

Returns

self.[which_rand_init]_init

Return type

function

get_transfer_function()[source]

Returns the transfer function.

Returns

transfer function, Default: tf.nn.relu.

Return type

function

glorot_gauss_init(connectivity)[source]

Initialize ndarray of shape connectivity with values from a glorot normal distribution.

Draws samples from a normal distribution centered on 0 with stddev = sqrt(2 / (fan_in + fan_out)) where fan_in is the number of input units and fan_out is the number of output units. Respects the connectivity matrix.

Parameters

connectivity (ndarray) – 1 where connected, 0 where unconnected.

Returns

ndarray(dtype=float, shape=connectivity.shape)

glorot_unif_init(connectivity)[source]

Initialize ndarray of shape connectivity with values from a glorot uniform distribution.

Draws samples from a uniform distribution within [-limit, limit] where limit is sqrt(6 / (fan_in + fan_out)) where fan_in is the number of input units and fan_out is the number of output units. Respects the connectivity matrix.

Parameters

connectivity (ndarray) – 1 where connected, 0 where unconnected.

Returns

ndarray(dtype=float, shape=connectivity.shape)

save(save_path)[source]

Save initializations to save_path.

Parameters

save_path (str) – File path for saving the initializations. The .npz extension will be appended if not already provided.

Loss Functions

Classes:

LossFunction(params)

Set the loss function for the RNN model.
class psychrnn.backend.loss_functions.LossFunction(params)[source]

Bases: object

Set the loss function for the RNN model.

Parameters

params (dict) – Dictionary of parameters including the following keys:

Dictionary Keys

  • loss_function (str) – String indicating what loss function to use. If params["loss_function"] is not mean_squared_error or binary_cross_entropy, params[params["loss_function"]] defines the custom loss function. Default: “mean_squared_error”.

  • params[“loss_function”] (function, optional) – Defines the custom loss function. Must have the same signature as mean_squared_error() and binary_cross_entropy().

Methods:

binary_cross_entropy(predictions, y, output_mask)

Binary cross-entropy.

mean_squared_error(predictions, y, output_mask)

Mean squared error.

set_model_loss(model)

Returns the model loss, calculated as indicated by type (inferred from params["loss_function"].
binary_cross_entropy(predictions, y, output_mask)[source]

Binary cross-entropy.

Binary label values are assumed to be 0 and 1.

loss = mean(output_mask * -(y * log(predictions) + (1-y)* log(1-predictions)))

Parameters

  • predictions (tf.Tensor(dtype=float, shape =(N_batch, N_steps, N_out ))) – Network output.

  • y (tf.Tensor(dtype=float, shape =(?, N_steps, N_out ))) – Target output.

  • output_mask (tf.Tensor(dtype=float, shape =(?, N_steps, N_out ))) – Output mask for N_batch trials. True when the network should aim to match the target output, False when the target output can be ignored.

Returns

Binary cross-entropy.

Return type

tf.Tensor(dtype=float)

mean_squared_error(predictions, y, output_mask)[source]

Mean squared error.

loss = mean(square(output_mask * (predictions - y)))

Parameters

  • predictions (tf.Tensor(dtype=float, shape =(N_batch, N_steps, N_out ))) – Network output.

  • y (tf.Tensor(dtype=float, shape =(?, N_steps, N_out ))) – Target output.

  • output_mask (tf.Tensor(dtype=float, shape =(?, N_steps, N_out ))) – Output mask for N_batch trials. True when the network should aim to match the target output, False when the target output can be ignored.

Returns

Mean squared error.

Return type

tf.Tensor(dtype=float)

set_model_loss(model)[source]

Returns the model loss, calculated as indicated by type (inferred from params["loss_function"].

'mean_squared_error' indicates mean_squared_error(), 'binary_cross_entropy' indicates binary_cross_entropy(). If type is not one of the above options, custom_loss_function is used. The custom loss function would have been passed in to params as params[type].

Parameters

model (RNN object) – Model for which to calculate the regularization.

Returns

Model loss.

Return type

tf.Tensor(dtype=float)

Regularizations

Classes:

Regularizer(params)

Regularizer Class
class psychrnn.backend.regularizations.Regularizer(params)[source]

Bases: object

Regularizer Class

Class that aggregates all types of regularization used.

Parameters

params (dict) – The regularization parameters containing the following optional keys:

Dictionary Keys

  • L1_in (float, optional) – Parameter for weighting the L1 input weights regularization. Default: 0.

  • L1_rec (float, optional) – Parameter for weighting the L1 recurrent weights regularization. Default: 0.

  • L1_out (float, optional) – Parameter for weighting the L1 output weights regularization. Default: 0.

  • L2_in (float, optional) – Parameter for weighting the L2 input weights regularization. Default: 0.

  • L2_rec (float, optional) – Parameter for weighting the L2 recurrent weights regularization. Default: 0.

  • L2_out (float, optional) – Parameter for weighting the L2 output weights regularization. Default: 0.

  • L2_firing_rate (float, optional) – Parameter for weighting the L2 regularization of the relu thresholded states. Default: 0.

  • custom_regularization (function, optional) – Custom regularization function. Default: None.

    Args:

    • model (RNN object) – Model for which to calculate the regularization.

    • params (dict) – Regularization parameters. All params passed to the Regularizer will be passed here.

    Returns:

    tf.Tensor(dtype=float)– The custom regularization to add when calculating the loss.

Methods:

L1_weight_reg(model)

L1 regularization

L2_firing_rate_reg(model)

L2 regularization of the firing rate.

L2_weight_reg(model)

L2 regularization

set_model_regularization(model)

Given model, calculate the regularization by adding all regualarization terms (scaled with the parameters to be either zero or nonzero).
L1_weight_reg(model)[source]

L1 regularization

regularization = L1\_in * ||W\_in||_1 + L1\_rec * ||W\_rec||_1 + L1\_out * ||W\_out||_1

Parameters

model (RNN object) – Model for which to calculate the regularization.

Returns

The L1 regularization to add when calculating the loss.

Return type

tf.Tensor(dtype=float)

L2_firing_rate_reg(model)[source]

L2 regularization of the firing rate.

regularization = L2\_firing\_rate * ||relu(states)||_2^2

Parameters

model (RNN object) – Model for which to calculate the regularization.

Returns

The L2 firing rate regularization to add when calculating the loss.

Return type

tf.Tensor(dtype=float)

L2_weight_reg(model)[source]

L2 regularization

regularization = L2\_in * ||W\_in||_2^2 + L2\_rec * ||W\_rec||_2^2 + L2\_out * ||W\_out||_2^2

Parameters

model (RNN object) – Model for which to calculate the regularization.

Returns

The L2 regularization to add when calculating the loss.

Return type

tf.Tensor(dtype=float)

set_model_regularization(model)[source]

Given model, calculate the regularization by adding all regualarization terms (scaled with the parameters to be either zero or nonzero).

The following regularizations are added: L1_weight_reg(), L2_weight_reg(), and L2_firing_rate_reg().

Parameters

model (RNN object) – Model for which to calculate the regularization.

Returns

The regularization to add when calculating the loss.

Return type

tf.Tensor(dtype=float)

Curriculum

Classes:

Curriculum(tasks, **kwargs)

Curriculum object.

Functions:

default_metric(curriculum_params, ...)

Default metric to use to evaluate performance when using Curriculum learning.
class psychrnn.backend.curriculum.Curriculum(tasks, **kwargs)[source]

Bases: object

Curriculum object.

Allows training on a sequence of tasks when Curriculum is passed into train().

Parameters

  • tasks (list of Task objects) – List of tasks to use in the curriculum.

  • metric (function, optional) – Function for calculating whether the stage advances and what the metric value is at each metric_epoch. Default: default_metric().

    Arguments

    • curriculum_params (dict) – Dictionary of the Curriculum object parameters, containing the following keys:

    Dictionary Keys

    • stop_training (bool) – True if the network has finished training and completed all stages.

    • stage (int) – Current training stage (initial stage is 0).

    • metric_values (list of [float, int]) – List of metric values and the stage at which each metric value was computed.

    • tasks (list of :class:`psychrnn.tasks.task.Task` objects) – List of tasks in the curriculum.

    • metric (function) – What metric function to use. default_metric() is an example of one in terms of inputs and outputs taken.

    • accuracies (list of functions) – Accuracy function to use at each stage.

    • thresholds (list of float) – Thresholds for each stage that accuracy must reach to move to the next stage.

    • metric_epoch (int) – Calculate the metric and test if the model should advance to the next stage every metric_epoch training epochs.

    • output_file (str) – Optional path for saving out themetric value and stage. If the .npz filename extension is not included, it will be appended.

    • input_data (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task inputs.

    • correct_output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out ))) – Correct (target) task output given input_data.

    • output_mask (ndarray(dtype=float, shape = (N_batch, N_steps, N_out ))) – Output mask for the task. True when the network should aim to match the target output, False when the target output can be ignored.

    • output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out ))) – The network’s output given input_data.

    • epoch (int) – The epoch number in training.

    • losses (list of float) – List of losses, computed during training.

    • verbosity (bool) – Whether to print information as training progresses.

    Returns

    tuple

    • advance (bool) – True if the the stage should be advanced. False otherwise.

    • metric_value (float) – Value of the computed metric.

  • accuracies (list of functions, optional) – Optional list of functions to use to calculate network performance for the purposes of advancing tasks. Used by default_metric() to compute accuracy. Default: [tasks[i].accuracy_function for i in range(len(tasks))].

  • thresholds (list of float, optional) – Optional list of thresholds. If metric = default_metric, accuracies must reach the threshold for a given stage in order to advance to the next stage. Default: [.9 for i in range(len(tasks))]

  • metric_epoch (int) – Calculate the metric and test if the model should advance to the next stage every metric_epoch training epochs. Default: 10

  • output_file (str) – Optional path for saving out the metric value and stage. If the .npz filename extension is not included, it will be appended. Default: None.

Methods:

batch_generator()

Returns a generator for the current task.

get_generator_function()

Depcreated method to return a generator for the current task.

metric_test(input_data, correct_output, ...)

Evaluates whether to advance the stage to the next task or not.
batch_generator()[source]

Returns a generator for the current task.

Returns

generator iterator for the current task

Return type

Generator[tuple, None, None]

Yields

tuple –

  • stimulus (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))): Task stimuli for N_batch trials.

  • target_output (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))): Target output for the network on N_batch trials given the stimulus.

  • output_mask (ndarray(dtype=bool, shape =(N_batch, N_steps, N_out ))): Output mask for N_batch trials. True when the network should aim to match the target output, False when the target output can be ignored.

  • trial_params (ndarray(dtype=dict, shape =(N_batch ,))): Array of dictionaries containing the trial parameters produced by generate_trial_params() for each trial in N_batch.

get_generator_function()[source]

Depcreated method to return a generator for the current task. Use batch_generator() instead.

metric_test(input_data, correct_output, output_mask, test_output, epoch, losses, verbosity=False)[source]

Evaluates whether to advance the stage to the next task or not.

Parameters

  • input_data (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task inputs.

  • correct_output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – Correct (target) task output given input_data.

  • output_mask (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – Output mask for the task. True when the network should aim to match the target output, False when the target output can be ignored.

  • test_output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – The network’s output given input_data.

  • epoch (int) – The epoch number in training.

  • losses (list of float) – List of losses, computed during training.

  • verbosity (bool, optional) – Whether to print information as metric is computed and stages advanced. Default: False

Returns

True if stage advances, False otherwise.

psychrnn.backend.curriculum.default_metric(curriculum_params, input_data, correct_output, output_mask, output, epoch, losses, verbosity)[source]

Default metric to use to evaluate performance when using Curriculum learning.

Advance is true if accuracy >= threshold, False otherwise.

Parameters

  • curriculum_params (dict) – Dictionary of the Curriculum object parameters, containing the following keys:

    Dictionary Keys

    • stop_training (bool) – True if the network has finished training and completed all stages.

    • stage (int) – Current training stage (initial stage is 0).

    • metric_values (list of [float, int]) – List of metric values and the stage at which each metric value was computed.

    • tasks (list of :class:`psychrnn.tasks.task.Task` objects) – List of tasks in the curriculum.

    • metric (function) – What metric function to use. default_metric() is an example of one in terms of inputs and outputs taken.

    • accuracies (list of functions with the signature of psychrnn.tasks.task.Task.accuracy_function()) – Accuracy function to use at each stage.

    • thresholds (list of float) – Thresholds for each stage that accuracy must reach to move to the next stage.

    • metric_epoch (int) – Calculate the metric / test if advance to the next stage every metric_epoch training epochs.

    • output_file (str) – Optional path for where to save out metric value and stage.

  • input_data (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task inputs.

  • correct_output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – Correct (target) task output given input_data.

  • output_mask (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – Output mask for the task. True when the network should aim to match the target output, False when the target output can be ignored.

  • output (ndarray(dtype=float, shape = (N_batch, N_steps, N_out))) – The network’s output given input_data.

  • epoch (int) – The epoch number in training.

  • losses (list of float) – List of losses, computed during training.

  • verbosity (bool) – Whether to print information as training progresses. If True, prints accuracy every time it is computed.

Returns

  • advance (bool) – True if the accuracy is >= the threshold for the current stage. False otherwise.

  • metric_value (float) – Value of the computed accuracy.

Return type

tuple

Simulation

Simulators implement the forward running of RNN models in NumPy, outside of the TensorFlow framework.

Classes:

BasicSimulator([rnn_model, params, ...])

Simulator implementation for psychrnn.backend.models.basic.Basic and for psychrnn.backend.models.basic.BasicScan.

LSTMSimulator([rnn_model, params, ...])

Simulator implementation for psychrnn.backend.models.lstm.LSTM and for psychrnn.backend.models.lstm.LSTM.

Simulator([rnn_model, params, weights_path, ...])

The simulator class.

Functions:

relu(x)

NumPy implementation of tf.nn.relu

sigmoid(x)

NumPy implementation of tf.nn.sigmoid
class psychrnn.backend.simulation.BasicSimulator(rnn_model=None, params=None, weights_path=None, weights=None, transfer_function=<function relu>)[source]

Bases: Simulator

Simulator implementation for psychrnn.backend.models.basic.Basic and for psychrnn.backend.models.basic.BasicScan.

See Simulator for arguments.

Methods:

rnn_step(state, rnn_in, t_connectivity)

Given input and previous state, outputs the next state and output of the network as a NumPy implementation of psychrnn.backend.models.basic.Basic.recurrent_timestep and of psychrnn.backend.models.basic.Basic.output_timestep.

run_trials(trial_input[, t_connectivity])

Test the network on a certain task input, optionally including ablation terms.
rnn_step(state, rnn_in, t_connectivity)[source]

Given input and previous state, outputs the next state and output of the network as a NumPy implementation of psychrnn.backend.models.basic.Basic.recurrent_timestep and of psychrnn.backend.models.basic.Basic.output_timestep.

Additionally takes in t_connectivity. If t_connectivity is all ones, rnn_step()’s output will match that of psychrnn.backend.models.basic.Basic.recurrent_timestep and psychrnn.backend.models.basic.Basic.output_timestep. Otherwise W_rec is multiplied by t_connectivity elementwise, ablating / perturbing the recurrent connectivity.

Parameters

  • state (ndarray(dtype=float, shape=(N_batch , N_rec))) – State of network at previous time point.

  • rnn_in (ndarray(dtype=float, shape=(N_batch , N_in))) – State of network at previous time point.

  • t_connectivity (ndarray(dtype=float, shape=(N_rec , N_rec))) – Matrix for ablating / perturbing W_rec.

Returns

  • new_output (ndarray(dtype=float, shape=(N_batch, N_out ))) – Output of the network at a given timepoint for each trial in the batch.

  • new_state (ndarray(dtype=float, shape=(N_batch, N_rec ))) – New state of the network for each trial in the batch.

Return type

tuple

run_trials(trial_input, t_connectivity=None)[source]

Test the network on a certain task input, optionally including ablation terms.

A NumPy implementation of test() with additional options for ablation.

N_batch here is flexible and will be inferred from trial_input.

Repeatedly calls rnn_step() to build output and states over the entire timecourse of the trial_batch

Parameters

  • trial_batch ((ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task stimulus to run the network on. Stimulus from psychrnn.tasks.task.Task.get_trial_batch(), or from next(psychrnn.tasks.task.Task.batch_generator() ). To run the network autonomously without input, set input to an array of zeroes. N_steps will still indicate for how many steps to run the network.

  • t_connectivity ((ndarray(dtype=float, shape =(N_steps, N_rec, N_rec ))) – Matrix for ablating / perturbing W_rec. Passed step by step to rnn_step().

Returns

  • outputs (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Output time series of the network for each trial in the batch.

  • states (ndarray(dtype=float, shape =(N_batch, N_steps, N_rec ))) – Activity of recurrent units during each trial.

Return type

tuple

class psychrnn.backend.simulation.LSTMSimulator(rnn_model=None, params=None, weights_path=None, weights=None)[source]

Bases: Simulator

Simulator implementation for psychrnn.backend.models.lstm.LSTM and for psychrnn.backend.models.lstm.LSTM.

See Simulator for arguments.

The contents of weights / np.load(weights_path) must now include the following additional keys:

Dictionary Keys

  • init_hidden (ndarray(dtype=float, shape=(N_batch , N_rec ))) – Initial state of the cell state.

  • init_hidden (ndarray(dtype=float, shape=(N_batch , N_rec ))) – Initial state of the hidden state.

  • W_f (ndarray(dtype=float, shape=(N_rec + N_in, N_rec ))) – f term weights

  • W_i (ndarray(dtype=float, shape=(N_rec + N_in, N_rec ))) – i term weights

  • W_c (ndarray(dtype=float, shape=(N_rec + N_in, N_rec ))) – c term weights

  • W_o (ndarray(dtype=float, shape=(N_rec + N_in, N_rec ))) – o term weights

  • b_f (ndarray(dtype=float, shape=(N_rec, ))) – f term bias.

  • b_i (ndarray(dtype=float, shape=(N_rec, ))) – i term bias.

  • b_c (ndarray(dtype=float, shape=(N_rec, ))) – c term bias.

  • b_o (ndarray(dtype=float, shape=(N_rec, ))) – o term bias.

Methods:

rnn_step(hidden, cell, rnn_in)

Given input and previous state, outputs the next state and output of the network as a NumPy implementation of psychrnn.backend.models.lstm.LSTM.recurrent_timestep and of psychrnn.backend.models.lstm.LSTM.output_timestep.

run_trials(trial_input)

Test the network on a certain task input, optionally including ablation terms.
rnn_step(hidden, cell, rnn_in)[source]

Given input and previous state, outputs the next state and output of the network as a NumPy implementation of psychrnn.backend.models.lstm.LSTM.recurrent_timestep and of psychrnn.backend.models.lstm.LSTM.output_timestep.

Parameters

  • hidden (ndarray(dtype=float, shape=(N_batch , N_rec ))) – Hidden units state of network at previous time point.

  • cell (ndarray(dtype=float, shape=(N_batch , N_rec ))) – Cell state of the network at previous time point.

  • rnn_in (ndarray(dtype=float, shape=(N_batch , N_in))) – State of network at previous time point.

Returns

  • new_output (ndarray(dtype=float, shape=(N_batch, N_out ))) – Output of the network at a given timepoint for each trial in the batch.

  • new_hidden (ndarray(dtype=float, shape=( N_batch , N_rec ))) – New hidden unit state of the network.

  • new_cell (ndarray(dtype=float, shape=( N_batch , N_rec ))) – New cell state of the network.

Return type

tuple

run_trials(trial_input)[source]

Test the network on a certain task input, optionally including ablation terms.

A NumPy implementation of test() with additional options for ablation.

N_batch here is flexible and will be inferred from trial_input.

Repeatedly calls rnn_step() to build output and states over the entire timecourse of the trial_batch

Parameters

trial_batch ((ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task stimulus to run the network on. Stimulus from psychrnn.tasks.task.Task.get_trial_batch(), or from next(psychrnn.tasks.task.Task.batch_generator() ). To run the network autonomously without input, set input to an array of zeroes. N_steps will still indicate for how many steps to run the network.

Returns

  • outputs (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Output time series of the network for each trial in the batch.

  • states (ndarray(dtype=float, shape =(N_batch, N_steps, N_rec ))) – Activity of recurrent units during each trial.

Return type

tuple

class psychrnn.backend.simulation.Simulator(rnn_model=None, params=None, weights_path=None, weights=None, transfer_function=<function relu>)[source]

Bases: ABC

The simulator class.

Note

The base Simulator class is not itself a functioning Simulator. run_trials and rnn_step must be implemented to define a functioning Simulator

Parameters

  • rnn_model (psychrnn.backend.rnn.RNN object, optional) – Uses the psychrnn.backend.rnn.RNN object to set alpha and rec_noise. Also used to initialize weights if weights and weights_path are not passed in. Default: None.

  • weights_path (str, optional) – Where to load weights from. Take precedence over rnn_model weights. Default: rnn_model.get_weights(). np.load(weights_path) should return something of the form weights.

  • transfer_function (function, optonal) – Function that takes an ndarray as input and outputs an ndarray of the same shape with the transfer / activation function applied. NumPy implementation of a TensorFlow transfer function. Default: relu().

  • weights (dict, optional) – Takes precedence over both weights_path and rnn_model. Default: np.load(weights_path). Dictionary containing the following keys:

    Dictionary Keys

    • init_state (ndarray(dtype=float, shape=(1, N_rec ))) – Initial state of the network’s recurrent units.

    • W_in (ndarray(dtype=float, shape=(N_rec. N_in ))) – Input weights.

    • W_rec (ndarray(dtype=float, shape=(N_rec, N_rec ))) – Recurrent weights.

    • W_out (ndarray(dtype=float, shape=(N_out, N_rec ))) – Output weights.

    • b_rec (ndarray(dtype=float, shape=(N_rec, ))) – Recurrent bias.

    • b_out (ndarray(dtype=float, shape=(N_out, ))) – Output bias.

  • params (dict, optional) –

    Dictionary Keys

    • rec_noise (float, optional) – Amount of recurrent noise to add to the network. Default: 0

    • alpha (float, optional) – The number of unit time constants per simulation timestep. Defaut: (1.0* dt) / tau

    • dt (float, optional) – The simulation timestep. Used to calculate alpha if alpha is not passed in. Required if alpha is not in params and rnn_model is None.

    • tau (float) – The intrinsic time constant of neural state decay. Used to calculate alpha if alpha is not passed in. Required if alpha is not in params and rnn_model is None.

Methods:

rnn_step(state, rnn_in, t_connectivity)

Given input and previous state, outputs the next state and output of the network.

run_trials(trial_input[, t_connectivity])

Test the network on a certain task input, optionally including ablation terms.
abstract rnn_step(state, rnn_in, t_connectivity)[source]

Given input and previous state, outputs the next state and output of the network.

Note

This is an abstract function that must be defined in a child class.

Parameters

  • state (ndarray(dtype=float, shape=(N_batch , N_rec))) – State of network at previous time point.

  • rnn_in (ndarray(dtype=float, shape=(N_batch , N_in))) – State of network at previous time point.

  • t_connectivity (ndarray(dtype=float, shape=(N_rec , N_rec))) – Matrix for ablating / perturbing W_rec.

Returns

  • new_output (ndarray(dtype=float, shape=(N_batch, N_out ))) – Output of the network at a given timepoint for each trial in the batch.

  • new_state (ndarray(dtype=float, shape=(N_batch, N_rec ))) – New state of the network for each trial in the batch.

Return type

tuple

abstract run_trials(trial_input, t_connectivity=None)[source]

Test the network on a certain task input, optionally including ablation terms.

A NumPy implementation of test() with additional options for ablation.

N_batch here is flexible and will be inferred from trial_input.

Parameters

  • trial_batch ((ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Task stimulus to run the network on. Stimulus from psychrnn.tasks.task.Task.get_trial_batch(), or from next(psychrnn.tasks.task.Task.batch_generator() ). If you want the network to run autonomously, without input, set input to an array of zeroes, N_steps will still indicate how long to run the network.

  • t_connectivity ((ndarray(dtype=float, shape =(N_steps, N_rec, N_rec ))) – Matrix for ablating / perturbing W_rec. Passed step by step to rnn_step.

Returns

  • outputs (ndarray(dtype=float, shape =(N_batch, N_steps, N_out ))) – Output time series of the network for each trial in the batch.

  • states (ndarray(dtype=float, shape =(N_batch, N_steps, N_rec ))) – Activity of recurrent units during each trial.

Return type

tuple

psychrnn.backend.simulation.relu(x)[source]

NumPy implementation of tf.nn.relu

Parameters

x (ndarray) – array for which relu is computed.

Returns

np.maximum(x,0)

Return type

ndarray

psychrnn.backend.simulation.sigmoid(x)[source]

NumPy implementation of tf.nn.sigmoid

Parameters

x (ndarray) – array for which sigmoid is computed.

Returns

1/(1 + np.exp(-x))

Return type

ndarray