brainiak.reconstruct package¶

Inverted encoding model for recreating continuous representations.

Submodules¶

brainiak.reconstruct.iem module¶

Inverted Encoding Model (IEM)

Method to decode and reconstruct features from data.

The implementation is roughly based on the following publications:

[Kok2013] “1.Kok, P., Brouwer, G. J., Gerven, M. A. J. van & Lange, F. P. de. Prior Expectations Bias Sensory Representations in Visual Cortex. J. Neurosci. 33, 16275–16284 (2013).

[Brouwer2011] “2.Brouwer, G. J. & Heeger, D. J. Cross-orientation suppression in human visual cortex. J. Neurophysiol. 106(5): 2108-2119 (2011).

[Brouwer2009] “3.Brouwer, G. J. & Heeger, D. J. Decoding and Reconstructing Color from Responses in Human Visual Cortex. J. Neurosci. 29, 13992–14003 (2009).

This implementation uses a set of sinusoidal basis functions to represent the set of possible feature values. A feature value is some characteristic of a stimulus, e.g. the angular location of a target along a horizontal line. This code was written to give some flexibility compared to the specific instances in Kok, 2013 & in Brouwer, 2009. Users can set the number of basis functions, or channels, and the range of possible feature values.

There are separate classes for reconstructing feature values in a 1-dimensional (1D) space or in a 2-dimensional (2D) space.

class brainiak.reconstruct.iem.InvertedEncoding1D(n_channels=6, channel_exp=5, stimulus_mode='halfcircular', range_start=0.0, range_stop=180.0, channel_density=180, stimulus_resolution=None)¶

Bases: sklearn.base.BaseEstimator

Basis function-based reconstruction method

Inverted encoding models (alternatively known as forward models) are used to reconstruct a feature represented in some N-dimensional space, here 1D, (e.g. color of a stimulus) from patterns across voxels in functional data. The model uses n_channels number of idealized basis functions and assumes that the transformation from stimulus feature (e.g. color) to basis function is one- to-one and invertible. The response of a voxel is expressed as the weighted sum of basis functions. In this implementation, basis functions were half-wave rectified sinusoid functions raised to a power set by the user (e.g. 6).

The model: Inverted encoding models reconstruct a stimulus feature from patterns of BOLD activity by relating the activity in each voxel, B, to the values of hypothetical channels (or basis functions), C, according to Equation 1 below.

B = W*C

where W is a weight matrix that represents the relationship between BOLD activity and Channels. W must be estimated from training data; this implementation (and most described in the literature) uses linear regression to estimate W as in Equation 2 below [note: inv() represents matrix inverse or pseudo-inverse].

W_est = B_train*inv(C_train)

The weights in W_est (short for “estimated”) represent the contributions of each channel to the response of each voxel. Estimated channel responses can be computed given W_est and new voxel activity represented in matrix B_exp (short for “experiment”) through inversion of Equation 1:

C_est = inv(W_est)*B_exp

Given estimated channel responses, C_est, it is straightforward to obtain the reconstructed feature value by summing over channels multiplied by their channel responses and taking the argmax (i.e. the feature associated with the maximum value).

Using this model: Use fit() to estimate the weights of the basis functions given input data (e.g. beta values from fMRI data). This function will execute equation 2 above.

Use predict() to compute predicted stimulus values from new functional data. This function computes estimated channel responses, as in equation 3, then computes summed channel output and finds the argmax (within the stimulus feature space) associated with those responses.

Use score() to compute a measure of the error of the prediction based on known stimuli.

This implementation assumes a circular (or half- circular) feature domain. Future implementations might generalize the feature input space, and increase the possible dimensionality.

Parameters

n_channels (int, default 5. Number of channels) – The number of channels, or basis functions, to be used in the inverted encoding model.
channel_exp (int, default 6. Basis function exponent.) – The exponent of the sinuoidal basis functions, which establishes the width of the functions.
stimulus_mode (str, default 'halfcircular' (other option is) – ‘circular’). Describes the feature domain.
range_start (double, default 0. Lowest value of domain.) – Beginning value of range of independent variable (usually degrees).
range_stop (double, default 180. Highest value of domain.) – Ending value of range of independent variable (usually degrees).
channel_density (int, default 180. Number of points in the) – feature domain.
stimulus_resolution (double, default None will set the stimulus) – resolution to be identical to the channel density. This sets the resolution at which the stimuli were presented (e.g. a spatial position with some width has a lower stimulus resolution).

channels_¶

matrix defining channel values

Type: [n_channels, channel density] NumPy 2D array

W_¶

relates estimated channel responses to response amplitude data

Type: sklearn.linear_model model containing weight matrix that

See get_params() for the rest of the attributes.

fit(X, y)¶

Use data and feature variable labels to fit an IEM

Parameters

X (numpy matrix of voxel activation data. [observations, voxels]) – Should contain the beta values for each observation or trial and each voxel of training data.
y (numpy array of response variable. [observations]) – Should contain the feature for each observation in X.

get_params()¶

Returns model parameters.

Returns: params
Return type: parameter of this object

predict(X)¶

Use test data to predict the feature

Parameters

X (numpy matrix of voxel activation from test trials) –
[observations –
Used to predict feature (voxels]) –
with the given observation. (associated) –

Returns

model_prediction

Return type

numpy array of estimated feature values.

score(X, y)¶

Calculate error measure of prediction. Default measurement is R^2, the coefficient of determination.

Parameters

X (numpy matrix of voxel activation from new data) – [observations,voxels]
y (numpy array of responses. [observations]) –

Returns

score_value – feature and predicted features.

Return type

the error measurement between the actual

set_params(**parameters)¶

Sets model parameters after initialization.

Parameters: parameters (structure with parameters and change values) –

class brainiak.reconstruct.iem.InvertedEncoding2D(stim_xlim, stim_ylim, stimulus_resolution, stim_radius=None, chan_xlim=None, chan_ylim=None, channels=None, channel_exp=7)¶

Bases: sklearn.base.BaseEstimator

Basis function-based reconstruction method

Inverted encoding models (alternatively known as forward models) are used to reconstruct a feature represented in a N-dimensional space, here 2D, (e.g. position on a projector screen) from patterns across voxels in functional data. The model uses some number of idealized basis functions that cover the 2D space, and assumes that the transformation from stimulus feature (e.g. 2D spatial position) to basis function is one- to-one and invertible. The response of a voxel is expressed as the weighted sum of basis functions. In this implementation, basis functions were half-wave rectified sinusoid functions raised to some power (set by the user).

The documentation will refer to the ‘stimulus space’ or ‘stimulus domain’, which should be a 2D space in consistent units (e.g. screen pixels, or degrees visual angle). The stimulus space is the domain in which the stimulus is reconstructed. We will refer to the each point in this 2D stimulus domain as a ‘pixel’.

The model: Inverted encoding models reconstruct a stimulus feature from patterns of BOLD activity by relating the activity in each voxel, B, to the values of hypothetical channels (or basis functions), C, according to Equation 1 below.

B = W*C

where W is a weight matrix that represents the relationship between BOLD activity and Channels. W must be estimated from training data; this implementation (and most described in the literature) uses linear regression to estimate W as in Equation 2 below [note: inv() represents matrix inverse or pseudo-inverse].

W_est = B_train*inv(C_train)

The weights in W_est (short for “estimated”) represent the contributions of each channel to the response of each voxel. Estimated channel responses can be computed given W_est and new voxel activity represented in matrix B_exp (short for “experiment”) through inversion of Equation 1:

C_est = inv(W_est)*B_exp

Given estimated channel responses, C_est, it is straightforward to obtain the reconstructed feature value by summing over channels multiplied by their channel responses and taking the argmax (i.e. the feature associated with the maximum value).

Using this model: Use fit() to estimate the weights of the basis functions given input data (e.g. beta values from fMRI data). This function will execute equation 2 above.

Use predict() to compute predicted stimulus values from new functional data. This function computes estimated channel responses, as in equation 3, then computes summed channel output and finds the argmax (within the stimulus feature space) associated with those responses.

Use score() to compute a measure of the error of the prediction based on known stimuli.

Parameters

stim_xlim (list of 2 floats Specifies the minimum and maximum x-values) – of the area to be reconstructed. In order to be estimated properly, a stimulus must appear at these limits. Specifying limits outside the range of the stimuli can lead to spurious estimates.
stim_ylim (list of 2 floats Specifies the minimum and maximum y-values) – of the area to be reconstructed. In order to be estimated properly, a stimulus must appear at these limits. Specifying limits outside the range of the stimuli can lead to spurious estimates.
stimulus_resolution (float or list of 2 floats. If a single float is) – given, it will be expanded to a list (i.e. we will assume that the reconstructed area is composed of square pixels).
stim_radius (float, or sequence of floats [n_stim], default None. If the) – user does not define the design matrix of the encoding model (e.g. C in B = W*C), it will be defined automatically on the assumption that each observation was for a 2D circular stimulus of some radius.
chan_xlim (list of 2 floats, default None. Specifies the minimum and) – maximum x-values of the channels, or basis functions.
chan_ylim (list of 2 floats, default None. Specifies the minimum and) – maximum y-values of the channels, or basis functions.
channels ([n_channels, n_pixels] NumPy 2D array, default None. If None at) – initialization, it can be defined with either define_basis_functions_sqgrid() or define_basis_functions_trigrid(), each of which tiles the given 2D space with some grid (square or triangular/hexagonal, respectively). Alternatively, the user can specify their own channels.
channel_exp (int, default 7. Basis function exponent. The exponent of the) – sinuoidal basis functions, which helps control their width.

channels¶

Type: [n_channels, n_pixels] NumPy 2D array defining channels

W_¶

channel responses to response data

Type: sklearn.linear_model containing weight matrix that relates estimated

See get_params() for the rest of the attributes.

define_basis_functions_sqgrid(nchannels, channel_size=None)¶

Define basis functions (aka channels) arrange in a square grid. Sets the self.channels parameter.

Parameters

nchannels (number of channels in the x (horizontal) direction) –
channel_size (the desired full-width half-maximum (FWHM) of the) – channel, in stimulus space.

Returns

self.channels (defines channels, a [nchannels, npixels] matrix.)
channel_centers (numpy array of the centers of each channel, given as) – [nchannels x 2] matrix

define_basis_functions_trigrid(grid_radius, channel_size=None)¶

Define basis functions (aka channels) arranged in a triangular grid.

Returns

self.channels (defines channels, [nchannels, npixels] matrix.)
channel_centers (numpy array of the centers of each channel)

fit(X, y, C=None)¶

Use data and feature variable labels to fit an IEM

Parameters

X (numpy matrix of voxel activation data. [observations, voxels]) – Should contain the beta values for each observation or trial and each voxel of training data.
y (numpy array of response variable. [observations]) – Should contain the feature for each observation in X.
C (numpy matrix of channel activations for every observation (e.g.) – the design matrix C in the linear equation B = W*C), matrix size [observations, pixels]. If None (default), this assumes that each observation contains a 2D circular stimulus and will define the activations with self._define_trial_activations(y).

get_params()¶

Returns model parameters.

Returns: params
Return type: parameter of this object

predict(X)¶

Use test data to predict the feature

Parameters: X (numpy matrix of voxel activation from test trials [observations,) – voxels]. Used to predict feature associated with the given observation.
Returns: model_prediction
Return type: numpy array of estimated feature values.

predict_feature_responses(X)¶

Takes channel weights and transforms them into continuous functions defined in the feature domain.

Parameters: X (numpy matrix of data. [observations, voxels]) –
Returns: pred_response – reconstruction in the channel domain. [pixels, observations]
Return type: predict response from all channels. This is the stimulus

score(X, y)¶

Calculate error measure of prediction, assuming that the predicted feature is at the maximum of the reconstructed values.

To score the reconstructions against expected features defined in the stimulus domain (i.e. in pixels), see score_against_reconstructed().

Parameters

X (numpy matrix of voxel activation from new data) – [observations,voxels]
y (numpy array of stimulus features. [observations, 2]) –

Returns

score_value – feature and predicted features, [observations].

Return type

the error measurement between the actual

score_against_reconstructed(X, y, metric='euclidean')¶

Calculates a distance metric between reconstructed features in the 2D stimulus domain (i.e. reconstructions in pixels) given some observations X, and expected features y. Expected features must also be in the pixel stimulus domain.

To score the reconstructions against the expected maxima, see score().

Parameters

X (numpy matrix of voxel activation from new data) – [observations, voxels]
y (numpy array of the expected stimulus reconstruction values [pixels,) – observations].
metric (string specifying the distance metric, either "euclidean" or) – “cosine”.

Returns

score_value – values as the expected values, [observations].

Return type

the error measurement between the reconstructed feature

set_params(**parameters)¶

Sets model parameters after initialization.

Parameters: parameters (structure with parameters and change values) –