Implementing a Real-Time fMRI Cloud-Based Framework

Overview

This notebook walks through an example using our cloud-based software framework for real-time fMRI studies, henceforth referred to as rtcloud framework. We have created a sample script (sample.py) in this notebook directory which builds a two-class classifier on a region of interest (ROI). This notebook will generate synthetic data for training and classification using the BrainIAK fmrisim_real_time_generator function.

We will begin wrapping the sample.py script within the rtcloud projectInterface. In a normal deployment the projectInterface (and sample.py) would then run in the cloud and the scanner images would be sent to the cloud where our sample.py script would build the classifier model and do classification. The projectInterface handles remote file communication for reading the scanner images and providing the classification results. It also has a web-based user interface for viewing and controlling the experiment and changing settings. In this notebook the different process all run on the the computer that jupyter is running on.

In the steps below, we will first start the projectInterface which will watch for dicom files to process. We will then start the synthetic data generator which will generate a new dicom image every 2 seconds. And finally we will start a web browser within this notebook for controlling the experiment. Once you start the web interface you can click the 'Run' button to start the sample.py script watching for and processing images via the projectInterface. (You can also open the web interface on a different tab in your browser, http://localhost:8888.)

Annotated Bibliography

  1. Mennen, A.C., Turk-Browne, N.B., Wallace, G., Seok, D., Jaganjac, A., Stock, J., deBettencourt, M.T., Cohen, J.D., Norman, K.A. & Sheline, Y.I. (2020). Cloud-based fMRI neurofeedback to reduce the negative attentional bias in depression: a proof-of-concept study. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. link Describes the first implementation of the rt-cloud software framework in a closed-loop fMRI study that provided continuous neurofeedback to participants based on a multivariate pattern classifier.

Table of Contents

Before Running This Notebook

Before you are able to run this notebook, you have to complete the installation instructions found in the accompanying instructions README. Also, remember that you have to complete the following steps every time before you are able to run this notebook:

  1. Activate the conda environment for the rtcloud framework:
    conda activate rtcloud
  2. On the command line, create a global variable to the full path for the rtcloud framework repo. You must do this in order to use the functions we have created for the framework. And don't forget the forward slash "/" at the end.
    export RTCLOUD_PATH=/PATH_TO_RTCLOUD/rt-cloud/
    Double check that you did this correctly by typing the following command.
    ECHO $RTCLOUD_PATH
    This should print the full path to the rtcloud framework folder.

Import Necessary Modules and Declare Important Variables

Step 1: Start the ProjectInterface Web Server


IMPORTANT:

You can only run this cell to start the projectInterface web server only once or else you will get the following runtime error:

Web Server already running

If you want to re-start the web server, you have to first SHUT DOWN the kernel. Instructions for doing this can be found here.


Step 2: Start the Synthetic Data Generator

We will be using a BrainIAK function to create synthetic fMRI data. After you run the following cell, you can execute sim_settings in a separate cell to take a look at the different parameter settings. For instance, you will find that the synthetic data is being produced at a rate of 2 seconds per TRs.


IMPORTANT:

You can only run this cell to generate the synthetic data in "real-time" once unless you delete the data that you have already created. Otherwise, all the data (which is found in the /tmp/notebook-simdata folder) will be automatically available when you open the web server, instead of having the File Watcher wait for incoming data one TR at a time.


Step 3: Open the Web Server on the localhost

Here you will open the web-based user interface to view and control the real-time experiment, where a classification analysis will be performed. If you are interested in learning more about the details of a classifier analysis in real-time, take a look at the real-time tutorial on the BrainIAK website. You will also be able to change the settings in the "Settings" tab.

The first time you open the web server, either in the cell below or in a separate browser tab, you will be prompted to enter a username and password. In this demonstration, you will use "test" for both. Once you have logged in you will be on the Run tab. You can start the sample.py script running and waiting for dicom data to process by clicking the 'run' button within the webpage 'Run' tab. Then you will see the output as the script progresses within the view pane of the Run tab.

If you want to re-run the sample.py script after making changes, you will most likely get the following error message: Error: Client thread already runnning, skipping new request. You will have to restart the kernel like in step 1.

Summary

You have now completed running a real-time experiment using a distributed computing pipeline (managed by the projectInterface). The synthetic data was sent to the sample.py script in real-time as it was generated, and the script started by building a classifier and then performing classification.

As a next step you can try modifying or using your own script in place of sample.py, and you can also try running the projectInterface on a cloud VM which eliminates the need for expensive computer hardware within the control room.

We're excited to see what interesting real-time experiments you will create!