This notebook walks through an example using our cloud-based software framework for real-time fMRI studies, henceforth referred to as rtcloud framework. We have created a sample script (sample.py) in this notebook directory which builds a two-class classifier on a region of interest (ROI). This notebook will generate synthetic data for training and classification using the BrainIAK fmrisim_real_time_generator function.
We will begin wrapping the sample.py script within the rtcloud projectInterface. In a normal deployment the projectInterface (and sample.py) would then run in the cloud and the scanner images would be sent to the cloud where our sample.py script would build the classifier model and do classification. The projectInterface handles remote file communication for reading the scanner images and providing the classification results. It also has a web-based user interface for viewing and controlling the experiment and changing settings. In this notebook the different process all run on the the computer that jupyter is running on.
In the steps below, we will first start the projectInterface which will watch for dicom files to process. We will then start the synthetic data generator which will generate a new dicom image every 2 seconds. And finally we will start a web browser within this notebook for controlling the experiment. Once you start the web interface you can click the 'Run' button to start the sample.py script watching for and processing images via the projectInterface. (You can also open the web interface on a different tab in your browser, http://localhost:8888.)
link
Describes the first implementation of the rt-cloud software framework in a closed-loop fMRI study that provided continuous neurofeedback to participants based on a multivariate pattern classifier.Before you are able to run this notebook, you have to complete the installation instructions found in the accompanying instructions README. Also, remember that you have to complete the following steps every time before you are able to run this notebook:
conda activate rtcloud
export RTCLOUD_PATH=/PATH_TO_RTCLOUD/rt-cloud/
Double check that you did this correctly by typing the following command.
ECHO $RTCLOUD_PATH
This should print the full path to the rtcloud framework folder.!echo $RTCLOUD_PATH
import warnings; warnings.simplefilter('ignore')
#---- Import the necessary python modules
import os
import sys
import threading
import argparse
#---- Load important brainiak modules
import brainiak.utils.fmrisim_real_time_generator as sim
#---- Load important rtcloud modules
# add the path to the rtcloud repo to PYTHONPATH to access rtCommon functions
path_to_rtcloud = os.getenv('RTCLOUD_PATH')
if path_to_rtcloud == None:
print("Please set RTCLOUD_PATH, see instructions")
raise ValueError
sys.path.append(path_to_rtcloud)
from rtCommon.projectServer import ProjectServer
from rtCommon.structDict import StructDict
#---- Declare and append important paths
# declare the path to this jupyter notebook
path_to_notebook = os.getcwd() # check and change notebook path as needed
# declare the scripts that will be wrapped by the projectInterface and
# accessed through the webServer
scriptToRun = os.path.join(path_to_notebook, 'sample.py')
initScript = os.path.join(path_to_notebook, 'initialize.py')
finalizeScript = os.path.join(path_to_notebook, 'finalize.py')
#---- Declare the total number of TRs (timepoints) you want to generate, and how many should be training
num_TRs = 200
num_training = 100
#---- Set up the config and parameters for this rtcloud tutorial
# NOTE: you can also change these parameters in the Settings tab on the web server
config = StructDict({
'title' : 'rtCloud Tutorial', # study name
'sessionId' : '20200101T120000', # session ID on the scanner
'subjectName' : '001_synthetic', # subject ID on the scanner
'datestr' : '20200101', # session date
'isSynthetic' : True, # are we using synthetic data?
'numSynthetic' : num_TRs, # total number of synthetic TRs
'numTrainingTRs' : num_training, # number of TRs used for training the classifier
'imgDir' : '/tmp/notebook-simdata', # location of synthetic TRs
'subjectNum' : 101, # subject number
'subjectDay' : 1, # study day (relevant if multi-day study)
'sessionNum' : 1, # session number
'runNum' : [1], # list of scanning runs that were done during the session
'scanNum' : [14], # list of corresponding scan numbers for the runs
# Plotting settings
'plotTitle' : 'Realtime Plot', # plot title
'plotXLabel' : 'TR #', # plot x-axis label
'plotYLabel' : 'Classifier Prediction', # plot y-axis label
'plotXRangeLow' : num_training, # plot x-axis minimum limit
'plotXRangeHigh' : num_TRs, # plot x-axis maximum limit
'plotYRangeLow' : -1, # plot y-axis minimum limit
'plotYRangeHigh' : 1, # plot y-axis maximum limit
'plotAutoRangeX' : False, # do we want the x-axis limit to automatically fit the range?
'plotAutoRangeY' : True, # do we want the x-axis limit to automatically fit the range?
# important variables about the dicom files
'dicomNamePattern' : "rt_{TR:03d}.dcm", # naming pattern for the dicom files
'minExpectedDicomSize' : 200000, # expected size for the dicom data
})
args = argparse.Namespace()
args.projectName = 'sample'
args.projectDir = path_to_notebook
args.mainScript = scriptToRun
args.initScript = initScript
args.finalizeScript = finalizeScript
args.dataRemote = False
args.subjectRemote = False
args.test = True
args.config = config
args.port = 8889
#---- Start the project server
def runProjectServer(args):
projectServer = ProjectServer(args)
projectServer.start()
try:
project_thread = threading.Thread(name='projectServer',
target=runProjectServer,
args=(args,))
project_thread.setDaemon(True)
project_thread.start()
except RuntimeError as err:
# ignore event loop already running error
if str(err) != 'This event loop is already running':
raise
IMPORTANT:
You can only run this cell to start the projectInterface web server only once or else you will get the following runtime error:
Web Server already running
If you want to re-start the web server, you have to first SHUT DOWN the kernel. Instructions for doing this can be found here.
We will be using a BrainIAK function to create synthetic fMRI data. After you run the following cell, you can execute sim_settings
in a separate cell to take a look at the different parameter settings. For instance, you will find that the synthetic data is being produced at a rate of 2 seconds per TRs.
#---- Set up the parameters for the synthetic data generator
sim_settings = sim.default_settings
sim_settings['save_dicom'] = True
sim_settings['save_realtime'] = True
sim_settings['numTRs'] = num_TRs
sim_settings['different_ROIs'] = True
sim_settings['scale_percentage'] = 1
outdir = '/tmp/notebook-simdata'
if not os.path.exists(outdir):
os.makedirs(outdir)
#---- Run sim.generate_data(outdir, sim_settings) as a thread
syndata_thread = threading.Thread(name='syndata',
target=sim.generate_data,
args=(outdir, sim_settings))
syndata_thread.setDaemon(True)
syndata_thread.start()
IMPORTANT:
You can only run this cell to generate the synthetic data in "real-time" once unless you delete the data that you have already created. Otherwise, all the data (which is found in the /tmp/notebook-simdata
folder) will be automatically available when you open the web server, instead of having the File Watcher wait for incoming data one TR at a time.
Here you will open the web-based user interface to view and control the real-time experiment, where a classification analysis will be performed. If you are interested in learning more about the details of a classifier analysis in real-time, take a look at the real-time tutorial on the BrainIAK website. You will also be able to change the settings in the "Settings" tab.
The first time you open the web server, either in the cell below or in a separate browser tab, you will be prompted to enter a username and password. In this demonstration, you will use "test" for both. Once you have logged in you will be on the Run tab. You can start the sample.py script running and waiting for dicom data to process by clicking the 'run' button within the webpage 'Run' tab. Then you will see the output as the script progresses within the view pane of the Run tab.
If you want to re-run the sample.py script after making changes, you will most likely get the following error message: Error: Client thread already runnning, skipping new request
. You will have to restart the kernel like in step 1.
%%html
<iframe src="http://localhost:8889" width="800" height="600"></iframe>
You have now completed running a real-time experiment using a distributed computing pipeline (managed by the projectInterface). The synthetic data was sent to the sample.py script in real-time as it was generated, and the script started by building a classifier and then performing classification.
As a next step you can try modifying or using your own script in place of sample.py, and you can also try running the projectInterface on a cloud VM which eliminates the need for expensive computer hardware within the control room.
We're excited to see what interesting real-time experiments you will create!