The Quantitative Imaging Network The National Cancer Institute Then: 1939 And Now: 2016

Eureka! Clinical Analytics
Andrew Post, MD, PhD,¹ Ashish Sharma, PhD,¹ Fred Prior, PhD^2
1Emory University, Atlanta, GA, ²Univ. of Arkansas Medical Center, Little Rock, AR
Introduction: Leveraging clinical data for cancer research continues to be hampered by a lack of interoperable tools for access to and management of integrated clinical and imaging data for correlation and prediction.
Methods: Eureka! Clinical Analytics is a web-based system that supports extracting diverse research and clinical data from files and databases, transforming its structure and semantics into a common data model, and loading the transformed data into the i2b2 data warehousing system. It supports specifying and computing phenotypes during the loading process as frequency, sequence and threshold patterns in clinical and research data. Integration of Eureka with the i2b2 software supports the use of i2b2’s web client for query and access to the resulting data and phenotypes through a web browser.
Results: We have prototyped an extension of Eureka that supports offering i2b2 projects as a web-based service. This involves specifying a common data model, creating data adapters to extract source clinical data and imaging metadata (e.g., web links to image repositories) from files and databases, configuring Eureka to map the source data into the common model, rapidly creating new i2b2 data marts and registering them within an i2b2 deployment, integration with enterprise authentication systems, and appropriate user authorization to projects and data.

We have successfully tested this architecture by creating an i2b2 project containing the clinical data from the National Lung Screening Trial (NLST). In addition to i2b2’s built-in support for specifying inclusion and exclusion criteria and querying for patient counts, customizing the i2b2 web client through its plugin architecture could provide authorized views of query results that could range from summary statistics to participant-level data.

Conclusions: This architecture supports the implementation of a flexible web-based service for accessing diverse clinical datasets in a standardized form for research through a common browser-based interface. It also could support interoperability with online image repositories through web links and RESTful programming interfaces. We expect it could form the basis for services that allow flexible query of datasets created and used by QIN investigators.

No-gold-standard evaluation of quantitative imaging biomarker quantification methods
Abhinav K. Jha, Eric Frey
Johns Hopkins University
Objective: Several quantitative imaging network (QIN) groups are developing new and improved imaging methods for reliable measurement of quantitative metrics from patient images for subsequent evaluation as imaging biomarkers. For clinical translation, these imaging methods need to be evaluated on the clinical task of reliably estimating the metrics. Ideally this evaluation must be performed with patient data, but the unreliability or unavailability of a gold standard for most patient studies makes such evaluation impractical or impossible. To overcome this issue, there is an important need for techniques to objectively evaluate imaging methods in the absence of a gold standard.
Methods: A no-gold-standard (NGS) technique has been developed that, in the absence of any gold-standard data, estimates a figure of merit that can evaluate the quantitative imaging methods based on how precisely they measure the true quantitative values. The NGS technique has been validated using numerical experiments and realistic simulations conducted in the context of evaluating reconstruction methods for quantitative SPECT imaging¹ and tumor-segmentation methods for diffusion MRI². Further, methods have been developed to overcome practical difficulties in applying the NGS technique to patient data. The resulting NGS framework has been applied to evaluate tumor-segmentation methods for PET imaging with oncologic patient data on the task of estimating the metabolic tumor volume (MTV)³.
Results: Validation with realistic simulation studies conducted in the context of quantitative SPECT showed that the NGS technique accurately evaluated the imaging methods on the task of estimating the tracer uptake in a known region of interest more than 97.5% of the noise realizations. Further, application of the NGS framework to oncologic PET imaging data demonstrated that when MTV measurements from more than 80 lesions were available, the NGS technique provided consistent results.
Conclusions: Results from several studies provide strong evidence supporting the potential of the NGS framework to evaluate methods for imaging-biomarker quantification. The developed NGS framework could have an important role in several imaging-method-evaluation-related activities of the quantitative imaging network, such as the various QIN challenges.

Acknowledgements: This work was financially supported by National Institute of Health grant U01 CA140204.


References:

1. Jha AK, Caffo B, Frey EC. A no-gold-standard technique for objective assessment of quantitative nuclear-medicine imaging methods. Phys Med Biol 2016;61(7):2780-800.

2. Jha AK, Kupinski MA, Rodriguez JJ, Stephen RM, Stopeck AT. Task-based evaluation of segmentation algorithms for diffusion-weighted MRI without using a gold standard. Physics in Medicine and Biology 2013;58(1):183-183.

3. Jha AK, Mena E, Caffo B, Ashrafinia S, Rahmim A and others. Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography. Journal of Medical Imaging 2017; 4(1):011011-011011.

Quantitative Imaging to Assess Response in Cancer Therapy Trials

John M Buatti MD, Thomas L. Casavant PhD, Michael M. Graham PhD MD, Milan Sonka PhD, Vincent Magnotta, PhD, John J Sunderland PhD,

Brian J Smith PhD, Reinhard Beichel PhD
University of Iowa

The University of Iowa QIN team has been consistently committed to improving and developing tools for quantitative image analysis both for assessment of response and for tumor targeting.

Specific Aim 1: Develop a novel, robust imaging genomics-based decision support platform using a combination of our successful Phase-I developed and validated highly automated quantitative image analysis methods applied to linked and publicly-available well curated image (TCIA) and molecular (The Cancer Genome Atlas–TCGA) data warehouses along with an established outcomes database for H&N cancers.

During this past year, our efforts have been focused on refinement and enhancement of our genomic variant analysis pipeline necessary to identify highly-informative features for prediction and decision support. This novel informatics pipeline utilizes currently-available TCGA data for H&N cancers for which TCIA data is also available.

Specific Aims 2 and 3: Build and innovate based on Phase-I developed and validated image analysis tools and create a novel link between our established work in PET quantification and calibration phantoms with our image analysis and decision support tools to create a clinically practical open source automated phantom analysis tool that can be applied to national efforts aimed to improve quantitative imaging quality assurance for clinical trials.
We have updated and improved our publicly released open-source software for quantitative PET image analysis, consisting of 3D Slicer PET Tumor Segmentation, 3D Slicer PET DICOM Extension, PET Liver Uptake Measurement, and 3D Slicer PET-IndiC Extension as well as supporting libraries. To better document the released software and facilitate the broad dissemination, we have published a summary paper, which describes details of the implemented lesion segmentation algorithm as well as its validation [1]. We have also established a website: http://qin.iibi.uiowa.edu providing instructional videos and demonstrations that are fully narrated for public use. We are hopeful that this will facilitate a broad utilization of our developed tools.
We have developed a fully-automated quantitative PET phantom analysis algorithm, which allows the user to segment ACR/ACRIN-ECOG, SNMMI/CTN, and NEMA NU-2 image quality phantoms and will help to simplify the process of PET scan image quality assessment.

Development activities for an FLT based tool for head and neck cancer as well as for DOTATOC tumor burden assessment in the liver are ongoing.

Specific Aim 4: Adapt, enhance and extend quantitative image-based response assessment in clinical trial decision-support through relevant active clinical trials.

We are pursuing imaging methods to assess tumor response to pharmacological ascorbate as an adjuvant to standard of care therapy. We have pursued the development of MR based methods capable of directly quantitating ascorbate within the tumor and measuring markers attributable to the reduction of labile iron by ascorbate. Since high concentrations of ascorbate have been shown in vitro to reduce labile Iron(III) to Iron(II), we have pursued the evaluation and reliability of T₂* relaxation times and tissue susceptibility to detect subtle changes in the net iron oxidation state that may occur after ascorbate infusion.
1. Beichel RR, Van Tol M, Ulrich EJ, Bauer C, Chang T, Plichta KA, Smith BJ, Sunderland JJ, Graham MM, Sonka M, Buatti JM. Semiautomated segmentation of head and neck cancers in 18F-FDG PET scans: A just-enough-interaction approach. Med Phys. Jun 2016;43(6):2948. PMID: 27277044. PMCID: 4874930.

Multiparametric and Multimodality Quantitative Imaging
for Evaluation of Response to Cancer Therapy
¹Johns Hopkins University, Department of Radiology and Radiological Science

²Washington University in St. Louis, Mallinckrodt Institute of Radiology
E.C. Frey¹*, M.A. Jacobs¹ and R.L. Wahl²

* Corresponding PI
The underlying hypothesis of this project is that combining multiple quantitative image-derived parameters, whether different quantities from the same modality, multiple modalities or multiple tracers, can provide a more robust prediction and assessment of treatment response than a single imaging metric. Substantial effort has been focused on developing single-modality metrics and assessing and reducing their variability. Modalities investigated include DWI and ADC MRI, FDG and FLT PET/CT , In-111 octreotide, Tc-99m MDP and Y-90 and SPECT/CT. In MRI we have studied the stability and reproducibility of DWI and ADC mapping. In PET/CT we have developed investigated the variability of SUV_max and SUV_peak and proposed an index of defect heterogeneity. We have previously evaluated sources of variability in SPECT/CT including scanner, patient, and quantum noise. We have evaluated test-retest variability in FLT PET/CT. We have accrued 10 patients in a test-retest study involving FDG and repeat imaging of patients on PET/CT and PET/MR scanners. These data should inform our ability to define multiparametric quantitative imaging phenotypes. An important issue in developing quantitative imaging methods or protocols in general is to chose the best method or protocol among several alternatives. Often, the true value of the underlying quantitative parameter is not known. We have developed no-gold-standard methods for ranking such quantitative methods and protocols that do not require knowledge of truth. A paper has been recently published on the use of these methods to optimize segmentation methods used for estimating metabolic tumor volume in FDG PET. Integrating multiparametric and multimodality imaging data is an important part of this project. We have developed a new Radiomics and Informatics method termed IRIS. The method integrates clinical and imaging parameters from different modalities to provide surrogate phenotyping and is an important step in tailoring treatment for the individual patient. The IRIS method has been validated in a subset of patients who were ER+ and candidates for the OncotypeDX gene array test. These initial studies provide insight into the molecular underpinning of the surrogate radiologic features and provide the foundation to relate these changes to the OncotypeDX score. These methods provide a basis for a comprehensive interrogation of the relationship and interactions between imaging parameters, clinical variables, and the tumor environment in the clinical arena using an Radiomics/informatics model. Finally, we have developed methods for quantifying Tc-99m MDP bone SPECT. We have applied our radiomics methods to SPECT/CT scans to provide optimal combination of the multimodality data for the purposes of segmentation of tumors and bone. We have also developed clustering-based segmentation methods for quantitative bone SPECT that use information from registered CT scans as a prior. These methods are being applied to develop quantitative indices of response in cases, such as advanced prostate cancer, where assessing response of boney lesions is an important issue.
Computing, Optimizing, and Evaluating Quantitative Cancer Imaging Biomarkers:

The Stanford Quantitative Imaging Feature Pipeline (QIFP)

Sandy Napel PhD, Daniel L. Rubin MD MS, Dev Gude BS, Sheryl John MS

Stanford University

Introduction: Many QIN and other researchers are developing software to compute radiomics features (e.g., shape, texture, edge sharpness and histogram) from within segmented regions of 2D and 3D images. It is now important to discover the best quantitative imaging features for constructing cancer imaging biomarkers that predict clinical variables, e.g., response to new therapeutics, cancer subtype, mutation status, and cancer genomics. Challenges to this effort are (1) multiple algorithms purportedly, but not actually, computing identical features, (2) multiple algorithms purportedly computing different features, but actually computing the same ones, (3) lack of resources and architectures required to compute, compare, evaluate, and disseminate these quantitative imaging features within the QIN and the broader community, and (4) lack of resources to easily build and compare predictive models linking quantitative imaging features to clinical variables. The Stanford Quantitative Imaging Feature Pipeline (QIFP) will give researchers these capabilities for characterizing images of tumors and surrounding tissues for use in multi-center clinical trials and patient monitoring in general. It will also allow researchers to add their own algorithms for computing and comparing novel quantitative image features, and for building predictive models using these features for their own studies, and for the benefit of the community.

Methods: The QIFP will embody the following key attributes: (1) a web-based, graphical user interface for development of configurable quantitative image feature processing pipelines that will enable researchers to explore combinations of quantitative imaging features, (2) an expandable and sharable library of quantitative image features algorithms, (3) support for a variety of languages for quantitative image feature algorithms, e.g., Matlab, Java, and C/C++, via Docker containers, (4) connectivity to images and other data stored in (i) the Cancer Imaging Archive (TCIA), (ii) ePAD systems (another QIN project for image annotation/curation), (iii) local data stores and, (iv) PACS systems via DICOM, (5) a cloud-based cache of data and software, (6) machine learning algorithms that permit researchers to efficiently establish how well an imaging biomarker built from quantitative image features predicts a clinical or molecular variable, and (7) a user interface for constructing workflows and applying them to cohorts.

Results: The QIFP is currently available by invitation at https://qifp.stanford.edu, with limited functionality: (1) Connection to TCIA, PACS, ePAD, and desktop for cohort submission, (2) pre-configured and user-configurable pipeline construction for computing and linking quantitative and semantic features with clinical data to a clinical variable to be predicted, (3) algorithms for computing 3D quantitative features (shape, texture, histogram, and edge-sharpness, SIFT), (4) a LASSO module for building and testing predictive models based on image features (quantitative and/or semantic). In the near future, the QIFP will contain additional algorithms (e.g., 2D feature computation, support vector machines for predictive model building and testing), and support user upload of algorithms in Docker containers for use in workflow construction and execution.

Conclusion: The QIFP is a flexible ecosystem that supports standardizing and comparing algorithms for feature computation and construction of predictive models. It currently has a selection of feature computation and predictive model-building modules, and is expandable to support an unlimited variety of user-developed software modules for use in constructing and executing workflows for building and evaluating quantitative imaging biomarkers.

Qualification and Deployment of Imaging Biomarkers of Cancer Treatment Response
Daniel Rubin, Richard Abramson, Tom Yankeelov, Mia Levy,

Mark Rosen, and Sandy Napel
Stanford University, Vanderbilt University, University of Texas at Austin,

and ECOG ACRIN Cooperative Group
As cancer treatments being evaluated in clinical trials evolve from cytotoxic agents to targeted therapies, there is a pressing need to incorporate new imaging biomarkers, such as those being developed by centers in the Quantitative Imaging Network (QIN), into these trials in order to detect treatment response with better accuracy than current, simple linear measure-based assessments of cancer. Progress has been thwarted, however, by three major challenges: (1) inability of current image assessment tools to compute new imaging biomarkers, due to their closed architectures and lack of support of different programming languages in which biomarker algorithms are developed, (2) lack of decision support tools to assess treatment response in patients or drug effectiveness in clinical trial cohorts using new imaging biomarkers, and (3) lack of approaches to repurpose the vast collections of image data acquired in clinical trials to acquire evidence for qualifying new imaging biomarkers as surrogate endpoints. We are developing a software platform to enable translating novel quantitative imaging biomarkers being developed by the QIN and others into clinical trials, and methods to enable qualifying them. We will evaluate the success of our platform by deploying new imaging biomarkers in two clinical trials in individual sites and in the ECOG-ACRIN cooperative group. To accomplish these goals, in this project (1) We are developing a platform and tools through which to deploy new imaging biomarkers into clinical trials, extending our previously developed Web-based image viewing tool and developing four unique capabilities: a plugin mechanism to execute new quantitative imaging algorithms developed by us or by others in different programming languages, decision support tools for evaluating patient response and treatment effectiveness, and tools that facilitate the workflow of collecting novel imaging biomarkers in clinical trials, that evaluate their benefit over conventional biomarkers, and that collect data which, across clinical trials, will help to qualify them as surrogate endpoints; (2) We are developing methods to repurpose existing imaging data from clinical trials for studying new imaging biomarkers by developing automated image segmentation methods to enable efficient calculation of novel quantitative imaging biomarkers; and (3) We are deploying and evaluating our platform and tools in two cancer centers and the ECOG-ACRIN national cooperative group. We will demonstrate the ability to efficiently collect image biomarker data and to facilitate the qualification of new imaging biomarkers. Through the public availability of our platform, its plugin mechanism for introducing new quantitative imaging biomarkers in clinical trials, the intuitive graphical user interfaces for collecting these biomarkers in the image interpretation workflow, the methods for de-centralized coordination and oversight of image interpretation in clinical trials, and the tools for decision support, our developments will serve the needs of the QIN and the broader research community, ultimately accelerating clinical trials and the translation of novel image surrogate biomarkers into clinical practice, which will improve the assessment of patient response to new cancer treatments.

ECOG-ACRIN QIN Resource for Advancing

Quantitative Cancer Imaging in Clinical Trials
Mitchell D. Schnall, David Mankoff, Paul E. Kinahan, Mark Rosen
ECOG-ACRIN
The goal of this project is to accelerate the development and deployment of quantitative imaging methods that improve the effectiveness and efficiency of clinical trials by using the combined resources of the ECOG-ACRIN cooperative group and the QIN.

Aim 1: Results from the Centers of Quantitative Imaging Excellence (CQIE) Program were analyzed and published [1.2] demonstrating the challenges in quality control standards for quantitative imaging in clinical research trials. The CQIE database was used to develop a database of QIN sites that being used to create a site profile in the Qualification Utility for Imaging Clinical Trials (QUIC) of qualified QIN sites. This is intended to be dynamic as QIN sites change we will update the QUIC dataset. This will form the basis of later studies in quality control standards within the QIN.
Aim 2: We established a prioritized list of completed imaging trials with datasets that QIN sites felt were best positioned to support QIN development needs. We are now collaborating with NCI to develop standard guidelines and workflows for transferring the datasets to TCIA, resulting in and initial list of 4 trial datasets (with others to follow). In parallel, EA QIN leveraged the ACR’s commitment to development of a workflow involving anonymization methods which ensure compliance with safe harbor regulations and HIPAA standards. In addition to the datasets transferred to TCIA, fully anonymized datasets for these trials and for other ACRIN legacy trials will be made accessible to QIN and other researchers. The intent it to reduce the burden associated with transfers of large clinical trial datasets.
Aim 3: Working with NCI QIN leadership, we organized a one-day planning meeting (12/13/16) that brought together thought leaders from the NCTN together with QIN members and other stakeholders. Leading oncologists from different NCTN groups shared their perspectives on the value of quantitative imaging to therapy, while QIN representatives shared their vision and perspective on how quantitative imaging may be able to benefit oncologists. A list of QI tools currently offered by QIN sites was provided. Breakout sessions focused on 4 areas: Response assessment, quantitative imaging biomarkers, informatics and precision medicine, and image/data curation and archiving. The planning meeting identified several opportunities to incorporate prospective testing of quantitative imaging tools in national level clinical trials that will be further developed.
[1] Rosen M, et al. Academic Radiology. vol.24(2):232–245, 2017.

[2] Scheuermann JS, et al. Journal of Nuclear Medicine (accepted) 2017.

Working Groups

Abstracts