Skip to content

3-D ROC Evaluation Criteria & Point Allocation

Basis Upon Which Submissions Will Be Evaluated

Solutions must show publication-quality data demonstrating:

  • A 3-D human retina organoid system that mimics the physiological and morphological features of the in vivo biology, consists of the major retina cell types (rod and cone photoreceptors, horizontal, bipolar, amacrine, and ganglion cells and Muller glia) with appropriate lamination and synaptic organization, and represents their biological functions and interplay. Components (neurons, retinal pigment epithelium [RPE], glia) may be produced separately or dissociated and recombined (1) if protocol addresses a significant biological or technical hurdle and (2) if in the process of re-assembly, specific functions/roles of cell types are delineated. Three-dimensional assembly may be achieved using various approaches, for example through self-organization that recapitulates natural development or through bioengineering with scaffolds, bioprinting, and/or microfluidic apparatuses.
  • Retina organoids that are generated entirely from human cells (e.g. derived from iPSCs, hESCs, multipotent cells, or adult cells subjected to a combination of transdifferentiation and/or reprogramming methods).
  • A 3D human retina organoid system that either 1) can be used for modeling and treating retinal disease; or 2) can be used for testing and developing drug (i.e., high content screening) therapies (for details, see Evaluation Criteria 4 below).

Explants are not of interest for this Challenge. Tissue-on-a-chip systems that use cells grown in 2-D co-culture and do not fully represent the structure, morphology, and function of the human retina are also not of interest. However, creative approaches that incorporate use of microfluidics or perfusion to enhance culture or extend duration of survival for 3-D organoid systems are acceptable.

In general to the four review categories, reviewers will be asked to focus on the following areas. Each submission can earn a total of 100 points.  

  • Significant advances over currently available protocols in areas such as duration of culture, yield, and maturity/differentiation of all cell types in appropriate numbers and ratio.
  • Potential impact on understanding the biology of the retina.

Evaluation Criterion 1 - Impact and Innovation (20 points)

Submissions will be evaluated according to the impact it has on potential end users, clinical implications, and therapeutic advances.  Solutions presented should have a sustained, powerful influence on the understanding of retinal diseases and accelerating research toward new therapies.  Data should demonstrate feasible methods and technology to generate retina organoids with maximum relevance to human physiology and disease.  Submissions should address significant physiological barriers to the field (e.g. vasculature, additional cell types, fovea, etc.). Solutions should overcome significant technological barriers of human retina organoid development (e.g. speed up organoid differentiation/maturation, improve yield, or increase reproducibility). The premise and data of the submission should be thoroughly validated.  Innovative approaches that are novel, groundbreaking, or paradigm-shifting, or demonstrate a creative application of existing approaches are sought. Solutions will be awarded full points only if the following elements in their retina organoids are robustly demonstrated:

  • The approach significantly advances current retina organoid generation protocols. (4 points)
  • The solution will have a lasting impact on how retinal diseases and how treatments are studied. (4 points)
  • Current substantial physiological barriers to the field are shown to be overcome in significant ways.  (4 points)
  • Existing technological barriers of in vitro human retina organoid development are addressed.  (4 points)
  • The submitted data has been well validated in a robust manner.  (4 points)

Evaluation Criterion 2 - Cell Types, Structure, and Function (25 points)

Solutions will be evaluated for establishment of a human PSC-derived in vitro retina model system that resembles the morphology of a healthy-native retina and is viable through formation of photoreceptor outer segments and/or long-term survival of retinal ganglion cells with extension of axonal processes. Characterization of retinal cell types, retina organoid structure, and retina organoid function are expected. Solutions will be need to address the following for full points:

Cell Types:  9 points total

  • Demonstrates all major retinal neuron cell types: Rod and photoreceptor (at least 2 types of cones), horizontal cells, amacrine cells (at least 5 types), bipolar cells (representatives in 3 major groups-rod bipolar; ON-Cone; Off-cone), and Retinal Ganglion Cells (identify at least three subtypes).  (3 points)
  • Data shows a majority of non-neuronal cell types are present including astrocytes, Muller glia, RPE cells, pericytes, endothelial cells, and microglia. (3 points)
  • Analysis showing that cell types and cell populations reflect the proportions of what is seen in the human eye using robust methods including gene expression (e.g. RNAseq), immunohistochemistry or flow cytometry, and/or histology.  (3 points)

Structure: 8 points total

  • The retina organoids have foveae that are highly representative of human anatomy (2 points)
  • The retina organoids have retinal and choroidal vasculature or is recapitulated with microfluidics or accounted for by other perfusion techniques. (2 points)
  • A 5 layered neural retina (3 cell layers, 2 synaptic layers) has been validated in a robust manner and the RPE lamina is morphologically well-defined. (2 points)
  • Ribbon synapses and optic nerve/retinal nerve fiber layer are definitively shown (2 points)

Function: 8 points total

  • Electrophysiology of light perception demonstrated. Photoreceptor signal transduction/phototransduction is robustly evidenced, functional synapses at inner and outer plexiform layer are shown, RGC action potentials in response to light stimuli are exhibited. (4 points)
  • Characterization of distinct On- and Off- pathways.  (4 points)

 Do you want all of these components? Or are some OR all of the components appropriate?

Evaluation Criterion 3 - Reproducibility, Quality Control, and Standardization (25 points)

Reproducibility and robustness, standardization and quality control, and the commercialization strategy will be evaluated with regards to the following criteria:

  • Demonstration of inter-laboratory consistency of the protocol and its applicability to multiple cell lines, will be shown with the statistical rigor of the presented data (e.g. error bars) highlighted. The protocol will be sufficiently detailed to ensure its reproducibility.  (5 points)
  • The protocol’s robustness in terms of organoid production and yield rate and the ability to decrease the differentiation time, etc. is well explained and justified in the context of the prescribed utility of the organoid. (5 points)
  • A standardization strategy describing the scalability of the protocol to a large production, as well as a streamlined assay platform for higher throughput, is well explained and aligned with the proposed commercialized utility of the organoid. (5 points)
  • The methods and criteria employed to ensure proper quality control are clearly explained (e.g. free of microbial contamination and chromosomal defects via karyotyping, STR analysis, FISH analysis).  (5 points)
  • A commercialization strategy including details of target users, a market analysis, and a comparative analysis with competing technologies. Filed patents should be listed. Discussions with FDA, third-party commercialization/entrepreneurship consultant, or enrollment in commercialization training programs should be documented. Any agreements with industry, or transferability to industry partners (MOUs, licensing, CRADA, in-kind or discounted validation) should be presented. (5 points)

Evaluation Criterion 4 - Endpoint Assay Specific Goals (30 points for EITHER category)

BIOLOGY/DISEASE MODELING (note: NEI is agnostic to which disease is chosen, as long as technology is developed to robustly recapitulate the disease)

The reproduction of morphological hallmarks of disease will be evaluated using the following criteria:

  • Evidence that the model recapitulates the disease of study; i.e.  distinctive and disease-relevant anatomical, biochemical and/or morphological features. Demonstration of good concordance between organoid disease models and diseased human tissues at different time points/stages of disease. Includes a comparison of organoid morphology, anatomy, and/or biochemistry in both the healthy normal retina and disease-modeling retinal organoid. (15 points)

The phenotypic measurements in endpoint assays will be evaluated by:

  • Demonstration of functional assays that show functional deficits in the in vitro disease model as compared to normal retina organoids is robust and validated. Comparison with other established disease models or datasets are made. (15 points)


Screening scalability, its validated use with various iPSC and ESC cell lines, and transferability of the screening method will be evaluated using the following criteria:

  • Data robustly shows retina organoid screening protocols are amendable to high content screening, which may include high content imaging, drug validation/toxicology, or functional genomic screening (e.g. does not include materials known to show strong compound absorption). Methods to mass-produce (e.g., 100s) organoids are presented. Multiple replicates of plates are compared in the same experiment to show variability is minimal. (5 points)
  • Data shows minimal variation between different stem cell lines under screening conditions. Morphological and functional features are maintained so readouts are easy to normalize. (5 points)
  • Effects of shipping or freezing/thawing on screening methods are shown.  Viability of cells and transferability of screening protocols to other labs or companies shown to be robust and reproducible. (5 points)

Drug screening data should be provided.

  • Data robustly show the retina organoids recapitulate known retina toxicities or known drug effects based on morphological and functional readouts. (15 points)

Last updated: January 29, 2021