The Need for RNAi Screening Standards

A couple of months ago at the CHI Discovery on Target Conference, Hakim Djaballah, Director of the HTS Core Facility at the Memorial Sloan Kettering Cancer Center, gave a unique and insightful presentation highlighting the challenges RNAi screening to identify lethal loss of function interaction in oncogenic systems.

For the exceptional benefits RNAi offers as a targeted tool to elucidate gene function, it is still a relatively new technology with limitations, potential, and idiosyncrasies that remain somewhat undefined.  These features are especially evident when RNAi is adapted for large-scale screening of gene function where small details in the set-up, screening process, quality of the reagents, and types of cells can significantly affect the variation and consistency in the large amount of data generated.  By highlighting some disappointing follow-up results from initially exciting high-profile publications, Dr. Djaballah identified a few critical benchmarks for evaluating RNAi screening.

Dr. Djaballah’s group looked at three potentially high value cancer targets identified in independent loss-of-function RNAi screens.  In addition to the publications, his group reviewed the primary screening data in more detail to evaluate the procedure and statistical significance of the data. The first screen, published in May 2009 in Cell (Scholl, et al.), identified STK33 as required for KRAS oncogenic activity.  Initially, a very exciting discovery, a number of groups pursued STK33 as a potential therapeutic target.  However, subsequent groups (Barbie, et al. and Luo, et al.) failed to find the STK33 among the strong hits in similar screens.  A recent publication in September 2011 by Babij, et al. in Cancer Research, which also does not see STK33 as a hit in a similar shRNA screen, presents compelling data indicating STK33 is not, in fact, generally essential for survival KRAS-dependant cells.  Although, based on recent letters to the editor in Cancer Research, this does not seem to be the end of the discussion between these two groups, the initial excitement of STK33 seems to have been premature at best.

The KRAS synthetic lethal screens by Barbie et al. mentioned above that did not pick-up STK33 as a strong hit their screen did, however,  identify another potentially interesting gene—the IκB kinase TBK1—that appeared essential for, but previously unknown to be involved in,  KRAS lethality.  This target was not found by other groups and has yet to be confirmed, but Dr. Djaballah had some reservations as to whether statistical analysis of the data really supported this as a true “hit” or simply an outlier.  Dr. Djaballah also had similar concerns with a recent Nature Letter in Oct. 2011 by Zuber et al that identified Brd4 as an essential gene and possible therapeutic target in acute myeloid leukemia cells.  A more detailed review of the data from this screen revealed some issues with the confidence of this hit, as there was significant variability in the CV values and a couple of the shRNAs were enriched by as much as million fold.

Dr. Djaballah’s discussion was clearly not intended to disparage any specific study, but rather to demonstrate the slippery potential of over-interpreting the extensive data produced by such a powerful approach.  Based on the amount of resources and effort put into a complex screen, it can be difficult to maintain the reserve required to coldly and rigorously analyze the experimental design and results in a detached manner and properly assess which candidates really meet the criteria for follow up.  As a relatively new screening technology without much in the way of standards and defined good practices, it is easy to prematurely “fall in love” with potentially interesting targets that may be just noise in the data.  From his analysis, Dr. Djaballah suggests paying particular attention to the following three aspects:

1. Do the infections at the appropriate MOI and with sufficient cells to assess the effect. Although Dr. Djaballah was primarily talking about arrayed screens (with single shRNA plasmids in wells), the points is also very valid for pooled screening.  It is critical to ensure there are each cells containing each shRNA to be assayed to generate reliable reproducible results.
2. Use correct passage times. Whether the screen requires looking at knockdowns or survival, it is important that the cells are maintained for a long enough passage number to produce a significant differential between affected and non-affected cells.  Conversely, too many passages will introduce too much noise.
3. Pay attention to general data and overall results. It is important to see if the overall screening results make sense.  For example, are known lethal genes showing up in the hits?

While pointing out the importance of these considerations in evaluating a screening, Dr. Djaballah contended overall that there are currently few standards of practice to provide guidelines around these and similar procedural details.  We at Cellecta would agree, having focused most of our effort in the last several years toward optimizing the many subtleties of pooled shRNA screening to enable consistent, robust, and interpretable results.