FDA advisory panel meets, debates digital pathology

Last Thursday and Friday the U.S. FDA convened an advisory panel of pathologists and statisticians to seek input on regulation of digital pathology.  Day one was devoted to scientific and technical issues, and day two was study design and other regulatory considerations.

Each day began with some presentations by invited speakers and FDA scientists and examiners, followed by lunch, and then one hour was set aside for “public” speakers; basically anyone could ask for a speaking slot, and they ended up being six minutes each.  Following that, on each day the panel was asked to debate and answer a series of questions posed by the FDA.

I have posted the FDA advisory panel Agenda (which includes a roster of the panel members), the Questions posed by the FDA to the advisory panel, and a Background Briefing document provided by the FDA.

Day One: Scientific and Technical considerations for digital pathology

• Dorothy Adcock, panel chairperson, gave some opening remarks about the purpose of the panel.
• Louise Magruder, panel secretary and designated federal officer, made a compliance statement and reviewed some of the rules of order.
• The panel members were each asked to introduce themselves.
• There was a brief ceremony to recognize Louise Magruder for her years of service (she is apparently retiring).
• Tremel Faison (FDA Scientific Reviewer) gave a historical overview of FDA regulation of imaging applications.  There is a special exemption for optical microscopes (OM) as Class I devices. This exemption does not automatically apply to whole-slide imaging (WSI) devices.
• Michael Descour (University of Arizona and until recently, CTO of DMetrix) presented: the physical characteristics and measurements of optical imaging systems.  Very detailed technical discussion of optical microscopes (OM).
• Ulysses Balis (University of Michigan) presented: the features essential for interpretation of glass slides.  Another very detailed technical discussion.
• Michael Becich (University of Pittsburg) presented: digitization of glass slides, hardware and software issues.  UPMC have been using WSI for some time; reviewed old studies done using 1st generation scanner and 2nd generation scanner.  These studies generally showed that pathologists reach the same diagnosis with WSI as they do with OM, despite artifacts, focus issues, etc.
• Kevin Lorick (FDA Scientific Reviewer) next presented: basics of digital imaging.
• Aldo Badano (FDA Research Physicist) talked about: display of digital images, hardware and software.  A detailed discourse on monitor characteristics.

Lunch!  After lunch was an hour set aside for public speakers, of whom there were seven, each given a six minute slot.

• George Netto (Johns Hopkins), a colleague of Juan Rosai (Centro Diagnostico Italiano), read a wonderful letter from Dr. Rosai to the FDA panel.  I must tell you my heart swelled with pride to have the foremost pathologist in the world singing the praises of digital pathology, including mention of Aperio’s SecondSlide pilot in which he diagnosed cases referred to him by correspondents in Peru, Argentina, and Brazil.
• An FDA lawyer (didn’t catch his name) mentioned DICOM Working Group 26 and the need to incorporate WSI into DICOM.  This is a good effort but separate from regulatory efforts.
• Dirk Soenksen (in his capacity as President of the Digital Pathology Association) gave a presentation: the applications and adoption of Digital Pathology.
• Ole Eichhorn (oh yeah him) presented: Measuring Accuracy in Digital Pathology.
• Bob Dunstan (Biogen-Idec) talked about the importance of digital pathology for clinical studies.
• Gene Cartright (CEO of Omnyx) gave some thoughts on digital pathology from an Omnyx perspective.
• Lasko Fanand (University of Budapest and affiliated with 3D Histech) talked about studies they’ve conducted on use of WSI.
• Next the panel were invited to ask questions of any of the public speakers.
• Anand Agrawal (FDA Electrical Engineer) and Max Robinowitz (FDA Senior Medical Officer) presented: preclinical bench testing of WSI systems.  The basic thesis was that some kind of ‘phantom’ (synthetic specimen) should be created to provide an objective measure of image quality and a standard of comparison between OM and WSI.
• Next the floor was opened for the panel to ask questions of the FDA presenters.

After a break the FDA questions for the panel were presented and discussed.  (See Questions document, Day One.)

• Q1: What standards apply to the characterization of OM and WSI?  Summary of panel’s answer: no standards exist.
• Q2: Are there objective methods using ‘phantoms’ useful in the validation of WSI compared to OM?  Summary of panel’s answer: phantoms are not that useful.
• Q3: Are there biological specimens that could be used to challenge the performance of a human observer in a controlled way?  Summary of panel’s answer: challenging human observers doesn’t seem useful.
• Q4: What features of image quality are critical requirements and must be achieved for safe and effective use?  Summary of panel's answer: image quality features are not critical requirements.
• Q5: Discuss the risks and benefits of one z-plane of focus in WSI.  Summary of panel's answer: one z-plane doesn’t represent much risk.

Day Two: Study characteristics and other regulatory considerations for digital pathology

• Day two began similarly to day one, with Dorothy Adcock giving an introduction, and Louise Magruder giving a compliance statement.
• Tremel Faison gave a summary of day one; a summary of the agenda and questions.
• Kyle Myers (FDA Division Director) reviewed the history and lessons from FDA regulation of digital radiology.
• Shanti Gomatam (FDA Mathematical Statistician) presented statistical considerations in the evaluation of WSI devices.
• Brandon Gallas (FDA Mathematician) discussed evaluation of reader performance and variability.
• Tan Nguyen (FDA Medical Officer) discussed clinical trial design for digital pathology.  Interestingly and helpfully he used his presentation to ask questions and tee up issues.
• Next the panel were given a chance to question the FDA presenters.

Lunch!  After lunch there was an hour set aside for public speakers, there were six.

• Andy Evans (University of Toronto) presented a brief overview of their use of WSI for frozen sections in clinical practice.  A nice presentation illustrating that WSI is being used clinically today.
• Paul Valenstine spoke on behalf of the CAP, and placed a document into the public record (CAP Statement to FDA panel).  Noted WSI is at an inflection point in adoption.
• Robert Monroe (CMO Bioimagene) spoke for two-six minute slots; he also spoke on behalf of Ajit Singh (CEO Bioimagene) who was present but [apparently] was unable to speak.  Described his rural practice and the usefulness of digital pathology.
• Eric Yeo (COB of Aurora) and Pierre LeFèvre (CEO of Aurora) tag teamed on a presentation regarding Aurora’s suggestions for viewing and conversion/compression of digital pathology images.
• Next the panel asked questions of the public presenters.

Next the FDA questions for the panel were presented and discussed. (See attached Questions document, Day Two.)

• Q1: FDA clearance / approval of mammography requires lossless compression. Should lossless compression be required for WSI devices? Summary of panel's answer: lossless compression should not be required.
• Q2: Should WSI be approved / cleared for all organ systems at once, based on evidence from a single or limited number of organ systems?  Summary of panel's answer: difficult to use organ- or disease- specific approach, use fit-for-purpose model.
• Q3: Comment on various components of clinical study.
• Q3.1: Reference standard for diagnostic truth.  Summary of panel's response: no general agreement on expert panel vs intra- or inter-reader. Group agrees diagnoses is important point of comparison.
• Q3.2: Type of clinical studies.  Summary of panel's response: it is important for whole slide (maybe whole case) to be included, retrospective is okay (could recut slides to optimize), some enrichment is okay important to get all features, washout not an issue.

There was a short break, after which it was noted that further comments could be submitted by *anyone* to Maria Chan (maria.chan@fda.hhs.gov).

• Q3.3: Minimization of reader variability.  Panel generally agreed a “fully crossed” study design was best.
• Q3.4: Selection of study pathologists.  Summary of panel's response: panel can’t really answer question; introduce as much variability as possible.
• Q3.5: Performance assessment.  Summary of panel's response: study should record as much as possible which was on pathology report, indicate confidence as well as diagnosis, when deferred indicate deferral reason.
• Q4: Discuss factors that would be important for inclusion in precision studies.  Summary of panel's response: measure precision (variability) of device, separate from reader (if possible).
• Q5: Discuss training required for study pathologists evaluating WSI and for pathologists in postmarket use of WSI.  Summary of panel's response: training is required for study participants and in clinical use, manufacturer is responsible, a training set might be a good practice.
• Q6: Are postmarket studies required (safety and effectiveness issues that can only be assessed postmarket)?  Summary of panel's answer: postmarket studies are not required.
• Maria Chan (FDA IVD Director) thanked the panel for their great participation and input.
• Bob Becker (FDA IVD CMO) gave a closing summation.

 
And so ended the panel discussion.  All in all a fascinating and informative couple of days, and I believe the FDA got a lot of good information they can use for deciding how to regulate digital pathology systems going forward.