Explaining Blackboxes

By the time this post is published, it will probably already be 2021, so I would like to start by wishing everyone a happy, healthy, and successful year!

During the last blog post we have talked about how we want to represent patients as vectors composed of our patients’ biomarkers. The question at hand is what biomarkers are relevant to determining the proper treatment, and how they are relevant. We know of existing biomarkers such as sites in the atria in which fibrosis enables the maintenance of re-entries. Such structures can be identified using MRI and their functional contribution to AF can be confirmed using ECGi. Implementing such a solution would be a top-down approach- beginning with existing knowledge and applying that knowledge to come to a conclusion. We refer to such a reasoning process as “deductive”- applying a theory to an observation and making a conclusion. The alternative approach is the bottom-up approach, in which large amounts of data are used to train a machine learning model to directly recommend a treatment. Such machine learning models use inductive reasoning (they observe patterns and generate rules), and usually outperform top-down systems in terms of accuracy but are of limited use in the medical field since a complementary explanation is critical if such system is to be applied in clinical practice. One of the larger contributions we are hoping to make is employing such machine learning models as data mining tools, and identify what they base their decision on. Can we make the black box explain to us what it sees and how it arrives at its recommendation? If the machine learning model outperforms the classical top-down system, maybe our knowledge is not exhaustive, and further phenomena may be discovered.