Imaging Your Mind’s Eye | Mary Lou Jepsen | TEDxBeaconStreet
hey everybody uh I ship stuff I probably don't look like an engineer but I actually sleep on the factory floor is the world design really things people think are kind of impossible get them to work and get them to ship in high-volume mass production I've done this in holography in head-mounted displays and high-volume consumer electronics with things that were seemingly impossible the way I did that is by re-engineering subcomponents optical electronic electronic display optical sub components and so that's really what I want to talk to you about today but sort of played into a different area and I have to do a disclaimer first this is my hobby my full-time jobs at Facebook and oculus but this isn't work I do there it's work I did before I got there and outside of it one of the big problems of our time one of the big goals one of the big focuses of Technology in science is on how to reverse-engineer the human brain we see the White House brain initiative the European brain study the National Academies putting this work front and center as this is the moment to figure out how our brains work to finally understand and hopefully cure things like mental disease neurodegenerative disease and to understand ourselves and understand how we think in each other and hopefully communicate better with each other so I've been working on this for many years and over the last five years I've been really fascinated by the advances with this technology in its functional magnetic resonance imaging or fMRI many of you have probably laid in such a machine but it's being used to decode what we're thinking by looking at oxygen flow the oxygen flow can be measured in small cubes like these kind of cubes on the stage but very small one meter voxels or cubes by looking at the oxygen flowing in your brain where the oxygen is is where you're thinking or where the brain is active and where it isn't is where it's not active and using this technology we can and it's been shown decode using big data collection and and analytical algorithms in the computer see what you're imagining here the music in your head see and read the words that you're thinking of and this has been shown in study after study after study and I've just I was completely sucked in by this so I read a lot of neuroscience on the side again my hobby and I'm just like whoa how do we reduce this down to a cap so that we can communicate with human thought because I think that could change everything and I think it's in reach in the near future not so distant future and so I guess the question really is on this system the resident this is a very big system and you look at this and you think how can you get higher resolution cuz right now we can see images but they're grainy and a little bit out of focus here most of music see a lot of words but how do we get this to higher resolution and there's really three things that need to happen we need to increase the resolution of the system itself which is very hard because this is already essentially a very huge magnet we also need to increase the dataset acquisition and that's hard because there's only about tops a few hundred of these systems in the world today being used for functional magnetic resonance imaging to do functional brain scanning and then finally can we improve the analytic analytics to decode what we're thinking better with those three improvements the implication for our ability to communicate are profound for example if you imagine a movie director who wakes up with an idea for a new scene in her head to be able to dump a rough cut to the computer so she could share this new idea she had with her creative team without the usual confusion or a musician if he wakes up with an idea for a new song in his head but the layering of the music is so hard to get out in the usual process it takes weeks or months it could really change the way creatives work and all of us work if we could be able to dump rough cuts of our ideas directly to the computer but these systems not only are granion a little bit out of focus they're also super expensive there are a couple few million dollars apiece and they cost about a million dollars a year or more in upgrade in uptake costs and so it's very hard to get more big datasets because there's so few of them and so if we cut to last year this system came online and matched the resolution of fMRI system and this is called the system used here is called diffuse optical tomography it looks like a science project it kind of is it's a seven-figure science project it's still expensive but with it we can match the resolution of fMRI which means the computer can see what you're imagining read the words that are in your head and hear the music that's in your head and so while this is kind of unwieldy I was dazzled when I saw this because I I do optics and I thought whoa this is my stuff how can I help slim this down I mean even putting this thing on takes 20 minutes so if we could do that we could make brain imaging systems as consumer electronics and it could transform not just this new field that's emerging but also low-cost medical diagnostics where MRI is used to diagnose all kinds of conditions but it's very expensive in Boston we have these but throughout the developing world to make it affordable the impact could be profound there but also the idea of being able to export our ideas to digital media could change everything so just to be clear I'm talking about consumer electronics as wearable communications where we can communicate with thought alone so this system you see the fiber optics going to the head but above it our detectors with four orders of magnitude resolution and there's 80 different fibers that are shining light into the head invisible light infrared light that hat is frequency encoded there's a heartbeat to it and that light can penetrate about five centimeters deep because the brain scatters that light and it scatters more or less depending upon whether there's oxygen there because the oxygen from the from the blood either absorbs more light or reflects more light depending upon whether the oxygen is there so that's what the detectors do and again they need four orders of magnitude resolution and to get that they actually are sort of cooled and above the head they can't be close to the to the noggin if you will they have to be away single photon detectors very expensive picosecond timing that's ten to the negative twelve seconds of resolution and I look at this and I think this is really cool we got rid of the big magnet problem but it's optics but this is this is like neuroscientists did this and they're they're really good at neurosciences but they're not really good at high volume consumer electronics distinguished by optics and after electronics and so the thing is what I did was go back to the basic math to see if I could figure out a better way to do this and I went back to the basic math and found this landmark paper in 1999 that proved diffuse optical tomography was mathematically impossible that you couldn't solve for the diffusion versus the reflectance at the same time and this killed the field for a decade because somebody proved it was completely impossible so all work stopped everybody left except well as you know not everybody because some people love working on impossible things and so luckily some people kept working and it took a decade till this guy best-in hurich proved that while that was mathematically impossible if you make a simple simplification of what's called piecewise approximation in other words a bit like taking a picture of you all and then looking at the pixels that show up on the camera little segments the way basically we solve most computation problems what we do with most images if we do that it's no longer intractable you can solve it diffuse up that optical tomography works game back on for the system and five years after this this was in 2009 that fiber-optic wig contraption I showed a couple slides ago was done and has exceeded the resolution of magnetic resonance imaging so just a lesson when somebody tells you it's impossible don't necessarily give up so I went back to the math and took a fresh look at it with what all but all that I know about consumer electronics not do electronics and I looked at basically what I could do to rethink this design a lot of people say you have to have perpendicular incidence of light and I thought well that doesn't really make sense and so I've now come up with a design that I'm building using flexible liquid crystal displays inside of a cap illuminated with invisible light and so that liquid crystal displays that's like for your TV or your laptop or your cell phone that's just a display well liquor crystals modulate light and can bend light in different ways and the infrared light can see into your head you can it's well it's Oh peg clearly to white light you can't see in my head with this white light the infrared light gets in and we can get to five centimeters of depth to look at what we're seeing this gives much higher resolution not 80 detectors and emitters as was shown in the fiber-optic wig but every cell phone you have in your pockets today has about 8 million sub pixels those could be emitters cameras on your cell phone have more than 8 million detectors and so using this approach we can eliminate the room size fMRI system or the fiber-optic wig with scaffolding to make a much smaller system this small and so you're probably looking at this and you're thinking yeah sure this is crazy right how how is this possible just looking at oxygen flow if you if just imagine these are one millimeter cubes filled filling your brain looking at oxygen flow how is it possible to collect human thought many people focus on studying how the neuron works and some people believe there's six Nobel prizes just to understand how the neuron works and then the connections between them and the computation is intractable and on and on and on but the facts are study after study after study over the past five years his showing has shown that using the imaging of oxygen flow big data math sets and analytics we can compute and emit what image you're imagining what words you're thinking of what music you're hearing in your head and so forth not just 80 detectors but eight million can increase that resolution dramatically and I have to say one more time this is my hobby it's not working during a Facebook or oculus but it's not impossible and just like diffuse optical tomography was not impossible and I believe in the very near future wearable consumer electronics will exist where we can communicate with human thought alone the implications are profound and thank you very much for listening