Neuroimaging, GANs, and Dota2

Having recently attended Neurohackademy 2018 hosted by UW’s eScience Institute, I find myself swimming in an ocean of ideas that’s just waiting to be explored. The two-week conference and hackathon was hosted by a group of neuroimaging scientists dedicated to open science, and it was well-received by both students and instructors. I believe the lingering feels from the hackademy plus the spirit of open science inspired many students, including myself, to begin blogging and sharing their journeys in science. During the hackademy, OpenAI held a benchmark event in San Francisco where robots competed against humans in a game of Dota2. This will be my attempt at telling a story that connects neuroimaing, machine learning, and the most competitive video game tournament in the world.

As a naive student early on in my PhD, I thought sensor resolution and data dimensionality are the biggest issues stopping neuroscientists from making advancements in the field. After all, aren’t we all shackled by the limits of technology? What else could possibly be bothering all these scientists with powerful computing clusters? Turns out, several discussion sessions at the Hackademy was all it took to change my mind. The difficulty and nuances of neuroimaging is best summed by up this comic shared through the Hackademy’s slack channel:

The comic was inspired by a forum discussion on effect sizes in neuroimaging studies. For example, magneto-encephalography (MEG) records tiny changes in magnetic field produced by billions of electrical currents being exchanged between neurons. This already sounds difficult. The experiment sounds even more ridiculous when you consider all the electrical wires in our walls running interference competing against our instruments. As for magnetic resonance techniques (MRI), we have to produces a magnetic field that’s hundreds to thousands times stronger than the earth’s magnetic field to align our atoms to detect the tiny signal given off by each atom’s spin. Then we have to shove all these data into pipelines for noise removal, head motion correction, normalize each scan to a common coordinate system, and map the small signals we detected back to our brain regions of interest. Factor in some sampling variability and differences in tool usage, it’s no wonder that people have difficulty replicating or building upon each other’s experiments.

We’ve been blessed by some brilliant minds who found ways to relate our recordings to brain function with limited knowledge and technical hurdles, but there are questions we still cannot answer definitively. Imagine 300 sensors surrounding a patient’s scalp recording brain activity at 1000 times per second, you can’t help but wonder: where inside the brain are these signals coming from? Now what if I told you there are about 100 billion neurons firing all over your brain volume, and they all contributed to your 300 sensor readings, doesn’t this sound impossible? This is essentially saying I have 300 solutions to a system of equations, and I have to solve for a combination of the 100 billion variables that gives me the solution I recorded. You can immediately see how this problem is “ill-posed”: there are numerous non-unique combination of variables that you can choose from to reproduce your recordings, how do you know which one is correct?

Some high dimensional data cannot be simplified into a direct input-output model, and with the availability of these “big data”, neuroscientists are turning to artificial intelligence (AI) to help us create models. The most successful efforts in this field belong to discriminative networks that takes high dimensional data, and reduces them into a label. For example, a neural network that takes patient demographic, brain scans, and cognitive test scores to determine if a person is healthy or at risk for neurodegeneration. But if you flip the question around and ask if a neural network can generate some of the patient’s info based on a single diagnosis, that is much more difficult.

At the Hackademy, I had the pleasure of learning about Generative Adversarial Nets (GANs) with Dr. Sanmi Koyejo. This framework locks a con-artist (generative model) and a cop (discriminative model) in an interrogation room. The cop is really good at spotting lies because we train it to be able to distinguish real data recordings from model generated data. The con-artist knows what the sample distribution of a real data looks like, but it has to generate lies out of the sample distribution to make the cop believe him. The competition in this game drives both adversaries to improve their craft, until the counterfeits are indistinguishable from the real. If you are comfortable with python programming, you can try your hands on this Github Repo to generate some “fake” fMRI data. For convenience, this is what GANs can do in neuroimaging:

Pretty impressive huh? This is incredibly useful in machine learning where researchers are required to have sets of training, testing, and validation data, but they only have so much resources to collect a limited amount. Ian Goodfellow, whom recently left openAI for Google, is the inventor of GANs, and he would argue that there are numerous other uses for this framework, but I will save that for another day. Soon after his departure, openAI robots were pitted against humans in a video game called Defense of of the Ancients (DotA2). The goal of the 5 on 5 game is to destroy the enemy’s buildings in an arena, while this might seem like a fun and simple task for our minds, it’s a whole different stories for AIs:

• The video game runs at 30-60 frames per second, and each game lasts about 40 minutes, meaning the robots have to perform at least 72000 actions, and each one of their actions have consequences in the near and far future.

• Characters in the game have limited vision around the arena, the robots cannot see their opponents unless they are close by, meaning the robots have to extrapolate information from what’s seen on the map only, making strategizing even more difficult. Remember the AI that beat humans in chess or Go? They had all the information they needed on the board!

• The robots have to work together as a team to defeat the team of humans, how do you tell independent robots to have teamwork?

• Dota2 has more than 100 characters for players to choose from, each character has a set of unique abilities, and they can obtain more than 100 different items to enhance their abilities. Then they have to perform actions in an arena full of interactive objects, can you see how the optimization of this high dimensional problem can get super complex?

Dota2’s complexity and competitive strategic battles are what makes it one of the most popular games in the world, with an annual tournament prize pool of more than $40M. For the past two years, openAI has been invited to compete against the best professional players at the most prestigious tournament: The International. To train for this tournament, the 5 bots were pitted against 5 bots in the arena without supervision. At first they just stood around and did nothing, but they slowly learned about advantagious actions and strategies. The best introduction to this whole project is OpenAI’s youtube video:

The openAI five bots defeated some high level players but fell short of defeating the pros, the progress they made just by telling these robots to fight it out amongst themselves is already impressive. The robots worked as a team, often times manuvered more efficiently than the pros. I feel now is an appropriate time for a PSA, AIs are optimized to perform a very specific task in a controlled training environment, they are nowhere near from being self-conscious and taking over the world, any small tweaks in the environment can cause an AI to fail. I should also clarify that openAI five did no use an exact GANs architecture, but the unsupervised adversarial training concept is the same.

GANs and OpenAI Five are some of the first successful model-free generative frameworks to-date, meaning we do not have a set of equations explaining how exactly these AIs are performing these tasks involving high dimensionality data. Yes, it is scary to think that the very minds that created these algorithms do not fully understand their mechanisms. For now, I’m just glad these powerful tools are at my disposal, so I can make some sense out of all the data being shared by the lovely open science community.