The Path from Neuroscience to ML Engineering at Money Forward

Hello, I’m Danny and I’m working as a Machine Learning Engineer (MLE) at MoneyForward. But before this, I was getting my PhD in Behavioral Neuroscience with a specialization in Computational Neuroscience. This is a fairly non-traditional route, so I’m hoping my perspective can motivate those with a similar background to explore the field of AI/ML as well. You may be wondering: how does the work between an ML Engineer and Neuroscientist overlap, if at all? When you think of neuroscience you may be thinking of petri dishes and microscopes, but that is just the data collection part of the job, and discovery happens once we sit down and analyze our hard earned data. In short: a high level of software engineering skills are necessary to build efficient data analysis pipelines to answer our questions in computational neuroscience.

For example, imagine a behavioral neuroscientist wants to answer the simple question: how does our brain know where we are in the world? It’s an ability we take for granted, which enables us to get to our destinations fairly effortlessly, from point A to point B. Our brains are able to make sense of the space around us, understand its context, and remember where we are, and being able to do this accurately is essential for survival. So, where could we begin? Well, we could start by tracking animal behavior using cameras, and calculating where the animal is and where it’s facing. Then, to find the relationship between the animal’s brain and behavior, we can record the electrical brain signals (electrophysiology) and investigate how the signals are linked to certain actions. In my research, I recorded and analyzed signals that were sampled up to 32,000 times a second, in up to 64 different regions of the brain simultaneously. That’s a lot of data! Such a rich dataset requires advanced signal processing techniques and statistics to make sense of.

Figure 1: (Left) Illustration of a rat turning its head counterclockwise (CCW) and clockwise (CW). (Right) My dog Winry pathfinding from the start to the end, maybe for a reward.

Data Science and Machine Learning

The fields of data science (DS), machine learning (ML), and artificial intelligence (AI), have rapidly evolved in the last two decades, where the roles have been renamed and redefined. Every job posting used to be under the umbrella of “Data Scientist” where the job could vary widely. Some data scientists are stuck making dashboards in Tableau, some are glorified business analysts, and some get to work on building statistical and machine learning models. It soon became apparent that many models were being built but not being deployed, and as a result the field has further specialized to fill these gaps. In some companies where AI is a core part of their business, we may see research scientists who develop cutting edge ML models (ex. Google Brain). However most businesses are simply looking for the most cost effective way to utilize ML, so generally there are people who manage the databases and data pipelines (data engineers), prepare data and build the models (data scientists), implement the model into existing applications (machine learning engineers (MLE)), and deploy the model (machine learning operations engineers (MLOPs)).

When I first started graduate school I didn’t know where I would fit best. But, since the roles weren’t well defined, then it was also an opportunity for people with unique skill sets to find their own place. My experience was always more on the computational and analytics side, where I handled hundreds of gigabytes of neural data like playdough, shaping it into my desired format. As I wrapped up my PhD, I eventually found that I sat somewhere between a data scientist and an ML engineer since I enjoyed research but I also enjoyed the beauty of designing elegant software that could multiply the throughput of my team members. There were some key projects I completed which really helped define my current role as an ML engineer. Rather than being used as a goal, ML was more utilized as a tool to answer our neuroscience questions.

How can we utilize Data Science and Machine Learning in Neuroscience?

Data Engineering

One of the most interesting projects I worked on was finding biomarkers of seizures leading to death in a genetically modified mouse model, where this gene mutation often causes death in human patients early in childhood. Our lab wanted to know where in the brain these fatal seizures originated and how early we could predict seizure onset. A better understanding of the mechanism means we can provide more targeted medicine and therapies to affected patients.

However, we didn’t know when the fatal seizure would happen in our animals. This means the only solution was to record data for 24 hours a day, 7 days a week until the seizure occurred. At 32,000 samples/sec across 64 channels we would record up to 1TB of data a day. With this much data, even after parallelization it could take nearly 24 hours to finish a processing job. It quickly became apparent that we needed an automated method to transfer, convert, and analyze the data, otherwise we would sit around for days waiting for the jobs to finish. Furthermore, it’s not that common to utilize version control in academia, despite my best efforts to convert people, so in my lab there would be different versions of analysis pipelines. Lastly, running the pipelines manually could actually be quite tricky, so I found it was necessary to standardize the work as a recurring job. To this end, I built a no-code batch processing pipeline on our clusters that triggered based on entries to Google Sheets since many users were non-technical. By decreasing the amount of menial work and standardizing our workflow, it was a great success with a lower barrier of entry.

Figure 2: Example usage of DeepLabCut to track a mouse in a box

Data Science

Many of the steps in the aforementioned data pipeline involved traditional signal processing like the Fourier transform to extract frequency components from brain signals (neural oscillations are really fun but a whole different article). However, we also mixed in modern machine learning techniques where it was appropriate. For example, to understand the relationship between goal-driven behaviors and brain signals, we needed to track the head position of the animal. We wanted to achieve this with the least amount of extra equipment, and found the simplest approach to be DeepLabCut, a markerless pose estimation toolbox powered by deep learning. They started development with a resnet-50 backbone for the model but recently they seem to be experimenting with vision transformers. This tool allowed us to extract the x/y coordinates of the animal’s ears at 30 frames/sec to determine where the animal was facing. We could further process this signal by taking the first derivative of position we get head velocity, the second derivative for head acceleration, and the third derivative for head jerk, all important variables to measure intent and vigor.

Figure 3: Illustration of the Omar lab’s automated data analysis pipeline. The University of Michigan’s high performing clusters were used for storage and compute, pipeline was triggered with Google Sheets, and the final figures were uploaded to Google Drive.

Another byproduct of recording 24 hours a day means we also track the animal’s natural sleep-wake cycle. As mammals sleep, most alternate in a cycle of rapid-eye movement (REM) sleep and non-REM sleep. These sleep states are easily distinguishable from neural oscillations in the cerebral cortex or hippocampus, making it a perfect candidate for a multi-class classification type of ML model. REM is clearly defined by strong theta oscillations (6-9 Hz), non-REM by strong delta oscillations (1-4 Hz), and awake states are identified by any periods of movement found by DeepLabCut. Using hundreds of hours of sleep scored data, we processed signals from multiple brain regions into 300 features and developed several ML model candidates. In the end, we found that XGBoost, a gradient boosted tree algorithm, along with some post-processing rules for temporal smoothing to be the most effective. I developed this model early in my data science career, so I would love another stab at it with models that can deal with time-based dependence, since animals are unlikely to wake up for 1-2 seconds and go back to sleep.

Figure 4: Sleep states identifiable by neural oscillations. (Top) Example of neural signals transformed into the frequency domain using the fourier transform. Red indicates more power and blue indicates less power. (Bottom) Sleep states classified as awake, unclassified, non-REM, and REM.


Here I described a few skills used in behavioral and computational neuroscience research that translate well for work in the AI/ML industry. I think data science is a natural transition to industry for academics, and I think those with strong research, statistical and/or engineering backgrounds should give it a shot. I found my skills were split between a researcher and an engineer, so I decided to pursue a route where I could utilize both sides evenly. This led me to my current role as an ML Engineer here at MoneyForward, and I’ve found plenty of overlap since there’s a need for analysis and visualization of different data types. I highly encourage people from non-traditional backgrounds to take a look, since I think the different perspective fosters greater creativity and insights in any company. It really is exciting how much there is to learn in this field, and I hope I could help spur some inspiration for those looking to break into AI/ML.