Tag: 神経科学

In November 2024, a preprint was posted on bioRxiv describing YORU, a behavior analysis tool that takes a different approach from existing behavior analysis tools.
In this post, I would like to explain what kind of tool it is.

YORU preprint

YORU documentation

What is YORU?

YORU is a deep learning-based tool for analyzing animal behavior.

Many deep learning-based tools, such as DeepLabCut and SLEAP, have already been released.
These tools use deep learning to track animal body parts and estimate posture (here I will call them tracking tools).

By estimating posture, they reveal what posture an animal is in and use this for behavior analysis.

YORU, by contrast, recognizes behavior using an object detection algorithm.
In short, it does not track body parts; instead, it defines behavior directly from the appearance of the behaving animal.

When we distinguish an apple from an orange, we rely on cues such as shape and color.
Object detection excels at classifying objects, so it makes exactly this kind of appearance-based distinction.

Applying this to animal behavior analysis means classifying animal behavior from its appearance.
Within YORU, this is referred to as a “behavior object.”

Whereas tracking tools represent behavior as points and lines, YORU analyzes behavior by enclosing it in a bounding box.

Advantages of defining behavior with object detection

The advantages of object detection-based behavior analysis include:

・Errors are less likely even as the number of individuals increases

・Less affected by the animal’s orientation

・Fast analysis speed

・Can capture behaviors that are hard to detect by tracking (such as a mouse crouching)

and so on.

It overcomes a weakness of tracking tools by making multi-animal behavior analysis simple and far less computationally costly.
It also makes it easy to analyze behaviors that are difficult to define from body part coordinates.

Consider, for example, the actions of opening and closing a hand.
With a tracking tool, you would track the fingertips with a camera and define the hand as open or closed, but if the fingertips become hidden when the hand closes, tracking fails and the positional information can no longer be computed accurately.

With object detection, however, the model looks at the overall shape of the open and closed hand, so it can still make an accurate call even when part of the hand is occluded.

As for robustness to increasing numbers of individuals: with a tracking tool, as the number of animals grows you must assign each body part to the correct individual in order to capture each animal’s behavior accurately, whereas YORU analyzes only the shape of each individual, so no identity assignment or part-to-individual matching is required.

Because analysis is also fast, real-time analysis is possible—you can trigger device control when a particular behavior occurs, enabling automation of behavioral experiments.

Disadvantages of defining behavior with object detection

There are, of course, drawbacks as well:

・Cannot capture a behavioral sequence

・Cannot identify individuals

・Cannot capture unknown behaviors

・Cannot capture behaviors that are not visually distinctive

・Provides no fine-grained detail about the defined behavior

These are precisely the strengths of tracking tools. Rather than one approach being better than the other, the two complement each other’s advantages and disadvantages.

Experimenters therefore need to choose the approach that best suits their situation.

What makes YORU remarkable?

Having covered the advantages and disadvantages of the object detection-based behavior analysis that underlies YORU, what exactly makes YORU itself stand out?

Everything runs through a GUI

YORU lets you perform behavior analysis without writing any code.

The same is true of tools such as DeepLabCut and SLEAP, and it goes a long way toward making these analyses accessible to biologists.

Design

Unlike previous tools, YORU’s design doesn’t really look like that of a research tool.

Built-in real-time analysis

Unusually for tools of this kind, YORU comes with a full GUI for real-time analysis.

Building a real-time analysis system is not that hard if you can program to some degree, but otherwise the barrier is quite high.

With YORU, however, you can carry out real-time analysis—and even fire external triggers—entirely without programming.

Adaptable to a wide range of experimental setups

The biggest hurdle in real-time analysis is figuring out how to combine the code that drives your own experimental apparatus with the analysis itself.

YORU adopts a plugin system that lets you choose the program used to control your external devices.

In other words, you simply select the plugin that matches your own system.

You can also write your own plugins: simply turning your device’s control program into a plugin makes it easy to link that hardware to YORU’s real-time analysis.

This makes for a remarkably user-friendly system that no other tool currently offers.

Closing thoughts

YORU has only just been released, so there are still rough edges in usability and clear gaps in the documentation.

That said, as these are addressed over time, I am very much looking forward to seeing how people put it to use.