Tag: Yolo

What Exactly Is YORU, a New Behavior Analysis Tool Powered by Object Detection?

In November 2024, a preprint was posted on bioRxiv describing YORU, a behavior analysis tool that takes a different approach from existing behavior analysis tools.
In this post, I would like to explain what kind of tool it is.

YORU preprint

YORU documentation

What is YORU?

YORU is a deep learning-based tool for analyzing animal behavior.

YORUのサイトより引用

Many deep learning-based tools, such as DeepLabCut and SLEAP, have already been released.
These tools use deep learning to track animal body parts and estimate posture (here I will call them tracking tools).

By estimating posture, they reveal what posture an animal is in and use this for behavior analysis.

YORU, by contrast, recognizes behavior using an object detection algorithm.
In short, it does not track body parts; instead, it defines behavior directly from the appearance of the behaving animal.

When we distinguish an apple from an orange, we rely on cues such as shape and color.
Object detection excels at classifying objects, so it makes exactly this kind of appearance-based distinction.

Applying this to animal behavior analysis means classifying animal behavior from its appearance.
Within YORU, this is referred to as a “behavior object.”

Whereas tracking tools represent behavior as points and lines, YORU analyzes behavior by enclosing it in a bounding box.

Advantages of defining behavior with object detection

The advantages of object detection-based behavior analysis include:

・Errors are less likely even as the number of individuals increases

・Less affected by the animal’s orientation

・Fast analysis speed

・Can capture behaviors that are hard to detect by tracking (such as a mouse crouching)

and so on.

It overcomes a weakness of tracking tools by making multi-animal behavior analysis simple and far less computationally costly.
It also makes it easy to analyze behaviors that are difficult to define from body part coordinates.

Consider, for example, the actions of opening and closing a hand.
With a tracking tool, you would track the fingertips with a camera and define the hand as open or closed, but if the fingertips become hidden when the hand closes, tracking fails and the positional information can no longer be computed accurately.

With object detection, however, the model looks at the overall shape of the open and closed hand, so it can still make an accurate call even when part of the hand is occluded.

As for robustness to increasing numbers of individuals: with a tracking tool, as the number of animals grows you must assign each body part to the correct individual in order to capture each animal’s behavior accurately, whereas YORU analyzes only the shape of each individual, so no identity assignment or part-to-individual matching is required.

Because analysis is also fast, real-time analysis is possible—you can trigger device control when a particular behavior occurs, enabling automation of behavioral experiments.

Disadvantages of defining behavior with object detection

There are, of course, drawbacks as well:

・Cannot capture a behavioral sequence

・Cannot identify individuals

・Cannot capture unknown behaviors

・Cannot capture behaviors that are not visually distinctive

・Provides no fine-grained detail about the defined behavior

These are precisely the strengths of tracking tools. Rather than one approach being better than the other, the two complement each other’s advantages and disadvantages.

Experimenters therefore need to choose the approach that best suits their situation.

What makes YORU remarkable?

Having covered the advantages and disadvantages of the object detection-based behavior analysis that underlies YORU, what exactly makes YORU itself stand out?

Everything runs through a GUI

YORU lets you perform behavior analysis without writing any code.

The same is true of tools such as DeepLabCut and SLEAP, and it goes a long way toward making these analyses accessible to biologists.

Design

Unlike previous tools, YORU’s design doesn’t really look like that of a research tool.

Built-in real-time analysis

Unusually for tools of this kind, YORU comes with a full GUI for real-time analysis.

Building a real-time analysis system is not that hard if you can program to some degree, but otherwise the barrier is quite high.

With YORU, however, you can carry out real-time analysis—and even fire external triggers—entirely without programming.

Adaptable to a wide range of experimental setups

The biggest hurdle in real-time analysis is figuring out how to combine the code that drives your own experimental apparatus with the analysis itself.

YORU adopts a plugin system that lets you choose the program used to control your external devices.

In other words, you simply select the plugin that matches your own system.

You can also write your own plugins: simply turning your device’s control program into a plugin makes it easy to link that hardware to YORU’s real-time analysis.

This makes for a remarkably user-friendly system that no other tool currently offers.

Closing thoughts

YORU has only just been released, so there are still rough edges in usability and clear gaps in the documentation.

That said, as these are addressed over time, I am very much looking forward to seeing how people put it to use.

2024-11-17
[Updated December 2022] How to Install YOLOX
What is YOLOX?

Among the many entries in the YOLO series, the latest one, YOLOX, was released in July 2021.

The YOLOX paper is available here (currently a preprint).
The GitHub repository is here.

YOLO is an object detection algorithm.
It is designed to run in real time and is extremely lightweight (inference alone can even run on a smartphone). It is also highly adaptable, and is widely used in IoT devices.

“Object detection algorithm” may sound complicated, but an easy analogy is a camera’s automatic face or eye focus.
A marker appears indicating “this is a face”; the same idea applies to other objects, and the algorithm can also classify what each one is.

There are many versions in the YOLO series: YOLO, YOLOv2, YOLOv3, YOLOv4, YOLOv5, and YOLOX.
It also runs not only on Python but on MATLAB as well, so you can adapt it to whichever platform you use.

Unlike the architecture used from YOLOv3 onward, YOLOX returns to the design of the original YOLO. It is reportedly faster and considerably more accurate.

Other blogs cover the details as well.

How to install YOLOX

Installation instructions are given on the GitHub page, but they are in English and do not cover using a virtual environment, so here I explain how to do it with an Anaconda virtual environment.

On both Windows and Mac, following the YOLOX manual exactly produced one error after another.
Errors will likely keep changing with future versions, so look them up as they come.

See below for how to use the conda command in the macOS Terminal.

Using Conda Commands on macOS

On Windows, I recommend using Anaconda Prompt.

Open the Terminal (Anaconda Prompt), and you will get a black window filled with text.
You should see (base) at the left of the last line.
This means you are not currently inside a virtual environment.

YOLOX will run without a virtual environment, but since environments can conflict with other packages, I recommend creating one.

First, enter the following code to create a virtual environment.
```
conda create -n YoloX python=3.8
```
YoloX is the name of the virtual environment; you can use any name you like.
You can also specify the Python version with python=3.8.

Then activate the virtual environment with the following code.
```
conda activate YoloX
```
If (base) on the left has changed to (YoloX), you are all set.

Installing CUDA

First, if this is your first time installing machine learning libraries and you plan to train YOLO on a GPU, install CUDA.
(Training on a CPU takes an absurdly long time, so I do not recommend it. Note that GPUs cannot currently be used on Apple Silicon machines.)
CUDA can be downloaded from the official site.

As for the CUDA version, I suggest checking which CUDA versions PyTorch supports and installing one of those.
PyTorch official site

To install an older version, click Download now and then, further down the page under Resources, use Archive of Previous CUDA Releases.

The latest version may work too, but for now, matching the version to PyTorch should let you run everything without trouble.

To check whether it is installed on Windows, open “Edit the system environment variables,” go to Advanced > Environment Variables, and see whether there is a path beginning with CUDA_PATH.
The number after the V indicates the version.

Installing PyTorch

Before installing, update pip.
Activate the virtual environment in Anaconda Prompt and run the following command.
```
python -m pip install --upgrade pip
```
Go to the PyTorch official site and choose the installation method that matches your environment.
One catch here is the Package field: be sure to select pip.
Installing with conda will cause errors later on.

Copy and paste the command under “Run this Command” to install it in your virtual environment.

Once the installation finishes, use the following command to check that PyTorch is installed.
```
pip list
```
As long as torch appears in the list, you’re good to go!!!
As a quick check, start python and run the following command.
```
import torch
print(torch.cuda.is_available())
```
If the output is True, the installation succeeded and torch is ready to run on the GPU.
(On a Mac or on a computer without a GPU it will return False, but as long as the import works you’re fine.)
If you get an error, go back and check the installation again.

Installing the packages required for YOLOX

Download (clone) the YOLOX files from GitHub.
If you can use Git commands, you can clone the repository with the code below.
```
git clone git@github.com:Megvii-BaseDetection/YOLOX.git
```
Next, move into the cloned folder and install the required packages.

First, edit the contents of requirements.txt in the cloned YOLOX folder as follows.
```
# TODO: Update with exact module version
numpy
#torch>=1.7
opencv_python
loguru
tqdm
#torchvision
thop
ninja
tabulate

# verified versions
# pycocotools corresponds to https://github.com/ppwwyyxx/cocoapi
#pycocotools>=2.0.2
#onnx==1.8.1
onnxruntime==1.8.0
onnx-simplifier==0.3.5
```
It seems that packages such as pycocotools cannot be installed with this command.

Once that’s done, save requirements.txt and run the command below in the terminal (Anaconda prompt).
```
cd YOLOX
pip3 install -r requirements.txt
pip3 install cython pycocotools
```
The downloaded YOLOX folder contains a file called “requirements.txt” that lists the required packages. The second command opens this file in read-only mode and installs the packages listed in it.
```
pip3 install cython pycocotools
```
This command installs cython and pycocotools.
It seems these have to be installed separately.

When you run this command, some of you may see the following error.
```
      building 'pycocotools._mask' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
```
Microsoft C++ Build Tools is apparently required by pycocotools. (Oddly enough, the error also appears on Mac, even though there is no Mac version.)
Windows users should install it just to be safe.

After installing it, try the installation again with the code below.
```
pip3 install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"
```
If that still doesn’t work, install it with the following command.
```
conda install -c conda-forge pycocotools
```
In general we’ve been installing these packages with pip, but since this one simply won’t install that way, we resort to installing it with conda.

Next, download yolox_x.pth.
The download starts as soon as you click the link.

Move the downloaded file into the YOLOX folder you cloned from GitHub.
This gives you the model used for detection in the demo, placed wherever you like.

Add the following lines to the top of tools/demo.py so that the yolox folder can be accessed.
```
#demo.pyの中身に書き込みましょう。
import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
```
Without this, you’ll get an error saying the yolox package cannot be found.

Then run the following command in the terminal.
This lets you check whether the demo runs correctly.
```
python tools/demo.py image -n yolox-x -c yolox_x.pth --path assets/dog.jpg --conf 0.25 --nms 0.45 --tsize 640 --save_result --device gpu 
#gpuで動かさない場合には、gpuをcpuに書き換えてください。
```
Replace –device gpu with –device cpu as appropriate for your setup.

If the following image appears in YOLOX/YOLOX_outputs/yolox_x/vis_res/20…….., everything is working.

Bonus

For those who aren’t sure which computer to buy, I’ve started offering PC purchase consultations on Coconala!
あなたの要望に合わせてパソコンを選び、提案しますパソコン選びに困っている方々へ！様々な目的に対応できます！
I often pick out computers and give advice, and many friends told me I could make money doing PC consultations.
Inspired by that, I decided to give it a try!

I’ll recommend a machine that fits how you plan to use it and your budget.
In particular, I’ve chosen and used many computers for machine learning.

And if you’d like, I can also advise you on what to look for the next time you buy a computer.

Computers I’ve picked out so far include analysis machines for the lab, everyday-use machines, game-streaming machines, machines for incoming university students, machines that can run CAD for architecture students, and simple all-purpose machines.

I use both Mac and Windows, so I can talk about and recommend either!

Please feel free to make use of it.
2022-11-15