Hayato M. Yamanouchi

Tag: マニュアル

What Exactly Is YORU, a New Behavior Analysis Tool Powered by Object Detection?

In November 2024, a preprint was posted on bioRxiv describing YORU, a behavior analysis tool that takes a different approach from existing behavior analysis tools.
In this post, I would like to explain what kind of tool it is.

YORU preprint

YORU documentation

What is YORU?

YORU is a deep learning-based tool for analyzing animal behavior.

YORUのサイトより引用

Many deep learning-based tools, such as DeepLabCut and SLEAP, have already been released.
These tools use deep learning to track animal body parts and estimate posture (here I will call them tracking tools).

By estimating posture, they reveal what posture an animal is in and use this for behavior analysis.

YORU, by contrast, recognizes behavior using an object detection algorithm.
In short, it does not track body parts; instead, it defines behavior directly from the appearance of the behaving animal.

When we distinguish an apple from an orange, we rely on cues such as shape and color.
Object detection excels at classifying objects, so it makes exactly this kind of appearance-based distinction.

Applying this to animal behavior analysis means classifying animal behavior from its appearance.
Within YORU, this is referred to as a “behavior object.”

Whereas tracking tools represent behavior as points and lines, YORU analyzes behavior by enclosing it in a bounding box.

Advantages of defining behavior with object detection

The advantages of object detection-based behavior analysis include:

・Errors are less likely even as the number of individuals increases

・Less affected by the animal’s orientation

・Fast analysis speed

・Can capture behaviors that are hard to detect by tracking (such as a mouse crouching)

and so on.

It overcomes a weakness of tracking tools by making multi-animal behavior analysis simple and far less computationally costly.
It also makes it easy to analyze behaviors that are difficult to define from body part coordinates.

Consider, for example, the actions of opening and closing a hand.
With a tracking tool, you would track the fingertips with a camera and define the hand as open or closed, but if the fingertips become hidden when the hand closes, tracking fails and the positional information can no longer be computed accurately.

With object detection, however, the model looks at the overall shape of the open and closed hand, so it can still make an accurate call even when part of the hand is occluded.

As for robustness to increasing numbers of individuals: with a tracking tool, as the number of animals grows you must assign each body part to the correct individual in order to capture each animal’s behavior accurately, whereas YORU analyzes only the shape of each individual, so no identity assignment or part-to-individual matching is required.

Because analysis is also fast, real-time analysis is possible—you can trigger device control when a particular behavior occurs, enabling automation of behavioral experiments.

Disadvantages of defining behavior with object detection

There are, of course, drawbacks as well:

・Cannot capture a behavioral sequence

・Cannot identify individuals

・Cannot capture unknown behaviors

・Cannot capture behaviors that are not visually distinctive

・Provides no fine-grained detail about the defined behavior

These are precisely the strengths of tracking tools. Rather than one approach being better than the other, the two complement each other’s advantages and disadvantages.

Experimenters therefore need to choose the approach that best suits their situation.

What makes YORU remarkable?

Having covered the advantages and disadvantages of the object detection-based behavior analysis that underlies YORU, what exactly makes YORU itself stand out?

Everything runs through a GUI

YORU lets you perform behavior analysis without writing any code.

The same is true of tools such as DeepLabCut and SLEAP, and it goes a long way toward making these analyses accessible to biologists.

Design

Unlike previous tools, YORU’s design doesn’t really look like that of a research tool.

Built-in real-time analysis

Unusually for tools of this kind, YORU comes with a full GUI for real-time analysis.

Building a real-time analysis system is not that hard if you can program to some degree, but otherwise the barrier is quite high.

With YORU, however, you can carry out real-time analysis—and even fire external triggers—entirely without programming.

Adaptable to a wide range of experimental setups

The biggest hurdle in real-time analysis is figuring out how to combine the code that drives your own experimental apparatus with the analysis itself.

YORU adopts a plugin system that lets you choose the program used to control your external devices.

In other words, you simply select the plugin that matches your own system.

You can also write your own plugins: simply turning your device’s control program into a plugin makes it easy to link that hardware to YORU’s real-time analysis.

This makes for a remarkably user-friendly system that no other tool currently offers.

Closing thoughts

YORU has only just been released, so there are still rough edges in usability and clear gaps in the documentation.

That said, as these are addressed over time, I am very much looking forward to seeing how people put it to use.

2024-11-17
How to Install SLEAP [Updated February 2024]
Here I’d like to walk through how to install SLEAP, a machine learning-based tool for tracking animal body parts.

DeepLabCut is the best-known tracking tool, and I’ve written about how to install it myself in the past, while many others have posted articles about it as well.

SLEAP, however, has relatively few articles written about it in Japanese.
SLEAP has many advantages over DeepLabCut, so I decided to write up the installation procedure to make it a viable option for more people.

SLEAP’s documentation covers everything from installation to usage very clearly, complete with videos, so anyone comfortable with English should find it easy to get started.

The SLEAP paper is available here

The official SLEAP website is here

The SLEAP GitHub repository is here

Introduction

First, a note about the approach described in the official SLEAP installation guide:
```
mamba create -y -n sleap -c conda-forge -c nvidia -c sleap -c anaconda sleap=1.3.3
```
the one-line command that is supposed to handle environment creation, package installation, and everything else at once. I tried it on several computers, but in every case it hung indefinitely during the environment creation step and never completed.

So in this post I’ll introduce the alternative installation method that the developers also suggest.

Installation

We’ll basically follow the installation instructions on the official SLEAP site.

Environment

・Windows 11 Pro
・Verified on an NVIDIA 3080
・SLEAP 1.3.3
・Python 3.7.12

Downloading the files

First, download the files from the SLEAP GitHub repository to your computer.

If you’re using Git, run the following command.
```
git clone https://github.com/talmolab/sleap
```
If the git command isn’t available, either install git or download the files directly from GitHub.

Installing the GPU driver

Install the NVIDIA driver.
(Skip this step if it’s already installed.)

Installing Anaconda

Next, download and install the Windows 10 64-bit version from the Anaconda website.
Click Free Download on the Anaconda site and scroll down; you’ll see a screen like the one below.
Install the Windows installer on the far left.

Once Anaconda is installed, you should find “Anaconda Prompt” in your Windows app list.
We’ll use it to run the commands below to create the virtual environment, install the packages, and finally launch SLEAP.

Creating the virtual environment and installing

Next, run the following commands to create the virtual environment and install SLEAP.
The environment will be named “sleap”.
```
cd sleap
conda env create -f environment.yml -n sleap
```
Incidentally, the commands above work on computers with a GPU;
on machines without one, use the following instead.
```
cd sleap
conda env create -f environment_no_cuda.yml -n sleap
```
That completes the installation.

Checking that the installation worked

Activate the virtual environment and launch SLEAP with sleap-label.
```
conda activate sleap
sleap-label
```
You’ll need to activate the virtual environment (the conda activate sleap command) every time you reopen Anaconda Prompt.

After activating the environment, the prompt should change from (base) to (sleap).

If it launches, the installation was successful.

Checking version information for reporting in a paper

To check the version, activate the sleap environment and then run the following command.
```
python -c "import sleap; sleap.versions()"
```
This outputs:
```
SLEAP: 1.3.3
TensorFlow: 2.7.0
Numpy: 1.21.5
Python: 3.7.12
OS: Windows-10-10.0.22621-SP0
```
which shows the SLEAP version and related information.

To check whether SLEAP can use the GPU, run:
```
python -c "import sleap; sleap.system_summary()"
```
If the GPU is available, you’ll see output like the following.
```
GPUs: 1/1 available
  Device: /physical_device:GPU:0
         Available: True
       Initialized: False
     Memory growth: None
```
In future posts, I hope to walk through how to actually use SLEAP.

Bonus

For those who aren’t sure which computer to buy, I’ve started offering PC purchase consultations on Coconala!
あなたの要望に合わせてパソコンを選び、提案しますパソコン選びに困っている方々へ！様々な目的に対応できます！
I often pick out computers and give advice about them, and many friends have told me I could make money doing PC consultations.
That inspired me to give it a try!

I’ll recommend a machine that fits both what you want to use it for and your budget.
In particular, I’ve chosen and used many computers intended for machine learning.

And if you’d like, I can also advise you on what to look for the next time you buy a computer.

Computers I’ve picked out so far include lab analysis machines, everyday-use machines, game streaming machines, machines for incoming university students, machines that can run CAD for architecture students, and simple entry-level machines.

I use both Mac and Windows, so I can discuss and recommend either one!

Please feel free to make use of it.
2024-01-21
[Updated December 2022] How to Install YOLOX
What is YOLOX?

Among the many entries in the YOLO series, the latest one, YOLOX, was released in July 2021.

The YOLOX paper is available here (currently a preprint).
The GitHub repository is here.

YOLO is an object detection algorithm.
It is designed to run in real time and is extremely lightweight (inference alone can even run on a smartphone). It is also highly adaptable, and is widely used in IoT devices.

“Object detection algorithm” may sound complicated, but an easy analogy is a camera’s automatic face or eye focus.
A marker appears indicating “this is a face”; the same idea applies to other objects, and the algorithm can also classify what each one is.

There are many versions in the YOLO series: YOLO, YOLOv2, YOLOv3, YOLOv4, YOLOv5, and YOLOX.
It also runs not only on Python but on MATLAB as well, so you can adapt it to whichever platform you use.

Unlike the architecture used from YOLOv3 onward, YOLOX returns to the design of the original YOLO. It is reportedly faster and considerably more accurate.

Other blogs cover the details as well.

How to install YOLOX

Installation instructions are given on the GitHub page, but they are in English and do not cover using a virtual environment, so here I explain how to do it with an Anaconda virtual environment.

On both Windows and Mac, following the YOLOX manual exactly produced one error after another.
Errors will likely keep changing with future versions, so look them up as they come.

See below for how to use the conda command in the macOS Terminal.

Using Conda Commands on macOS

On Windows, I recommend using Anaconda Prompt.

Open the Terminal (Anaconda Prompt), and you will get a black window filled with text.
You should see (base) at the left of the last line.
This means you are not currently inside a virtual environment.

YOLOX will run without a virtual environment, but since environments can conflict with other packages, I recommend creating one.

First, enter the following code to create a virtual environment.
```
conda create -n YoloX python=3.8
```
YoloX is the name of the virtual environment; you can use any name you like.
You can also specify the Python version with python=3.8.

Then activate the virtual environment with the following code.
```
conda activate YoloX
```
If (base) on the left has changed to (YoloX), you are all set.

Installing CUDA

First, if this is your first time installing machine learning libraries and you plan to train YOLO on a GPU, install CUDA.
(Training on a CPU takes an absurdly long time, so I do not recommend it. Note that GPUs cannot currently be used on Apple Silicon machines.)
CUDA can be downloaded from the official site.

As for the CUDA version, I suggest checking which CUDA versions PyTorch supports and installing one of those.
PyTorch official site

To install an older version, click Download now and then, further down the page under Resources, use Archive of Previous CUDA Releases.

The latest version may work too, but for now, matching the version to PyTorch should let you run everything without trouble.

To check whether it is installed on Windows, open “Edit the system environment variables,” go to Advanced > Environment Variables, and see whether there is a path beginning with CUDA_PATH.
The number after the V indicates the version.

Installing PyTorch

Before installing, update pip.
Activate the virtual environment in Anaconda Prompt and run the following command.
```
python -m pip install --upgrade pip
```
Go to the PyTorch official site and choose the installation method that matches your environment.
One catch here is the Package field: be sure to select pip.
Installing with conda will cause errors later on.

Copy and paste the command under “Run this Command” to install it in your virtual environment.

Once the installation finishes, use the following command to check that PyTorch is installed.
```
pip list
```
As long as torch appears in the list, you’re good to go!!!
As a quick check, start python and run the following command.
```
import torch
print(torch.cuda.is_available())
```
If the output is True, the installation succeeded and torch is ready to run on the GPU.
(On a Mac or on a computer without a GPU it will return False, but as long as the import works you’re fine.)
If you get an error, go back and check the installation again.

Installing the packages required for YOLOX

Download (clone) the YOLOX files from GitHub.
If you can use Git commands, you can clone the repository with the code below.
```
git clone git@github.com:Megvii-BaseDetection/YOLOX.git
```
Next, move into the cloned folder and install the required packages.

First, edit the contents of requirements.txt in the cloned YOLOX folder as follows.
```
# TODO: Update with exact module version
numpy
#torch>=1.7
opencv_python
loguru
tqdm
#torchvision
thop
ninja
tabulate

# verified versions
# pycocotools corresponds to https://github.com/ppwwyyxx/cocoapi
#pycocotools>=2.0.2
#onnx==1.8.1
onnxruntime==1.8.0
onnx-simplifier==0.3.5
```
It seems that packages such as pycocotools cannot be installed with this command.

Once that’s done, save requirements.txt and run the command below in the terminal (Anaconda prompt).
```
cd YOLOX
pip3 install -r requirements.txt
pip3 install cython pycocotools
```
The downloaded YOLOX folder contains a file called “requirements.txt” that lists the required packages. The second command opens this file in read-only mode and installs the packages listed in it.
```
pip3 install cython pycocotools
```
This command installs cython and pycocotools.
It seems these have to be installed separately.

When you run this command, some of you may see the following error.
```
      building 'pycocotools._mask' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
```
Microsoft C++ Build Tools is apparently required by pycocotools. (Oddly enough, the error also appears on Mac, even though there is no Mac version.)
Windows users should install it just to be safe.

After installing it, try the installation again with the code below.
```
pip3 install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"
```
If that still doesn’t work, install it with the following command.
```
conda install -c conda-forge pycocotools
```
In general we’ve been installing these packages with pip, but since this one simply won’t install that way, we resort to installing it with conda.

Next, download yolox_x.pth.
The download starts as soon as you click the link.

Move the downloaded file into the YOLOX folder you cloned from GitHub.
This gives you the model used for detection in the demo, placed wherever you like.

Add the following lines to the top of tools/demo.py so that the yolox folder can be accessed.
```
#demo.pyの中身に書き込みましょう。
import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
```
Without this, you’ll get an error saying the yolox package cannot be found.

Then run the following command in the terminal.
This lets you check whether the demo runs correctly.
```
python tools/demo.py image -n yolox-x -c yolox_x.pth --path assets/dog.jpg --conf 0.25 --nms 0.45 --tsize 640 --save_result --device gpu 
#gpuで動かさない場合には、gpuをcpuに書き換えてください。
```
Replace –device gpu with –device cpu as appropriate for your setup.

If the following image appears in YOLOX/YOLOX_outputs/yolox_x/vis_res/20…….., everything is working.

Bonus

For those who aren’t sure which computer to buy, I’ve started offering PC purchase consultations on Coconala!
あなたの要望に合わせてパソコンを選び、提案しますパソコン選びに困っている方々へ！様々な目的に対応できます！
I often pick out computers and give advice, and many friends told me I could make money doing PC consultations.
Inspired by that, I decided to give it a try!

I’ll recommend a machine that fits how you plan to use it and your budget.
In particular, I’ve chosen and used many computers for machine learning.

And if you’d like, I can also advise you on what to look for the next time you buy a computer.

Computers I’ve picked out so far include analysis machines for the lab, everyday-use machines, game-streaming machines, machines for incoming university students, machines that can run CAD for architecture students, and simple all-purpose machines.

I use both Mac and Windows, so I can talk about and recommend either!

Please feel free to make use of it.
2022-11-15
[Updated December 2022] How to Install YOLOv5 [Object Detection Tool]
What is YOLOv5?

YOLO is an object detection algorithm. The name stands for You Only Look Once.
It differs somewhat from object tracking: it detects objects and identifies what they are.

YOLOのHPより引用(https://pjreddie.com/darknet/yolo/)

In applications such as autonomous driving, objects are detected and classified the way a human would judge them—as a person, a pet, or a car, for example.

The classification algorithm itself is complex, so I will skip the explanation here, but a quick search turns up a great many sites that describe it.

New versions of YOLO are currently released roughly every other year. We now have YOLO v1 through YOLOv5, and in August 2021 a new version, YOLO X, was announced.
The differences between versions involve substantial changes in the details, but generally speaking accuracy improves with each new release.

Here I will walk through the steps for installing YOLO v5—which offers ample accuracy and, thanks to the wealth of information now available, is easy to set up—and verifying that it works.

Installation environment

OS: Windows 10 (this also worked on macOS Monterey)
Python: 3.9.7 (3.6 or later)
CUDA: 11.3

How to install

Creating a virtual environment

Create a virtual environment using Anaconda.
If you do not have Anaconda installed, install it first.

Installing YOLOv5 pulls in quite a few packages, so I recommend installing it inside a virtual environment.
You can name the environment anything you like; here we will call it “Yolov5_env”.
Enter the following command in Anaconda Prompt.
```
conda create -n Yolov5_env python=3.9
```
This should create the virtual environment.
Activate it with the following command.
```
conda activate Yolov5_env
```
Installing CUDA

First, if this is your first time installing machine learning libraries and you plan to train YOLO on a GPU, install CUDA. (Training on a CPU takes an absurdly long time, so I do not recommend it.)
CUDA can be downloaded from the official site.

As for which CUDA version to use, I suggest checking which CUDA versions PyTorch supports and installing that version.
PyTorch official site

To install an older version, click Download now and then, on the resulting page, use Archive of Previous CUDA Releases under Resources further down the page.

The latest version may well work too, but for now, matching the version PyTorch expects should let you run everything without trouble.

To check whether it is installed on Windows, open “Edit the system environment variables,” go to Advanced > Environment Variables,
and look for a path beginning with CUDA_PATH.
The number after the V indicates the version.

Installing PyTorch

Before installing, update pip.
Activate the virtual environment in Anaconda Prompt and run the following command.
```
python -m pip install --upgrade pip
```
Go to the PyTorch official site and choose the installation options that match your environment.
One thing to watch out for is the Package field: be sure to select pip here.
Installing with conda will cause errors later on.

Copy and paste the command shown under “Run this Command” to install it into your virtual environment.

Once the installation finishes, use the following command to confirm that PyTorch is installed.
```
pip list
```
If torch appears in the list, you are good to go!
As a quick check, start Python and run the following command.
```
import torch
print(torch.cuda.is_available())
```
If the output is True, the installation succeeded and torch is ready to run on the GPU.
(On a Mac or a computer without a GPU you will see False, but as long as the import goes through, you are fine.)
If you get an error, go back and review the installation.

Installing YOLOv5

Download (clone) the YOLOv5 files from GitHub.
```
git clone https://github.com/ultralytics/yolov5
```
If you cannot use the git command, either install git or download the files from the YOLOv5 GitHub repository.

Next, run the following command in your virtual environment in Anaconda Prompt.
```
cd yolov5
pip install -r requirements.txt
```
The first command moves you into the yolov5 folder you downloaded from GitHub. If you installed it with the git command, this is all you need; if you downloaded it directly from the website, you will have to navigate to that directory yourself.

The downloaded yolov5 folder contains a file called “requirements.txt” that lists the required packages. The second command opens this file in read-only mode and installs the packages listed in it.

Once the installation is finished, let’s check which packages were installed.
```
pip list 
```
You should see that quite a few packages were installed.
That completes the installation of YOLOv5 for now.

Running It

To check that everything works, let’s try it with the data that comes with the yolov5 folder.
Open Anaconda Prompt, activate the virtual environment, and use the cd command to move to the yolov5 folder.

Then run the following command.
```
python detect.py --source ./data/images/ --weights yolov5s.pt --conf 0.4
```
detect.py is the program that performs object detection using YOLO.
With source, you specify the path to the folder containing the material you want analyzed.
Here I used the images included in yolov5.

With weights, you choose which model to use.
Here we use yolov5.pt.

conf sets the likelihood threshold at which an object is recognized.
This is a term you will run into often when studying machine learning.
If you are not sure, leaving it as is should be fine.

After running it, you should find the annotated images in runsdetectexp inside the yolov5 folder.
This is how object detection works in YOLOv5.

Errors I Ran Into

・ModuleNotFoundError: No module named ‘torch’

You may see this error when trying to run YOLO.
It means PyTorch is not available, i.e., the module cannot be found.

If you get this error, first check whether torch is installed in your virtual environment.
```
pip list
```
If torch is not listed, install PyTorch.
If it is there but things still do not work and you are not sure why, launch python, import PyTorch, and check whether it can actually be used.
```
import torch
torch.cuda.is_acailable()
```
If it is usable, True will be returned; if not, you will likely see the error above.

One possible cause of this error is that PyTorch was installed with the conda command.
If PyTorch is installed via conda while the other packages are installed via pip, the error above can occur.

In that case, uninstall torch with the conda command and reinstall it with pip.

Module-not-found problems often come from mixing up whether you installed something with pip or with conda.
Here we use pip throughout, but it is a good habit to always keep track of which command you used to install a package.

Links

Anaconda
CUDA
PyTorch
YOLOv5 GitHub
YOLO homepage

Bonus

For those who aren’t sure which computer to buy, I’ve started offering PC purchase consultations on Coconala!
あなたの要望に合わせてパソコンを選び、提案しますパソコン選びに困っている方々へ！様々な目的に対応できます！
I often pick out computers and give advice, and many friends have told me I could make money doing PC consultations.
That inspired me to give it a try!

I’ll recommend a machine based on what you plan to use it for and your budget.
In particular, I’ve chosen and used many computers for machine learning.

And if you’d like, I can also advise you on what to look for the next time you buy a computer.

Computers I’ve picked out so far include analysis machines for the lab, everyday machines, PCs for game streaming, PCs for incoming university students, CAD-capable PCs for architecture students, and simple all-purpose machines.

I use both Mac and Windows, so I can discuss and recommend either!

Please give it a try!
2022-02-23
How to Generate Continuous Pulse Waves with Arduino
Introduction

There are times when you want to output a continuous pulse wave with an Arduino: blinking an LED, producing a sound, using it as a timer, and so on.

It comes up often and seems simple at first, but the more you look into it, the deeper the topic gets.

In this post, I introduce several ways to output a continuous pulse wave with an Arduino.
Changing the timing with delay

The simplest and easiest approach is to switch the output ON and OFF using the delay function.
```
//pinはピン番号
void loop(){
    digitalWrite(pin, HIGH);
    delay(1000);
    digitalWrite(pin, LOW);
    delay(1000);
}
```
In the program above, the output alternates between HIGH and LOW.
Since delay is specified in milliseconds, delay(1000) waits for one second.

In other words, it is a program that turns on and off at 1 Hz.

However, using the delay() function to set a frequency has many drawbacks.
With this approach, changing the output duration—say, outputting a 60 Hz signal for 5 seconds—requires a for loop, which is inconvenient.

That said, because it is so simple, I recommend it when you just want to try something out quickly.
Using tone()

The tone() function is commonly used to generate buzzer sounds.
Official Arduino reference

This function lets you specify the frequency and the duration.
So, unlike the delay approach, you can specify the frequency directly without having to calculate it.
```
//pinはピン番号
void loop() {
     tone(pin,60);
}
```
You can write it as tone(pin, frequency) or tone(pin, frequency, duration).

The duration is given in milliseconds, so it is written the same way as delay.

The problem with this function is that it cannot produce frequencies of 31 Hz or below.
In other words, you cannot generate an output at, say, 1 Hz.

For frequencies above 31 Hz, such as audio tones, it makes setting the frequency very easy, and the code is far shorter and more accurate than using the delay function.
Using PWM output and changing its frequency

Using PWM output offers the most flexibility—and it is also the reason this topic gets so deep.

PWM stands for pulse width modulation.
For details, see Wikipedia.

The term alone does not tell you much, but in simple terms, PWM is a way of modulating an output by changing the duty ratio.
The official Arduino explanation is here.

Normally, you would set the brightness of an LED by changing the current.
But when the current is fixed and you still want to change the brightness, you blink the LED at a very high frequency (the flicker fusion threshold for humans is said to be around 30–60 Hz).

Normally the ON and OFF periods are 1:1 (a duty ratio of 50%), but what happens if you make it 4:1 (80% duty) or 1:4 (20% duty)?
The former looks bright, and the latter looks dim.

Modulating the output by changing the pulse width in this way is what PWM output is.

On the Arduino you can not only produce this output but also change the PWM frequency.
The idea behind this method is that by changing register settings—that is, the underlying parts of the Arduino—you can change the PWM output frequency.
By default, the output frequency is 490 Hz, or 980 Hz on some pins.

I will explain how to do this in detail in a future post.
Searching for “PWM Arduino change frequency” turns up plenty of explanations.

After reading through them about four times, it starts to make sense.

Basically, why not try these approaches and find the one that fits your own purpose?
2021-10-22
How to Install Python Video Annotator
What is Python Video Annotator?

Python Video Annotator is an application that lets you analyze recorded videos and annotate events within them along a timeline.

Researchers in neuroscience and ethology can use it to record videos of animals and then analyze and quantify their behavior.
For example, suppose you are recording mouse behavior and want to score events such as tongue extension, tail flicks, or ear movements.
When does each event occur in the video, and how long does it last?
Watching the video and logging everything in Excel each time quickly becomes overwhelming when you have defined many behaviors.
With a tool like this, you can annotate the video directly and export the timing and event information.

There used to be an open-source application for scoring animal behavior called VCode.
The problem, however, is that it no longer runs on current computer operating systems.

Python Video Annotator runs on modern PCs, retains the features you need, and can additionally be combined with external sensor data (such as pressure gauges) and behavior quantification tools like DeepLabCut, making it a very useful tool for researchers.
How to install

The official site describes the installation procedure in detail, but I could not get it to install properly on my machine. (Installing it directly may have caused conflicts with packages already present on my system.)
So instead I followed the approach described on the GitHub page: building a virtual environment with Anaconda and installing there.

It sounds complicated when written out, but the steps are actually very simple.
As of now (October 20, 2021) it does not appear to support the latest macOS (Big Sur 11.6), though this will likely be fixed soon.
For that reason, I will use Windows as the example here.

That said, once you have installed Anaconda on macOS and can use the conda command, the steps are essentially the same, so please refer to this guide once support arrives. (For details, see my previous post.)

Install Anaconda and open the Anaconda Prompt.
Then create a virtual environment and activate it.
```
conda create -n videoannotator python=3.6
conda activate videoannotator
```
Next, install the required packages.
```
pip install opencv-python-headless pyqt5==5.14.1 pyqtwebengine==5.14.0
```
Then install Python Video Annotator.
```
pip install python-video-annotator
```
Once the various processes finish, the installation is complete.

To launch it, activate the virtual environment first, then run the command.
```
conda activate videoannotator
start-video-annotator
```
If the software starts up, you are all set.
2021-10-20
How to Install DeepLabCut 2.3 [Updated December 2023]
In this post, I’ll walk through how to install DeepLabCut.

Reference sites
・Japanese-language sites
https://qiita.com/auditorycortex/items/1b3a55101cddf09553b2
↑Very clear. Following this should get you through just fine.

https://note.com/sakulab/n/n9caeb32d74d6
↑Includes screenshots

・DeepLabCut homepage
・GitHub

Installation

Environment
・Windows 10
・Verified on NVIDIA 2060 SUPER, NVIDIA 1080 Ti, and NVIDIA 3080
(did not work on NVIDIA 1060 SUPER)
・DeepLabCut 2.1
・NVIDIA Driver, latest version
・Anaconda3
・Python 3.8
・CUDA 11.8
・tensorflow-gpu 2.5

Steps
First, download the master files from the GitHub page.

You can download them via “Download ZIP” under Code.

Next, download and install the Windows 10 64-bit version from the Anaconda website.

Install the NVIDIA driver. (Skip this step if it is already installed.)

Scroll down on the DeepLabCut website, click “DOWNLOAS CONDA FILE” on the right,
and download DEEPLABCUT.yaml.

Create a DeepLabCut folder somewhere on your PC (the Desktop or a directory on the C drive is recommended), and make an “environment” folder inside it.
Then place the DEEPLABCUT.yaml file you just downloaded into that folder.
This is just personal preference, but I like to keep environment files in one fixed location, so this is how I do it.

Launch Anaconda Prompt as an administrator. (A terminal-like window full of white text on black, similar to Command Prompt, will appear.)
Anaconda Prompt is installed together with Anaconda, so you should be able to find it in your list of installed programs.

Now build a virtual environment using the DEEPLABCUT.yaml file you downloaded.
In Anaconda Prompt, enter
```
conda env create -f C:(DLC-GPU.yamlファイルの場所)DEEPLABCUT.yaml
```
You can check the location of the DEEPLABCUT.yaml file from the file’s properties, so look it up there and enter it.

When you run this command, various components will be downloaded.
After that, activate the virtual environment you created.
```
conda activate DEEPLABCUT
```
The prompt should change from (base) to (DEEPLABCUT).
If you get that far, you’re good for now.

Next, install CUDA and cuDNN.
```
conda install -c conda-forge cudnn
```
This will download suitable versions of the CUDA toolkit and cuDNN for you.

Then enter the following four commands to complete the installation.
```
pip install numpy
pip install deeplabcut
pip install imgaug
pip install torch
```
Update DeepLabCut, and the installation is complete.
```
pip install --upgrade deeplabcut
```
Once you’ve made it this far, try launching it.
```
conda activate DEEPLABCUT
python -m deeplabcut
```
Entering this will launch the DeepLabCut GUI.

What to do when DeepLabCut won’t run on the GPU

Sometimes you start training and think, “Huh? Isn’t this slow?”
That’s because DeepLabCut is running on the CPU when you intended it to run on the GPU.

A common cause is a version mismatch among CUDA, cuDNN, and tensorflow.

Packages like tensorflow keep getting updated to newer versions,
so things occasionally stop working.

For the latest version information, please check DeepLabCut’s GitHub.

That covers the installation process.
In upcoming posts, I plan to explain how to use it.

Bonus

For those who aren’t sure which computer to buy, I’ve started offering PC purchase consultations on Coconala!
あなたの要望に合わせてパソコンを選び、提案しますパソコン選びに困っている方々へ！様々な目的に対応できます！
I often pick out computers and give advice on them, and many friends have told me I could make money doing PC consultations.
That inspired me to give it a try!

I’ll recommend a purchase that fits how you plan to use the computer and your budget.
In particular, I’ve chosen and used a lot of computers for machine learning.

And if you’d like, I can also advise you on what to look for the next time you buy a computer.

Computers I’ve helped select include analysis machines for research labs, everyday-use PCs, PCs for game streaming, PCs for incoming university students, CAD-capable PCs for architecture students, and simple stopgap machines.

I use both Mac and Windows, so I can discuss and recommend either one!

Please feel free to make use of it.

Addendum 1

I’ve also written an article on how to install “SLEAP,” another behavior tracking tool like DeepLabCut.

SLEAP is every bit as capable as DeepLabCut, so please give it a try.

SLEAPのインストール方法[2024年2月更新]
2021-03-26
Bonsai as a visual programming language

Bonsai is a visual programming language introduced in a 2015 paper.
It allows you to acquire data from sensors and process it at the same time.

A key feature of Bonsai is that complex processing pipelines can be assembled easily, much like putting together a puzzle, and the results of that processing can be checked in real time.

Bonsai can also be combined with other analysis software (DeepLabCut, Open Ephys, BonVision, and so on) and with hardware (cameras, controllers, microcontrollers, etc.), making it highly extensible.

Because you can carry out complex analysis and processing intuitively without having to learn a programming language that looks intimidating, anyone can use it easily, and it broadens the range of experiments you can run.

The paper describing Bonsai is available here.
You can install Bonsai from here.

Image taken from the paper.
Each circle is called a node and represents an individual operation.

This is what the screen looks like when Bonsai is actually running (taken from the paper).
On the left you can search for nodes and add the one you need; in the central panel you arrange nodes to build up a pipeline while visualizing what it does; and on the right are the detailed settings for each node.

The windows popping up in the main panel show the execution status of each node; for a camera capture node, for example, the live camera image can be displayed.

It may look difficult at first glance, but seen this way it turns out to be surprisingly straightforward.

Now let’s actually install it.
That said, all you need to do is download it from this site.
Just open the downloaded “Bonsai-*.*.*.exe” file and run the installer.
Note that security software such as Trend Micro may block the installation.

Once Bonsai is installed, both Bonsai and Bonsai (x86) will appear in your list of applications.
For normal use either one is fine, but occasionally one of them will fail to run; in that case, trying the other one often works.

One drawback is that Bonsai currently runs on Windows only, and is not supported on Linux or macOS.
That may change in the future, but it is something to keep in mind.

From the next post, I’ll start actually using Bonsai.

2021-03-24

Hayato M. Yamanouchi

PhD student, Kamikouchi Laboratory, Nagoya University / JSPS Research Fellow (DC1)
Systems Ethology (Neuroethology × Informatics)

© 2026 Hayato M. Yamanouchi

On this site

Elsewhere

Contact details are on the About page.