Tracking Everything Everywhere All at Once

Hey, this is Denis. Today, we are going to talk about a new AI algorithm by Google, Cornell University, and UC Berkeley for object tracking. Apart from its state-of-the-art results and impressive demos, it has an interesting name: Tracking Everything Everywhere All at Once. Will it be an AI Oscar winner?

Before we start, let me introduce you to our friendly newsletter: Byte-Sized AI. This newsletter covers the AI business landscape, including entrepreneur stories, new AI projects, news, and market insights. If you are interested in the latest AI developments, feel free to subscribe below:

Let's start with why we need object tracking. As humans, we can easily track objects in real life, even if they are partially occluded, like cars on a street. We can still follow their movements and predict their future. If we are driving a car, we can estimate whether a pedestrian is crossing the road or if that red car is going to turn right. If we play tennis, we can predict where and when a ball is going to land.

In computer vision, it is essential to have the same level of scene understanding. If we build a self-driving car, we need it to track all cars and pedestrians around it. If we build AI tennis analysis, we need ball tracking. And this is why a new algorithm is so important.

The new model works as a point tracker, meaning that you need to select the point on the first frame and it will automatically track it. You can select many points on the same image for tracking.

Although it is of very good quality, this model has a couple of limitations. First, it is really slow and far from real-time camera feed processing. Second, it sometimes struggles with fast movements and thin structures.

I see this tracking algorithm as having a number of important applications. Here are a couple of examples:

1. Ball/Player tracking in football, tennis, and other sports for AI analysis. For example, one can analyze tennis racket trajectory and predict the type of movement.

2. Together with object detection, tracking can be used to analyze human behavior in big malls or supermarkets. Some startups do "Google Analytics for shopping centers," showing different statistics on user behavior.

Project page: https://omnimotion.github.io/

One more thing. Nowadays, AI is becoming increasingly important. More and more companies are moving in that direction. If you or your company are interested in integrating AI into your operations, I can help.

Here are some of the ways I can assist you:

• Advising on how to integrate AI into your company.

• Recommending models and providing guidance on how to train and deploy them.

• Providing advice on how to start an AI startup, including details on the technological stack.

Why choose me for a consultation?

• I hold a PhD degree in Generative AI.

• Over the past four years, I've been building AI startups in Computer Vision, including reflexhealth.co and quickpose.ai.

To be transparent, my goal is to be helpful while also earning money. That's why my initial consultation fee is $100 per hour.

If you're interested, just ping me on Twitter.

Cheers,
Denis