The Curse of Dimensionality

August 17, 2018

Dov KatzYou can find more information about the following topic on my website

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces (often with hundreds or thousands of dimensions) that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience.” (

This is a funny definition if I’ve ever seen one. It implies that our everyday experience is low dimensional and therefore the data associated with it is easy to analyze and organize. But if the 3D world we live in was so simple, why is AI struggling to make sense of it?

The answer is that the above definition is both right and wrong. It’s wrong because the world we live in is very very very high dimensional. Sure, our space is three dimensional. But our visual perception of it is composed of millions of pixels refreshed multiple times per second. Therefore our visual data of the simple 3D space around us actually has millions of dimensions.

At the same time, this definition is exactly right. Or at least, it exposes a brilliant truth: it must be possible to organize the data pertaining to the world around us in a low dimensional form. Otherwise, how can any intelligent creature with finite resources (read: humans) make sense of it?

One of my favorite example is the dimensionality of a line. We all remember from school that a line can be described using the equation y=ax+b. In short, a line can be described using two parameters: a & b. Now, imagine tilting your head such that the line rotates to be parallel with the ground (or: x axis). Now, you can describe the line using a single number: y=c (or: the height of the line parallel to the x axis).

This simple example illustrates that perspective or representation is key. If you look at the data the right way, a high dimensional state space becomes lower dimensional. And, in the lower dimension representation, problems are easier to solve, and it becomes possible to make sense of a high dimensional world.

This article by Dov Katz found here:

Share this :

The Scientist in the Crib

August 09, 2018

Humanity’s advantage over the rest of the biosphere is out amazing abaility to learn and adapt. Alison Gopnik, Andrew N. Meltzoff and Patricia K. Kuhl. wrote a book called “The Scientist in the Crib” that discusses how babies learn.

Scientists often start with a crude, intuitive understanding of a phenomena. They then explore it by conducting some experiments. But, to conduct insightful experiments, scientists spend considerable amount of time learning what others have done. Typically, a scientific experiment is the last floor in a very tall building, sometimes extending decades or even centuries back.

So how do children learn? They begin with some crude knowledge provided by our genes. For instance, researchers have demonstrated that newborns already understand the concept of momentum. If you show a newborn a moving object that disappears behind a blanket, they expected it to reappear on the other based on its velocity and direction.

Of course, there’s only so much knowledge that can be hardcoded. Most things have to be learnt. Children appear to be spending much time playing. But, really, they are conducting serious experiments. What happens when I paint the wall? Will the glass break if I drop it? Is twisting the doorknob the right way to move the door?

Finally, there are teacher, also known as “parents”. The world is pretty complex. Making sense of it by running experiments alone isn’t practical. A parent, however, can give us a shove in the right direction to maximize our learning. This is what researchers refer to as “structure”. When you have a sense of how to organize things, discovery the logic behind a phenomena becomes easier.

The “Scientist in the Crib” does a good job describing these three components of learning: what we’re born with, what we learn from experimenting, and the role of teachers. It includes some fascinating examples demonstrating we are born with quite a bit of knowledge.

An interesting topic in the book is the brain’s plasticity. This is the notion that our brain is extremely adaptive. For instance, Japanese and English speakers are sensitive to different sounds. An adult speaker might not be able to hear a certain sound because of their cultural background. Babies, however, are able to hear all sounds regardless of their culture. This demonstrates brain plasticity — because culturally we don’t need to distinguish between two sounds, the adult brain adapts and can no more hear the unnecessary sound,

If you find this topic interesting, I highly recommend reading the book!

Check out my social media profile: for more information. Also, my personal webpage:

Dov KatzSource:

Share this :

Interactive Perception

August 07, 2018

I’ve been fascinated with robots forever. The idea of a machine that can think like a person but is physically far superior is intriguing. As I started doing research in robotics I quickly realized a surprising fact: building a brilliant machine is easy. Designing a machine that can do everything we do effortlessly is hard.

Here’s the truth about how far we’ve gotten in about 50 years of robotics: machines can beat almost every living person in chess, they can drive cars and navigate on Mars. And yet, they suck at opening a drawer, taking out some forks and placing them on the dinner table. This seems… weird. Why would robots excel at things so far out of what any 3 years old cares about and be so clueless in what every 3 years old can do effortlessly?

I believe the answer is that robots don’t get to grow up. They’ve never had to build from the grounds up, they never developed curiosity. Human children in the first three years of life are consumed by a desire to explore and experiment with objects. They are fascinated by causal relations between objects, and quite systematically explore the way one object can influence another object. They persistently explore the properties of objects using all their senses. For example, a child might gently tap a new toy car against the floor, listening to the sounds it makes, then try banging it loudly, and then try banging it against the soft sofa. This kind of playing around with the world, while observing the outcome of actions, is more than just play. It actually contributes to babies’ ability to solve the big, deep problems of disappearance, causality, and categorization.

Action and Perception

This explanatory drive tightly couples action and perception. This coupling was first observed in the 80s by the psychologist Gibson. Gibson’s research views perception as an active process, highly coupled with motor activities. Motor activities are necessary to perform perception and perception is geared towards detecting opportunities for motor activities. Gibson called these opportunities “affordances”. In my research in robotics I referred to this process as Interactive Perception.

Perceiving the world, making decisions, and acting to change the state of the world seem to be three independent processes. This is exactly why most people consider action and perception as separate. However, “enactive” approach to perception may be essential for surviving in a high-dimensional and uncertain world. Interactive Perception provides a straightforward way to formulate theories about the state of the world and directly test these theories through interactions.

For example, think about the first time a child encounters a pair of scissors. She has no sense of what this object does or how it works. Yes, she could spend some time looking at it and making educated guesses. But, what the child is most likely going to do is poke and probe it. This interaction will create motion, and this motion will make it easy to determine what scissors can do.

Interactive Perception imposes structure. It limits what needs to be perceived and explained. If we have hope of building robots that can do what toddlers do, I believe making them curious and letting them interact with the world to learn about it is essential.

And, maybe once they are expert toddlers, we can start thinking about sending them to preschool :-)

<iframe width="560" height="315" src="" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

Article First Posted on here

About Dov Katz

Dov Katz is currently a postdoc fellow at the Robotics Institute, Carnegie Mellon University.
I collaborate with Tony Stentz and Drew Bagnell.

Dubi Katz develops computer vision and machine learning algorithms for robotic manipulation.
My goal is to enable machines to intelligently interpret visual information in our everyday environments. This will enable exciting applications such as home robotics, space exploration, flexible manafacturing, autonomous driving, video understanding, and search & rescue robotics.

He received my PhD and MSc degrees in 2011 and 2008 from the University of Massachusetts Amherst, focusing on perception and learning for autonomous manipulation.
In 2004 I graduated with a BSc from Tel-Aviv University (Israel), double-majoring in computer science and electrical engineering.


Share this :

Dov Katz – Computer vision and machine learning expert

August 06, 2018
Dubi Katz, also known as Dov Katz has successfully positioned himself as the computer vision and machine learning expert. He was just a simple guy from Israel with a huge vision in mind and that is to let machine intelligently interpret visual information in the daily life; more of a machine and human interaction. Right after earning his bachelor’s degree in computer science at Tel-Aviv University in Israel, he moved to the United States.

He had his Ph.D. in the University of Massachusetts Amherst with a Ph.D. in robotics and his research was centered on vision and learning and the creation of the original position tracking system for the Oculus Rift VR system.  He became the vision engineer at Oculus before it was even acquired by Facebook. Because of his expertise and undeniable talent, he was promoted to senior computer vision engineer of Oculus VR. He led the Oculus’ computer vision R&D.

More information on Dov Katz
Share this :