What is the biggest gap in robotics research?

There are two major gaps that we should at least investigate as part of the process of defining robotics as a discipline.

1.     We don’t have positive laws.  Our history is still largely trial and error and scavenging from other disciplines, rather than the development of foundational design principles or laws governing the creation or operation of robots in general.

We do have recognizable (tongue-in-cheek) negative laws, roughly equivalent to Murphy’s Law.

The First Law of Robotics:  You’re wrong (you set the parameters wrong, the user doesn’t need the robot to do whatever it was you thought they wanted, your model of the environment was wrong, etc.);

The Second Law of Robotics:  On a good day, half the robots in your lab are actually working (alternate version:  Your robot is working a little over (or under) half the time.  This has actually been demonstrated in two longitudinal studies);

The Third Law of Robotics:  All demos are faked or staged in some way (if you want to demonstrate a particular behavior or capability, you need to stage the demo so that that specific behavior or capability is required);

The Fourth Law of Robotics:  The likelihood of something going wrong with the demo is proportional to the importance of the people you’re putting it on for;

Corollary to the Fourth Law:  The thing that goes wrong with your demo will be unrelated to what you are trying to demonstrate, but will completely prevent you from demonstrating the important part of your work.

2.     We have trouble defining the space in which we work.  For example, autonomy means to operate without outside control, but the human intervention axis is demonstrably not equivalent to the autonomy axis.

Since one of the most common categorization tools for levels of autonomy does not make this distinction clear, a simple example is provided.

We can define four extreme types of robots:

• Clockwork Automaton:  robots that do the same thing, every time you start them, without any feedback from sensors (or even communications) about their environment.  They are incapable of changing their actions, but require no human intervention once activated.  A wind-up toy is the simplest example of these, but extremely complex mechanisms have been built.
• Remote Control:  robots that need to be given explicit direction from a human operator at all times; without continuous input from the operator, they will simply stop moving.
• Mixed Initiative: robots that interact with their human operators to determine the best course of action – these are autonomous systems that can make suggestions to their operators and sometimes act on their operators’ suggestions and sometimes act on the basis of their own decisions.
• Autonomous: robots that can perform tasks robustly in unconstrained environments  based on local sensing without any human intervention.  These include everything from a Roomba(TM) to Chappie.

On the human intervention axis, clockwork and autonomous robots lie at the “less” end of the scale, while remote control and mixed-initiative systems both involve significant human interaction.  On the autonomy axis, clockwork and remote control robots lie at the “none” end of the scale while autonomous and mixed-initiative systems demonstrate significant autonomy.

Mixed initiative robots require less interaction than remote control robots, but the quality of that interaction requires more expertise and understanding of the problem and the environment.  The purpose of the human in a remote control system is to provide the low level feedback loop in the controller linking the sensory inputs and the motor outputs.  The purpose of the human in a mixed initiative system is to work with the robot to jointly develop a solution to a complex problem and to provide the additional context not available to the robot.  No robots have no interaction with a human.  Even clockwork automata and fully autonomous robots have at least one interaction with a human – neither can start until a human activates a mechanism or sends a command.

But the core problem associated with both gaps is that the space represented by this diagram involves many, many different kinds of robots, performing many, many different kinds of jobs.

In some ways, this is like mathematics — both robotics and mathematics involve the study of many apparently disparate approaches to understanding the world, unified by a philosophical approach to problem solving, and both result in tools that are useful to many disciplines outside their own.

In mathematics, we have logic and proofs, algebra, geometry, and calculus, set theory and  probability.  In robotics, we have physical platform designs, perception, actuation, and manipulation, planning and decision making.

In mathematics, we have a philosophical approach to understanding the world that revolves around the definition of number and the definition of object.  In robotics, we have a philosophical approach to understanding the world that revolves around the actions required to achieve a goal.

In mathematics we have interest rates and statistical analysis supporting the financial industry, we have Fourier transforms describing electrical and acoustic waveforms, enabling us to listen to recorded music and communicate with people in space and around the world, and we have boolean logic, providing the underpinning for all the binary manipulations occurring in every computer we make.

In robotics, we have vacuum cleaners in peoples’ homes, we have robots that package, robots that assemble, and robots that deliver, and we have robots that swim and fly and tunnel, gathering information about hurricanes and pipes and volcanos.

What mathematics has that robotics doesn’t have is clearly spelled out foundations.  In mathematics, we know that before you can do mathematics rather than arithmetic, before you can really think like a mathematician, you need a bunch of fundamental tools.  Before you can really understand calculus, it’s important to understand algebra, and before you understand algebra, it’s important to understand the basics:  addition, subtraction, multiplication, division, fractions, and graphs.  The mathematics community largely agrees on what those fundamental tools are, just like the electrical engineering community largely agrees that Ohm’s Law is a fundamental tool in understanding electricity.

So I suppose I should change my answer:  the largest gap in robotics as a discipline is the lack of agreement on what constitutes the fundamental tools of our trade.

In every research project, you eventually run into a wall.  The wall can take many specific forms, but in general it represents a point at which it seems like your idea can’t work.  There are at least four possible responses.

First, you can walk away.  You can abandon your approach to the problem, backtrack until you find an alternate path, and follow that path to its wall.

Second, you can keep battering at it until it gives way.  This only works if the wall isn’t a function of fundamental mechanisms of physics or mathematics.  Problems that are a function of processing power or available memory can be susceptible to this approach — if you wait long enough or have enough money, someone will build a computer capable of implementing your idea.  Sometimes the brute force approach is sufficient.

Third, you can revolve the problem in your mind, looking at it from all sides.  You can investigate other, apparently unrelated fields, you can attend talks on tangentially related work, and then you can come back to the problem and revolve it in your mind some more.  You can go talk to the people who will be using it, to see if there are any underlying assumptions you’ve missed, or something you’ve misunderstood.  And eventually you’ll find a path through the brambles around the side of the wall and come out on the other side.  You haven’t gained any understanding of the wall itself, but you have managed to find a new way to think about the problem, and you are likely to end up with an effective solution.  This is what happened in robotics in the 1980s.

Until that point, the software side of robotics was focused primarily on artificial intelligence approaches.  When they tried to make their robot operate even in a relatively simple environment, they hit a wall in terms of the required processor power because their interpretation of the problem assumed a high degree of object recognition and understanding of the environment.  Brooks’ key insight was that it is not necessary to understand the world in order to function within it.  The robot doesn’t need to differentiate between tables and chairs in order to avoid them.  It only needs to differentiate between floor and not-floor.  He sidestepped the wall (capturing and interpreting the complexity of the environment) and found a path through the brambles (abstracting the complexity of the world into only the things the robot needs to know).

Fourth, you can disassemble the wall, brick by brick, until you understand it.  This is the most theoretically sound approach, and often results in the cleanest solutions.

After the Wright brothers side-stepped their wall by discovering that wings needed to be rounded instead of sharp on the leading edge, engineers disassembled the leading-edge-shape wall until they thoroughly understood it, leading to modern aerodynamics.

The ViCoRob lab at the University of Girona ran into the wall of environmental factors when they attempted to stitch together imagery of the same undersea environment at different times.  Instead of hitting the problem with more and more computer time, or sidestepping the problem by mandating specific vehicle behaviors, they disassembled the wall into many individual image processing elements, each dealing with a specific confounding factor.  They compensated for brightness variation due to light placement, they compensated for particulates in the water, they compensated for chromatic variation as a function of distance, they compensated for bright and dark spots caused by light refracting through ripples on the surface, they compensated for vehicle motion estimate errors, and they compensated for scaling factors between images.  They took apart the wall, understanding it as they went, and were able to create stunningly beautiful images of large swaths of sea floor.  This approach is generally expensive and time-consuming.

What to do when you hit a wall is obviously going to be a function of many factors — how much time you have, how complex the problem is, how much money and hardware you have, and how you intuitively approach problems.  Some people will naturally gravitate towards each of these approaches, and an approach you’re good at is likely to be more effective for you than an approach you’re not good at.  It is, however, worthwhile to attain at least limited skill in each approach, if only so that you are aware of alternatives when your preferred approach fails.

What is the core of Robotics as a discipline?

You could argue that robotics is an engineering discipline, because we work on the design, building and use of robots.

You could also argue that robotics ought to be more a science than an engineering discipline alone, since we provide new kinds of tools and models for other disciplines in much the same way mathematics does (with the admittedly large difference that our tools exist as physical objects).

Or you could argue that mathematics and robotics lie on either side of engineering, with mathematics providing the inputs to engineering and science and robotics coming out the other end as the ultimate synthesis of engineering and scientific fields.

Or you could even argue that robotics sits to the side of traditional classifications – roboticists tend to have significant experience in a wide range of fields not limited to engineering.  Those fields can include electrical engineering, mechanical engineering and computer science, as well as some degree of familiarity with the fields in which their robots were supposed to be operating, with deep expertise in individual pockets of each of those fields dependent on the details of the robots and the problems in question.  When a roboticist moves from problem to problem within robotics, they can view all the breadth of their expertise as potential opportunities to develop new pockets of deep expertise in some new area within that breadth, rather than something outside their expertise to be hired out to someone else.

An individual might start working on a project requiring their expertise in visual sensors for ground robots in urban environments, and then discover a need to apply that skill to working on visual sensors in aerial environments.  Their next project might involve machine learning to support improved perception, which could in turn lead to work on machine learning for decision making.  As part of that work, this individual might move to the design or improvement of specific behaviors the robot is deciding between, from there to low level control of some actuator on the system to improve a specific behavior, and finally to a mechanical redesign of the arm, leg or propeller.

That individual will have worked through electrical engineering to computer science to a different branch of electrical engineering to mechanical engineering, but in context, the path is instead a seamless trajectory through robotics.

And every element of that path is more fundamentally part of robotics than part of the discipline in which it is currently based, because every element of that path is rooted in the core of robotics as a field:  how active machines interact with the world.  Robotics is not just about designing the robot; it is about understanding the environment in which the robot is expected to operate.  It is not just about building the robot, it is about ensuring it is capable of the desired range of behaviors.  It is not just about using the robot, it is about understanding and improving the relationship between the robot and the user.

We worry about signal processing not as an abstract tool in the electrical engineering toolbox that can be used to clean up signals for human consumption, but because it enables the robot to obtain better information about its environment.

We worry about machine learning because the world is a large and unpredictable place, and we can’t always constrain the environments our robots need to operate in to the environments we can fully model and understand (and we can’t fully model and understand any but the simplest environments right now).

We worry about the individual behaviors because we build robots to do things, and they should be able to do those things well.

We worry about low level control because that determines how effectively it can act in its environment, and we design new tools and legs because we want it to be capable of doing new things.

Every step in that path is about the robot performing a task, in a complex and potentially unknown environment.  That is the core of robotics.

What would enable faster progress in robotics research?

Part 1:  Librarians.

There are several answers to this question, but one of the most important things robotics needs is the ability to find existing solutions, and one key element of that is providing cataloging tools.

Every robot is built up of bits and pieces of other peoples’ algorithms.

The researchers focused on a given problem know what the state of the art in that area is.  But each robot is an assemblage of components – no matter which piece a given researcher is focused on, the rest of the robot must be present in order for that piece to be tested.  Unfortunately, no one researcher has time to be an expert in every component of a robot.

If a new SLAM loop closure algorithm needs to be demonstrated and tested on a robot, it will probably utilize a low level controller provided by the manufacturer and a sensory perception algorithm inherited from a previous student.   The designer of the loop closure algorithm is unlikely to have time to ensure that every component of his or her robot is using the optimal algorithm for that task, tuned properly for both the robot and the environment.

If the researcher is focused on multi-vehicle cooperation, the obstacle avoidance algorithm is likely to be whatever came with the robot, or whatever might have been installed on the robot by the last graduate student who used it, or the algorithm developed by the last researcher in that lab who worked on that problem, rather than the algorithm most suited for that robot operating in that environment.

Of course, we also need tools that will let us reuse algorithms across robots, and progress is being made in that area, and we need tools that will tune algorithm parameters so that for any given combination of algorithm and robot and environment, the algorithm is providing the best performance possible.  But right now, we don’t even know what algorithms are available and which ones are most suited to which applications without extensive research and a good dollop of luck.

Luck oughtn’t to be necessary, but unless you’re an expert in a given area, it’s not always abundantly clear that you have even managed to figure out the correct keywords to let you find the algorithm you’re looking for.

If we go to the ROS archive (http://ros.org) and search for SLAM, we find over 20 packages submitted by 12 people or groups, while “navigation” brings up 11 metapackages and 51 packages, none of which use “SLAM” in their text.  There is no way for the non-expert to tell which algorithm he or she should use. “avoid” brings up 12 packages, most of which don’t even implement obstacle avoidance but merely have the word “avoid” used in their descriptive text.  Neither “survey” nor “coverage” bring up any results, even though many algorithms exist outside of ROS that address those problems.

There’s a reasonably comprehensive list of MOOS modules available at http://oceanai.mit.edu/moos-ivp/pmwiki/pmwiki.php?n=Site.Modules, but the organization is done by hand and the modules primarily address marine vehicle applications.

The options for compiled code with a reasonable level of certainty that it will work are limited. Even finding algorithms through technical papers is distressingly hit or miss.  If you happen to choose the wrong keywords, it’s entirely possible you’ll fail to find the most relevant papers.  Non-native English speaking researchers are at a significant disadvantage, and sometimes form clusters of researchers working on problems in parallel with the native English speakers, since they’re using a different set of keywords to describe the same thing.  So if you don’t know precisely what keywords to use, you’ll probably find a community of researchers, using the keywords you think are intuitive, with a variety of solutions addressing your problem.  You have no way of knowing whether this community represents the state of the art or whether this community believes it represents the state of the art but itself missed the correct keywords.

Even within the English-speaking world, robotics is a parochial discipline.  Researchers in one branch of robotics may use entirely different keywords and be entirely ignorant of existing work addressing the problem they’re looking for.  Researchers in industrial robotics and researchers looking at problems facing commercial fixed-wing aircraft are unlikely to know what solutions the ground vehicle or the underwater vehicle community have developed.   Researchers with a background in controls will assume that any half-decent undergraduate robotics textbook should contain recursive Newton-Euler equations, while researchers with a background in behavior-based robotics still manage to find roboticists that have never heard of Braitenberg vehicles.  Without experience in controls, there is no guarantee an artificial intelligence specialist will know to look for recursive Newton-Euler equations to control their robot, and without experience in behavior-based robotics, a controls specialist is unlikely to consider a Braitenberg approach to behavior generation (the exception).

Robotics needs some form of taxonomy or Library of Congress organizing principle so that researchers can find the relevant behaviors and algorithms for their problem and robot.

We need librarians.