The creation of a robot chef represents a grand challenge for the field of robotics. Cooking is one of the most important activities that takes place in the home, and a robotic chef capable of following arbitrary recipes would have many applications in both household and industrial environments.

The kitchen environment is a semi- structured proving ground for algorithms in robotics. It provides many computational challenges, such as accurately perceiving ingredients in cluttered environments, manipulating objects, and engaging in complex activities such as mixing and chopping. We envision a robotic chef, the BakeBot, which can collect recipes online, parse them into a sequence of low-level actions, and execute them for the benefit of its human partners. We are working towards this goal, by combining techniques for object perception, manipulation, and language understanding to develop a novel end-to-end robot system able to follow simple recipes and by experimentally assessing the performance of these approaches in the kitchen domain.