1st Summer School on Cognitive Robotics

Dates: June 12-16, 2017

A course on creating capable cognitive robots, by combining mobility and decision layers

The goal of this unique summer school is to help catalyze the next generation of planning researchers in AI and robotics, to work together to develop robots that are capable at both the task and motion levels. Our school will help students become equally conversant in the methods of cognitive systems and robotics, and will help them to build bridges between robotics and AI methods. The intended audience for this summer school includes top graduate students, postdocs and junior researchers, from both the cognitive AI and Robotics disciplines.

The summer school will center around a common robotic execution architecture that unifies the many methods presented, together with daily robotics labs and a grand challenge for the week. Advanced tutorials will be offered on those major elements of planning and perception that are required to develop capable autonomous systems, and will center around the daily themes of robust execution, motion planning, activity planning, perception and manipulation, and planning under uncertainty and risk. Tutorials alone, however are not enough to have lasting impact. Hence we complement them by a novel robotic architecture and suite of labs.

A feature of our summer school is that we offer students a comprehensive autonomy architecture, to use and take home. This architecture includes a decision layer architecture, which the cognitive AI community uses to combine decision-making components into a planning and execution system, coupled to a mobility layer architecture, which the robotics community uses to combine its manipulation and perception components. The architecture that we provide in this summer school uses Enterprise for the decision layer, and the Robotics Operating System - ROS for the mobility layer. The idea behind offering a decision layer architecture is to tie the cognitive systems community together, much in the way ROS and similar architectures have done for robotics, and to bring these two communities together, through the hybrid Enterprise/ROS architecture.

Leveraging this architecture, the summer school includes daily labs, centered around the daily themes and tutorials, and a grand challenge that helps the students apply the combined methods that they learn during the summer school. Students and researchers will then be able to take the hybrid architecture and a corresponding integrated tool set home, as a catalyst for future research, studies and collaboration.

For more details about the summer school program, the tutorials, speakers, venue, directions, addresses, maps, and the rooms where the tutorials and labs will take place, please take a look at to the summer school booklet.

Important Dates

May 17, 2017 (closed) Deadline for early application and grant requests
May 22, 2017 Start of notification of admission and grant allocations
May 30, 2017 Registration deadline
June 12-16, 2017 Summer school


The Massachusetts Institute of Technology (MIT) Stata Center
Cambridge, MA

MIT is traditionally known for its advanced research and education in the physical sciences and engineering, biology, economics, linguistics, and management. It is often cited as among the world's top universities in the world. The Stata Center building in particular, where the school will be offered, is home to the Computer Science and Artificial Intelligence Laboratory (CSAIL), the Laboratory for Information and Decision Systems (LIDS), and the Department of Linguistics and Philosophy. Its striking design (featuring tilting towers, many-angled walls, glass ceilings and whimsical shapes) and its open-concept common areas provides a unique and exciting environment for students to collaborate and to be creative.

MIT stretches along the banks of the scenic Charles River, separating Cambridge from Boston. Cambridge and Boston are both vibrant areas, and include restaurants, theaters, museums, and shopping plazas, easily connected through a world-class public transportation system. Within a short distance are numerous other schools, including in or near Boston: Harvard, Boston College, Tufts, as well as the University of Connecticut (Storrs), Yale University (New Haven), and several campuses of the University of Massachusetts. There are also numerous high-tech firms in the area, most notably but not exclusively in the Boston area.


Please refer to the summer school booklet for more details about directions, addresses, maps, and the rooms where the tutorials and labs will take place.

Boston has its own international airport, the Logan International Airport (BOS), which is located at a convenient two miles from the city center, with several public airport transportation options from and to downtown, suburban locations, and Cambridge area. There are approximately 40 airlines that serve Boston. Boston is also conveniently accessible by train and car from other popular international airports, such as John F. Kennedy International Airport (JFK) and LaGuardia Airport (LGA).

From the Logan International airport, MIT is conveniently accessible by a free subway ride, as well as by taxi or car. Taxi fare from the airport is about $35 – $40 and a ride will take between 15 minutes (during non-rush hour) to 30 minutes (during rush hour). The subway fare from any two points in the network is $2.25 (Charlie Card) or $2.75 (cash) and a ride from Logan to MIT will take, under normal conditions, about one-half hour.


Summer school students, including graduate students, postdoctoral researchers, junior researchers, will have the affordable option of staying at a well-located hostel in downtown Boston, the Hostelling International Boston.
For those staying at the hostel, check-in is available after 3:00 p.m. and check-out is prior to 11:00 a.m. If you plan to arrive early/depart late, the Front Desk will be able to store your baggage for you. Please tell the Front Desk agent that you are with the MIT group, present a government-issued ID and you will receive your key.

As an alternative option for students, the Cambridge and Boston area has numerous hotel and Airbnb options at a range of prices. There are many hotels within 5 – 10 minutes walking distance ( ≤ 1 km) from the MIT Stata Center, such as Boston Marriott Cambridge, The Kendall Hotel, Le Meridien, and the Windsor Inn.

Schedule Overview

Summer school schedule. List of tutorials, speakers, and links to presented material.
Live streams are also available.

Day 1 - Monday, June 12
Theme: Robust Execution: Estimation, Monitoring and Scheduling
Room 32-D463 (Star) 4th floor, Stata Center, MIT

Day 1 Video

8:30am - 9:30am Registration
9:30am - 10:00am Introduction 1: Architectures for Autonomy
Speaker: Brian Williams, Massachusetts Institute of Technology
10:00am - 10:30am Break1
10:30am - 12:00pm Tutorial 2: Self-Monitoring, Self-Diagnosing Systems
Speaker: Brian Williams, Massachusetts Institute of Technology
12:00pm - 1:00pm Lunch
1:00pm - 2:30pm Tutorial 3: Temporal Networks for Dynamic Scheduling
Speaker: Luke Hunsberger, Vassar College
2:30pm - 3:00pm Break2
3:00pm - 6:00pm Lab 1
6:00pm - 7:00pm Open lab hours

Day 2 - Tuesday, June 13
Theme: Motion Planning
Room 32-D463 (Star) 4th floor, Stata Center, MIT

Day 2 Video

8:30am - 10:00am Tutorial 4: Sampling-based Motion Planning
Speakers: Mark Moll and Lydia Kavraki, Rice University
10:00am - 10:30am Break1
10:30am - 12:00pm Tutorial 5: Single-Robot and Multi-Robot Path Planning with Quality Guarantees
Speaker: Sven Koenig, University of Southern California
12:00pm - 1:00pm Lunch
1:00pm - 2:30pm Tutorial 6: Trajectory Optimization for Underactuated Robots
Speaker: Scott Kuindersma, Harvard University
2:30pm - 3:00pm Break2
3:00pm - 6:00pm Lab 2
6:00pm - 7:00pm Open lab hours
7:00pm Social Dinner

Day 3 - Wednesday, June 14
Theme: Activity Planning (AI)
Room 32-D463 (Star) 4th floor, Stata Center, MIT

Day 3 Video

8:30am - 10:00am Tutorial 7: Classical Planning@Robotics: Methods and Challenges
Speaker: Joerg Hoffmann, Saarland University
10:00am - 10:30am Break1
10:30am - 12:00pm Tutorial 8: Planning in Hybrid Domain
Speaker: Andrew Coles, King's College London
12:00pm - 1:00pm Lunch
1:00pm - 2:30pm Tutorial 9: Planning Concurrent Timelines
Speaker: David Wang, NuVu
2:30pm - 3:00pm Break2
3:00pm - 6:00pm Lab 3
6:00pm - 7:00pm Open lab hours

Day 4 - Thursday, June 15
Theme: Perception and Manipulation
Room 32-G449 (Kiva) 4th floor, Stata Center, MIT

Day 4 Video

8:30am - 10:00am Tutorial 10: Multi-vehicle Routing with Time Windows
Speaker: Phil Kilby, CSIRO Data61 / Australian National University
(Material: check section 1 of Lab 4 material)
10:00am - 10:30am Break1
10:30am - 12:00pm Tutorial 11: Generative Models for Perception
Speaker: John W. Fisher, Massachusetts Institute of Technology
12:00pm - 1:00pm Lunch
1:00pm - 2:30pm Tutorial 12: Fundamentals of Robotic Manipulation and Grasping
Speaker: Alberto Rodriguez, Massachusetts Institute of Technology
2:30pm - 3:00pm Break2
3:00pm - 6:00pm Lab 4
6:00pm - 7:00pm Open lab hours

Day 5 - Friday, June 16
Theme: Planning with Uncertainty and Risk
Room 32-D463 (Star) 4th floor, Stata Center, MIT

Day 5 Video

8:30am - 10:00am Tutorial 13: Probabilistic Planning
Speaker: Shlomo Zilberstein, University of Massachusetts Amherst
10:00am - 10:30am Break1
10:30am - 12:00pm Tutorial 14: Localization and Mapping
Speaker: Nicholas Roy, Massachusetts Institute of Technology
12:00pm - 1:00pm Lunch
1:00pm - 2:30pm Tutorial 15: Risk-bounded Planning and Scheduling
Speaker: Brian Williams, Massachusetts Institute of Technology
2:30pm - 3:00pm Break2
3:00pm - 6:00pm Lab 5
6:00pm - 7:00pm Open lab hours

Live Stream

How to Connect:

We will be streaming the tutorials live during the summer school from 8:30am to 2:30pm (EST). MIT CSAIL provide a steam service at https://replay.csail.mit.edu/ through Youtube channels.

You will be able to select the seminar room you want to watch. Tutorials preseneted on Monday, Tuesday, Wednesday and Friday will be in the room Star, while Tutorials on Thursday will be presented in room Kiva. Here are the specific links to the live stream in each room:


Lab 1 - Monday, June 12: Enterprise/ROS Familiarization and Robust Execution
MERS lab - 2nd floor, Stata Center, MIT Building 32


Through the summer school you will be using the Enterprise/ROS architecture, combined with various planners, to control robots. Enterprise manages decision making, while ROS manages mobility. The goal of this lab is twofold. The first goal is to start familiarizing you with the Enterprise/ROS architecture (henceforth Enterprise) and its different components. The second goal is to give you a first experience with the Enterprise system, applied to a simple, single-robot scenario. In the first section of the lab, we will walk you through a step-by-step setup of the code base on your laptops. In the second section, you will be writing an RMPL script, to program a robot to finish a set of tasks in a simulated environment.

Introduction Slides
Instructions and Material:

You should start by reading the lab description document: Lab1: Robust Execution. Then follow the instructions for environment setup to download and set up mpex and the simulation environment.

Lab 2 - Tuesday, June 13: Incorporating Trajectory Planning for Autonomous Vehicles
MERS lab - 2nd floor, Stata Center, MIT Building 32

Lab 3 - Wednesday, June 14: Incorporating Activity Planning
MERS lab - 2nd floor, Stata Center, MIT Building 32


In this lab we add a little excitement, by breaking into teams that participate in a friendly modeling competition! Teams will use “generative” planners that take as input an initial state and goal, together with a set of operators that are specified in terms of preconditions and effects. The planner then generates an action sequence that carries a system from initial state to goal. A number of increasingly expressive planners have been developed, based on heuristic forward search, which can quickly generate lengthy plans, without the need for additional search guidance.

The goal of this lab is to provide hands on experience with modeling generative activity planning problems in PDDL, and with several state of the art heuristic forward search activity planners. Teams model three planning problems using RMPL and PDDL, and solve them with different activity planners. The first problem covers RMPL and how to use it to model actions with durations, conditions, and effects. We translate our RMPL to PDDL and then use an activity planner to find a solution. The second covers temporal planning problems, in which actions have different durations, and problems have deadlines. The third covers mixed continuous/discrete planning problems, in which state has continuous valued variables, as well as discrete propositions. Each participant is given a specified planning problem to solve, while their team collectively solves all three problems. Given a problem description, participants model their problem using the subset of PDDL suitable for that problem (classic, metric time, hybrid). Finally, the participants evaluate their model on the respective activity planner, expressive enough for their problem, and check their performance.

Introduction Slides

Lab 4 - Thursday, June 15: Manipulation and Multi-vehicle Routing
MERS lab - 2nd floor, Stata Center, MIT Building 32


The goal of this lab is provide participants experience with planning at two very different scales, f ine-grained manipulation, and large scale coordination. In doing so, the students will employ a range of planning paradigms, including state-based and sample-based search, optimization, and constraint programming. First, participants will gain hands on experience with modeling and solving multi-vehicle routing problems, using a constraint programing framework. Second, they.will become familiar with some of the state-of-the-art motion planning for robot manipulation tasks. In the first section of the lab, participants will be given a multi-vehicle routing problem, will encode the problem as a constraint model, using the Minizinc constraint programming language, and will employ one of the Minizinc solvers to solve the model. This exercise of planning as constraint programming and optimization, offer a nice contrast to the state-space planning techniques and PDDL modelling language, exercised during the Day 3 lab. In the second section of the lab, participants will work on a block-stacking problem, using a manipulator that is controlled by different motion planning systems. The objective is to rearrange a set of blocks with different shapes, in order to achieve different goal positions. Participants will first perform this task using a simulation environment, and will then be able to test their solutions with a robotic arm.

Instructions and Material:

Section 1: Instructions and Supporting Materials

Lab 5 - Friday, June 16: Challenge - And all comes together …
MERS lab - 2nd floor, Stata Center, MIT Building 32

Invited Speakers


In a nutshell, the registration process for candidate attendees starts by filling out an online form and uploading a couple documents. The organizing committee will then evaluate the applications and select nominally 50 participants based on relevance. Those selected will receive a notification of acceptance starting May 22, including directions for finalizing the registration and for paying the registration fee. We will continue to accept applications until the participation is full. More details below.

How to Apply:

To apply to the summer school, fill out the online application form (see link below). Please include the following:

If you are requesting financial support for transportation to the venue, additional documentation is required in the online form. See the financial support section below.

To submit your application, please go to the Online Application Form, fill out the form and attach the required documents.

The registration will be restricted to roughly 50 participants, selected based on relevance.
Note: Preference will be given to graduate students, postdocs or junior scientists.

Registration Fee:

The registration fee is USD 150. It covers:

Payment information will be provided to selected participants in the notification email.

Financial Support for Transportation

We will provide financial support for transportation to/from the venue only for selected participants from US institutions. Acceptable means of transportation are airplanes, buses, and driving your own car. A limited number of grants for students will be available.

Request for such financial support should be indicated on the online application form. Grant applicants should provide:

Note: You are responsible for your own local transportation such as taxi, subway fees and parking.

Organizing Committee


Local organization:

The Summer School on Cognitive Robotics is being planned and run by
the Model-based Embedded and Robotic Systems (MERS) group at MIT.

Contact Us!

Send us questions and comments regarding the summer school at mers.cognitive.robotics at gmail dot com


National Science Foundation

AI Journal


MIT AeroAstro



Massachusetts Institute of Technology