Gestural Inference

Please note that this study is now concluded, and we are in the process of formulating and evaluating solutions to the underwater pointing gesture interpretation problem. We thank you for your interest and participation. The information is kept here for reference and historical purposes.

The University of Minnesota’s Interactive Robotics and Vision lab is recruiting participants for a research project in the area of human-robot interaction (HRI). To participate, we are asking for volunteers to record a short (approximately 30-second) video of themselves pointing at objects. 

If you agree to take part in this research study, you will receive a $10 Amazon e-gift card as a thank you for your time and effort. Each participant can receive a maximum of one (1) $10 Amazon gift card, regardless of the number of video submissions contributed. Receiving the thank you gift card is contingent upon the video submission meeting the study criteria. If your video submission does not contain pointing gestures as defined in the criteria below, you will not receive the thank you gift card. After the submission is reviewed and accepted, the gift card will be electronically delivered to eligible participants via the email they provide with their video submission on the intake form. 

If you would like to participate, please see the following instructions and attached information form for additional project information. Note that you must be age 18 or older to participate.
Thank you in advance!

  1. Read the attached information form to determine your eligibility and continued interest in participation. 
  2. Create a video/video(s) of yourself (of duration at least 30 seconds) performing one or more of the following classes of tasks. When performing each pointing gesture, please point in the direction intended or toward the object being indicated, and hold the pose for roughly two seconds. We suggest that you place yourself a moderate distance from the camera (approximately two meters), and we prefer videos made with uncluttered backgrounds.
    1. Gesturing for someone to go somewhere
      1. Directing people to a location during a tour
      2. Telling a pet to go to a location (another room, or outside, for example)
        Gesture for Go Somewhere: 
        Go There Pointing Gesture
    2. Gesturing to pick up an object
      1. Cooking with assistance (prepare, or pretend to prepare a meal where an assistant brings you any necessary utensils or ingredients)
      2. Performing tasks like handicrafts with assistants i.e including an indication of picking up things illustrated by hand gestures
        Gesture to Pick up an Object:
        Pick Up Object Pointing Gesture
    3. Gesturing to take a photo 
      1. Point at an object that you would like a photograph taken of
        Gesture for Take a Picture:
        Take Picture Pointing Gesture
    4. General Pointing gestures
      1. Pantomiming giving a weather forecast report
      2. Any other activity that involves making natural pointing gestures
        General Pointing Gestures may look like this:
        General Pointing Gestures

Please use the above images as reference for each of the four classes of gestures. You may make these gestures in any orientation.  

When capturing your video, please do not use any filters, or add color-correction afterwards. Likewise, please do not edit the video after capture, unless it is necessary to adjust the brightness or crop. Do not alter the shape or orientation of the gesture or make any other edits. 

We request that your video submission not contain geo-location data or audio. If included, these components of the video will be removed and the original video destroyed.

Sample video submissions are given here:





3. Please upload your video(s) using this google form: