Abstract:
The labeled dataset is a subset of the Raw Dataset. It is comprised of pairs of RGB and Depth frames that have been synchronized and annotated with dense labels for every image. In addition to the projected depth maps, we have included a set of preprocessed depth maps whose missing values have been filled in using the colorization scheme of Levin et al. Unlike, the Raw dataset, the labeled dataset is provided as a Matlab .mat file with the following variables: accelData – Nx4 matrix of accelerometer values indicated when each frame was taken. The columns contain the roll, yaw, pitch and tilt angle of the device. depths – HxWxN matrix of in-painted depth maps where H and W are the height and width, respectively and N is the number of images. The values of the depth elements are in meters. images – HxWx3xN matrix of RGB images where H and W are the height and width, respectively, and N is the number of images. instances – HxWxN matrix of instance maps. Use get_instance_masks.m in the Toolbox to recover masks for each object instance in a scene. labels – HxWxN matrix of object label masks where H and W are the height and width, respectively and N is the number of images. The labels range from 1..C where C is the total number of classes. If a pixel’s label value is 0, then that pixel is ‘unlabeled’. names – Cx1 cell array of the english names of each class. namesToIds – map from english label names to class IDs (with C key-value pairs) rawDepths – HxWxN matrix of raw depth maps where H and W are the height and width, respectively, and N is the number of images. These depth maps capture the depth images after they have been projected onto the RGB image plane but before the missing depth values have been filled in. Additionally, the depth non-linearity from the Kinect device has been removed and the values of each depth image are in meters. rawDepthFilenames – Nx1 cell array of the filenames (in the Raw dataset) that were used for each of the depth images in the labeled dataset. rawRgbFilenames – Nx1 cell array of the filenames (in the Raw dataset) that were used for each of the RGB images in the labeled dataset. scenes – Nx1 cell array of the name of the scene from which each image was taken. sceneTypes – Nx1 cell array of the scene type from which each image was taken.
Year: 2012
Booktitle: European Conference on Computer Vision, Pages: 746--760

URL: https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html
License: No license specified, the work may be protected by copyright.

@inproceedings{silberman2012indoor,
title= {nyu_depth_v2_labeled.mat},
author= {Silberman, Nathan and Hoiem, Derek and Kohli, Pushmeet and Fergus, Rob},
booktitle= {European Conference on Computer Vision},
pages= {746--760},
year= {2012},
organization= {Springer},
abstract= {The labeled dataset is a subset of the Raw Dataset. It is comprised of pairs of RGB and Depth frames that have been synchronized and annotated with dense labels for every image. In addition to the projected depth maps, we have included a set of preprocessed depth maps whose missing values have been filled in using the colorization scheme of Levin et al. Unlike, the Raw dataset, the labeled dataset is provided as a Matlab .mat file with the following variables:

accelData – Nx4 matrix of accelerometer values indicated when each frame was taken. The columns contain the roll, yaw, pitch and tilt angle of the device.
depths – HxWxN matrix of in-painted depth maps where H and W are the height and width, respectively and N is the number of images. The values of the depth elements are in meters.
images – HxWx3xN matrix of RGB images where H and W are the height and width, respectively, and N is the number of images.
instances – HxWxN matrix of instance maps. Use get_instance_masks.m in the Toolbox to recover masks for each object instance in a scene.
labels – HxWxN matrix of object label masks where H and W are the height and width, respectively and N is the number of images. The labels range from 1..C where C is the total number of classes. If a pixel’s label value is 0, then that pixel is ‘unlabeled’.
names – Cx1 cell array of the english names of each class.
namesToIds – map from english label names to class IDs (with C key-value pairs)
rawDepths – HxWxN matrix of raw depth maps where H and W are the height and width, respectively, and N is the number of images. These depth maps capture the depth images after they have been projected onto the RGB image plane but before the missing depth values have been filled in. Additionally, the depth non-linearity from the Kinect device has been removed and the values of each depth image are in meters.
rawDepthFilenames – Nx1 cell array of the filenames (in the Raw dataset) that were used for each of the depth images in the labeled dataset.
rawRgbFilenames – Nx1 cell array of the filenames (in the Raw dataset) that were used for each of the RGB images in the labeled dataset.
scenes – Nx1 cell array of the name of the scene from which each image was taken.
sceneTypes – Nx1 cell array of the scene type from which each image was taken.},
keywords= {Depth RGBD},
terms= {},
license= {},
superseded= {},
url= {https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html}
}