Traditional methods used to estimate 3D structure and camera motion in videos rely heavily on manual assumptions such as continuity and planarity.Three types of cognitive learning google researchers have now presented an alternative deep learning method which is able to obtain these assumptions from unlabelled video. They succeeded in training a deep network to predict the intrinsic parameters of the camera, the first method to address occlusions in geometric ways and substantially reduce the amount of semantic understanding needed to detect moving elements in video content.Three types of cognitive learning

We present a novel method for simultaneous learning of depth, egomotion, object motion, and camera intrinsics from monocular videos, using only consistency across neighboring video frames as supervision signal.Three types of cognitive learning similarly to prior work, our method learns by applying differentiable warping to frames and comparing the result to adjacent ones, but it provides several improvements: we address occlusions geometrically and differentiably, directly using the depth maps as predicted during training.Three types of cognitive learning we introduce randomized layer normalization, a novel powerful regularizer, and we account for object motion relative to the scene. To the best of our knowledge, our work is the first to learn the camera intrinsic parameters, including lens distortion, from video in an unsupervised manner, thereby allowing us to extract accurate depth and motion from arbitrary videos of unknown origin at scale.Three types of cognitive learning we evaluate our results on the cityscapes, KITTI and euroc datasets, establishing new state of the art on depth prediction and odometry, and demonstrate qualitatively that depth prediction can be learned from a collection of youtube videos. (arxiv).Three types of cognitive learning

It is known that current neural network models are data hungry, while detailed annotations are challenging to acquire at scale. Predictive learning emerges as a powerful way to train nns with the self-supervised signal from an abundant amount of video sequences online without much concern about the limited coverage of annotated data and scenarios.Three types of cognitive learning

In order to take advantage of the power of predictive learning, this paper explored the latent geometric structure between two successive frames.Three types of cognitive learning the latent structure is just as simple as geometric transformation between two frames (z’p’ = KRK−1zp + kt). By introducing inductive knowledge about the geometric/perspective structure and motion into the NN-based model design and relaxing the need for intrinsic camera parameters, it makes the learning much more data efficient and generalizable.Three types of cognitive learning

As a recent review paper pointed out: “ we conclude that current self-supervised methods are not ‘hard’ enough to take full advantage of large scale data and do not seem to learn effective high-level semantic representations.” there is no guarantee that a self-supervised algorithm can learn any meaningful high-level structure about the scene.Three types of cognitive learning instead of learning from scratch without any inductive knowledge, I can see that there is a trend in the field to leverage self-supervised learning with inductive knowledge to provide high-level supervision guidance.Three types of cognitive learning that’s what we develop at isee AI to train an autonomous driving systems with some common sense ( MIT technology review, fortune).

The criticism of this paper is they did not show many visualizations of the internal representation that the network learned, for example, the foreground mask and translation field.Three types of cognitive learning I guess the learned representation of the foreground mask and translation field is not well regularized. Some higher-level knowledge about the scene/object motion may be helpful to further improve the performance.Three types of cognitive learning