Learning Deep Architectures for AI

Untitled

2001 • 98,485 citations

Gradient-based learning applied to document recognition

1998 • 49,688 citations

Optimization by Simulated Annealing

1983 • 42,775 citations

The Nature of Statistical Learning Theory

1995 • 38,742 citations

Visualizing Data using t-SNE

2008 • 34,476 citations

Untitled

1995 • 29,657 citations

Reinforcement Learning: An Introduction

2005 • 27,230 citations

Learning representations by back-propagating errors

1986 • 25,825 citations

Classification and Regression Trees.

1984 • 23,117 citations

Classification and Regression Trees.

1986 • 21,574 citations

Reducing the Dimensionality of Data with Neural Networks

2006 • 18,756 citations

Compressed sensing

2004 • 17,943 citations

Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

1984 • 17,865 citations

A logical calculus of the ideas immanent in nervous activity

1943 • 16,532 citations

A Fast Learning Algorithm for Deep Belief Nets

2006 • 15,378 citations

Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations

1986 • 15,232 citations

Nonlinear Dimensionality Reduction by Locally Linear Embedding

2000 • 14,624 citations

Receptive fields, binocular interaction and functional architecture in the cat's visual cortex

1962 • 13,401 citations

A Global Geometric Framework for Nonlinear Dimensionality Reduction

2000 • 13,279 citations

Indexing by latent semantic analysis

1990 • 12,717 citations

A training algorithm for optimal margin classifiers

1992 • 11,054 citations

Backpropagation Applied to Handwritten Zip Code Recognition

1989 • 10,538 citations

The origins of intelligence in children.

1952 • 9,556 citations

An Information-Maximization Approach to Blind Separation and Blind Deconvolution

1995 • 8,891 citations

Analysis of a complex of statistical variables into principal components.

1933 • 8,511 citations

Independent Component Analysis

2004 • 8,232 citations

Nonlinear Component Analysis as a Kernel Eigenvalue Problem

1998 • 7,862 citations

Experiments with a new boosting algorithm

1996 • 7,607 citations

Learning long-term dependencies with gradient descent is difficult

1994 • 7,520 citations

Introduction to Reinforcement Learning

1998 • 7,149 citations

Decoding by Linear Programming

2005 • 7,139 citations

Extracting and composing robust features with denoising autoencoders

2008 • 6,787 citations

Stacked generalization

1992 • 6,073 citations

Advances in kernel methods: support vector learning

1999 • 5,756 citations

A unified architecture for natural language processing

2008 • 5,201 citations

Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

1980 • 5,064 citations

Training Products of Experts by Minimizing Contrastive Divergence

2002 • 4,852 citations

An introduction to hidden Markov models

1986 • 4,638 citations

An interactive activation model of context effects in letter perception: I. An account of basic findings.

1981 • 4,633 citations

Greedy Layer-Wise Training of Deep Networks

2007 • 4,621 citations

Dimensionality Reduction by Learning an Invariant Mapping

2006 • 4,523 citations

Random decision forests

2002 • 4,254 citations

Bayesian Learning for Neural Networks

1996 • 4,252 citations

Curriculum learning

2009 • 4,226 citations

An Introduction to Kolmogorov Complexity and Its Applications

2019 • 3,838 citations

Learning with Local and Global Consistency

2003 • 3,822 citations

Sparse coding with an overcomplete basis set: A strategy employed by V1?

1997 • 3,662 citations

Hybrid Monte Carlo

1987 • 3,558 citations

Semi-supervised learning using Gaussian fields and harmonic functions

2003 • 3,341 citations

A Learning Algorithm for Boltzmann Machines*

1985 • 3,285 citations

An Energy Budget for Signaling in the Grey Matter of the Brain

2001 • 3,148 citations

Parallel & distributed processing

2005 • 2,682 citations

Best practices for convolutional neural networks applied to visual document analysis

2005 • 2,593 citations

Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

2009 • 2,452 citations

Learning in Graphical Models

1998 • 2,447 citations

Efficient sparse coding algorithms

2007 • 2,403 citations

Untitled

2003 • 2,334 citations

Neural Networks: Tricks of the Trade

2012 • 2,092 citations

Three approaches to the quantitative definition of information^*

1968 • 2,060 citations

Learning and development in neural networks: the importance of starting small

1993 • 1,971 citations

Deep Boltzmann machines

2009 • 1,887 citations

Restricted Boltzmann machines for collaborative filtering

2007 • 1,862 citations

On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes

2001 • 1,818 citations

An Introduction to Mathematical Logic

2005 • 1,818 citations

A formal theory of inductive inference. Part II

1964 • 1,795 citations

Online convex programming and generalized infinitesimal gradient ascent

2003 • 1,761 citations

Information processing in dynamical systems: foundations of harmony theory

1986 • 1,646 citations

Stochastic Complexity in Statistical Inquiry

1998 • 1,592 citations

Self-taught learning

2007 • 1,522 citations

Efficient BackProp

1998 • 1,508 citations

Hierarchical Bayesian inference in the visual cortex

2003 • 1,476 citations

Learning methods for generic object recognition with invariance to pose and lighting

2004 • 1,321 citations

Exploiting Generative Models in Discriminative Classifiers

1998 • 1,303 citations

Auto-association by multilayer perceptrons and singular value decomposition

1988 • 1,296 citations

Slow Feature Analysis: Unsupervised Learning of Invariances

2002 • 1,294 citations

Semantic hashing

2008 • 1,266 citations

Numerical Continuation Methods

1990 • 1,216 citations

Input space versus feature space in kernel-based methods

1999 • 1,173 citations

The Helmholtz Machine

1995 • 1,167 citations

Learning Overcomplete Representations

2000 • 1,161 citations

Active Learning with Statistical Models

1996 • 1,138 citations

Gaussian Processes for Regression

1995 • 1,110 citations

Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition

2007 • 1,109 citations

Exploring Strategies for Training Deep Neural Networks

2009 • 1,108 citations

The "Wake-Sleep" Algorithm for Unsupervised Neural Networks

1995 • 1,100 citations

Learning and Relearning in Boltzmann Machines

2001 • 1,056 citations

Efficient Learning of Sparse Representations with an Energy-Based Model

2007 • 1,055 citations

Numerical Continuation Methods--An Introduction.

1992 • 1,045 citations

Autoencoders, Minimum Description Length and Helmholtz Free Energy

1993 • 1,043 citations

The Cost of Cortical Computation

2003 • 1,034 citations

An empirical evaluation of deep architectures on problems with many factors of variation

2007 • 1,016 citations

A combined SVM and LDA approach for classification

2006 • 996 citations

A Neural Probabilistic Language Model

2000 • 984 citations

The Complexity of Boolean Functions

1991 • 973 citations

An Information Measure for Classification

1968 • 956 citations

Scaling learning algorithms towards AI

2007 • 945 citations

Parallel models of associative memory

1981 • 920 citations

Sparse deep belief net model for visual area V2

2007 • 912 citations

On Kernel Target Alignment

2006 • 886 citations

Recursive distributed representations

1990 • 881 citations

Training restricted Boltzmann machines using approximations to the likelihood gradient

2008 • 875 citations

A Scalable Hierarchical Distributed Language Model

2008 • 874 citations

Explorations in parallel distributed processing

1988 • 860 citations

A Tutorial on Energy-Based Learning

2006 • 808 citations

Representational Power of Restricted Boltzmann Machines and Deep Belief Networks

2008 • 795 citations

Supervised Dictionary Learning

2008 • 793 citations

Learning distributed representations of concepts.

1989 • 792 citations

Practical issues in temporal difference learning

1992 • 765 citations

Supervised Dictionary Learning

2008 • 763 citations

Small codes and large image databases for recognition

2008 • 745 citations

Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises

1989 • 742 citations

Segmentation using eigenvectors: a unifying view

1999 • 738 citations

Sparse Feature Learning for Deep Belief Networks

2007 • 720 citations

Modeling Human Motion Using Binary Latent Variables

2007 • 712 citations

Classification using discriminative restricted Boltzmann machines

2008 • 663 citations

Almost optimal lower bounds for small depth circuits

1986 • 630 citations

The Entire Regularization Path for the Support Vector Machine

2004 • 629 citations

On Contrastive Divergence Learning.

2005 • 624 citations

Unsupervised Learning of Models for Recognition

2000 • 620 citations

Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series))

1997 • 613 citations

Three new graphical models for statistical language modelling

2007 • 593 citations

Fundamentals of statistical exponential families with applications in statistical decision theory

1986 • 584 citations

Separating the polynomial-time hierarchy by oracles

1985 • 573 citations

Regularization and Semi-supervised Learning on Large Graphs

2004 • 550 citations

Connectionist learning of belief networks

1992 • 549 citations

Large-Scale Kernel Machines

2007 • 530 citations

Predicting Structured Data

2007 • 522 citations

Universal Artificial Intellegence

2005 • 516 citations

Is Learning The n-th Thing Any Easier Than Learning The First?

1995 • 514 citations

Efficient Pattern Recognition Using a New Transformation Distance

1992 • 474 citations

Estimation of Non-Normalized Statistical Models by Score Matching

2005 • 473 citations

On the quantitative analysis of deep belief networks

2008 • 459 citations

Untitled

1997 • 459 citations

A survey of kernels for structured data

2003 • 457 citations

Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure

2007 • 449 citations

Exponential Family Harmoniums with an Application to Information Retrieval

2004 • 443 citations

A Hierarchical Model of Shape and Appearance for Human Action Classification

2007 • 425 citations

Fundamentals of Statistical Exponential Families

1987 • 408 citations

Charting a Manifold

2002 • 391 citations

Object Class Recognition and Localization Using Sparse Features with Limited Receptive Fields

2008 • 380 citations

Deep learning via semi-supervised embedding

2008 • 373 citations

Deep learning from temporal coherence in video

2009 • 367 citations

Factored conditional restricted Boltzmann Machines for modeling motion style

2009 • 366 citations

Products of experts

1999 • 362 citations

Principled Hybrids of Generative and Discriminative Models

2006 • 341 citations

Learning Eigenfunctions Links Spectral Embedding and Kernel PCA

2004 • 335 citations

Mean Field Theory for Sigmoid Belief Networks

1996 • 334 citations

The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training

2009 • 319 citations

To recognize shapes, first learn to generate images

2007 • 303 citations

Links between perceptrons, MLPs and SVMs

2004 • 302 citations

Using fast weights to improve persistent contrastive divergence

2009 • 287 citations

Unsupervised learning of distributions on binary vectors using two layer networks

1991 • 281 citations

On the power of small-depth threshold circuits

1991 • 261 citations

Generative models for discovering sparse distributed representations

1997 • 235 citations

Flexible shaping: How learning in small steps helps

2009 • 227 citations

Semi-supervised learning of compact document representations with deep networks

2008 • 223 citations

Learning internal representations

1995 • 215 citations

Global Continuation for Distance Geometry Problems

1997 • 215 citations

A quantitative theory of immediate visual recognition

2007 • 214 citations

Justifying and Generalizing Contrastive Divergence

2008 • 205 citations

Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition

2010 • 202 citations

Learning Multilevel Distributed Representations for High-Dimensional Sequences

2007 • 194 citations

Unsupervised Learning of Image Transformations

2007 • 190 citations

The Curse of Highly Variable Functions for Local Kernel Machines

2005 • 190 citations

Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes

2007 • 186 citations

Some extensions of score matching

2006 • 174 citations

Describing Visual Scenes Using Transformed Objects and Parts

2007 • 167 citations

Nonlinear Autoassociation Is Not Equivalent to PCA

2000 • 164 citations

Learning the Kernel Matrix with Semi-Definite Programming

2002 • 163 citations

Natural language processing with modular pdp networks and distributed lexicon

1991 • 161 citations

Rational Kernels: Theory and Algorithms

2004 • 160 citations

Nonlinear Optimization and Applications

1996 • 159 citations

Efficient Non-Parametric Function Induction in Semi-Supervised Learning

2004 • 159 citations

10.1162/jmlr.2003.4.7-8.1235

2000 • 157 citations

Shift-invariant sparse coding for audio classification

2007 • 156 citations

A Context-Sensitive Generalization of ICA

1996 • 153 citations

Training Hierarchical Feed-Forward Visual Recognition Models Using Transfer Learning from Pseudo-Tasks

2008 • 149 citations

Convex Neural Networks

2005 • 147 citations

A DAY OF GREAT ILLUMINATION: B. F. SKINNER'S DISCOVERY OF SHAPING

2004 • 138 citations

Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation

2006 • 136 citations

An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators

2008 • 135 citations

How to invent a lexicon: the development of shared symbols in interaction

2006 • 135 citations

Loss Functions for Discriminative Training of Energy-Based Models

2005 • 124 citations

Many-Layered Learning

2002 • 124 citations

Connectionist language modeling for large vocabulary continuous speech recognition

2002 • 117 citations

Modeling image patches with a directed hierarchy of Markov random fields

2007 • 105 citations

Computational complexity of neural networks: a survey

1994 • 103 citations

Differentiable Sparse Coding

2008 • 100 citations

A Unified Energy-Based Framework for Unsupervised Learning

2007 • 97 citations

Is postnatal neocortical maturation hierarchical?

2005 • 96 citations

Deep belief net learning in a long-range vision system for autonomous off-road driving

2008 • 88 citations

DECISION TREES DO NOT GENERALIZE TO NEW VARIATIONS

2010 • 81 citations

Learning Continuous Attractors in Recurrent Networks

1997 • 81 citations

Evaluating probabilities under high-dimensional latent variable models

2008 • 74 citations

Learning Nonlinear Overcomplete Representations for Efficient Coding

1997 • 72 citations

Non-Local Manifold Tangent Learning

2004 • 70 citations

Slow, Decorrelated Features for Pretraining Complex Cell-like Networks

2009 • 66 citations

Using Manifold Stucture for Partially Labeled Classification

2002 • 65 citations

Transformation Invariant Autoassociation with Application to Handwritten Character Recognition

1994 • 61 citations

A Nonparametric Bayesian Approach to Modeling Overlapping Clusters.

2007 • 57 citations

Nonlocal Estimation of Manifold Structure

2006 • 49 citations

Statistical models for partial membership

2008 • 48 citations

Constituent Parsing with Incremental Sigmoid Belief Networks

2007 • 48 citations

Self Supervised Boosting

2002 • 45 citations

Establishing Good Benchmarks and Baselines for Face Recognition

2008 • 44 citations

Learning kernel parameters by using class separability measure

2002 • 36 citations

A Two-Layer ICA-Like Model Estimated by Score Matching

2007 • 34 citations

Learning long-term dependencies is not as difficult with NARX recurrent neural networks

1995 • 32 citations

Efficient training of large neural networks for language modeling

2005 • 30 citations

Global data analysis and the fragmentation problem in decision tree induction

1997 • 29 citations

Training connectionist models for the structured language model

2003 • 22 citations

Advances in Neural Information Processing Systems 20 (NIPS 2007)

2008 • 21 citations

Smoothing Techniques for Macromolecular Global Optimization

1996 • 21 citations

Information Processing in Dynamical Systems: Foundations of Harmony Theory

1987 • 19 citations

A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels

2005 • 19 citations

A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images

2007 • 19 citations

Extracting distributed representations of concepts and relations from positive and negative propositions

2000 • 16 citations

A New View of ICA

2001 • 14 citations

Memoires associatives distribuees: Une comparaison (Distributed associative memories: A comparison)

1987 • 14 citations

How to Make a Low-Dimensional Representation Suitable for Diverse Tasks

1996 • 13 citations

Descartes' Rule of Signs for Radial Basis Function Neural Networks

2002 • 12 citations

Building continuous space language models for transcribing european languages

2005 • 11 citations

Learning despite concept variation by finding structure in attribute-based data

1996 • 11 citations

Parallel continuation-based global optimization for molecular conformation and protein folding

1994 • 10 citations

Complexity Lower Bounds for Approximation Algebraic Computation Trees

1999 • 7 citations

Deleted Work

1955 • 0 citations