Google brain team announced Swish activation function as an alternative to ReLU. Experiments show that swish overperforms ReLU for deeper networks
Modules and optimizers are daemon-resident objects with their own handles. Compose, train, and save PyTorch-compatible models from the shell.
# Steering Along Manifolds to Control Neural Networks Concept geometry provides a blueprint for controlling the behavior of neural networks—if you know how to look. Intervening on a model's internal representations to steer behavior, i.e., representation steering, promises lightweight, adaptable, and granular control of neural networks. That control can be leveraged during inference and training to design and align models. The typical approach is to steer in a straight line by adding a scaled "steering v
Netflix has long chased after methods of improving its movie recommendation algorithms, once even awarding a $1M prize to any team of people who could substantially improve on the then-current design. As part of that process, the company has been researching neural networks. Conventional neural
--> Exploring Strategies for Training Deep Neural Networks Hugo Larochelle, Yoshua Bengio, Jérôme Louradour, Pascal Lamblin. Year: 2009, Volume: 10 , Issue: 1, Pages: 1−40 Abstract Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions. However, until recently it was not clear how to train such deep networks, since gradient-based optimization starting from random initialization often appears to get stuck in
NeurIPS Proceedings Search Neural Nearest Neighbors Networks Tobias Plötz, Stefan Roth Advances in Neural Information Processing Systems 31 (NeurIPS 2018) Abstract Non-local methods exploiting the self-similarity of natural signals have been well studied, for example in image analysis and restoration. Existing approaches, however, rely on k-nearest neighbors (KNN) matching in a fixed feature space. The main hurdle in optimizing this feature space w.r.t. application performance is the non-differentiability
Reducing the size of neural network models is crucial for their effective deployment on edge devices
Abstract page for arXiv paper 1903.06758: Algorithms for Verifying Deep Neural Networks
PhD student at University of Texas at Austin 🤘. Doing systems for ML.
Deep Belief Networks and Autoencoders A DBN is a network that was created to overcome artificial neural networks problems