In early September, my PhD student Marta Andronic will be off to Turin to present our latest work "NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions" at the Field-Programmable Logic and Applications conference. Ahead of the detailed presentation at the conference, this blog post provides a short accessible summary of her exciting work. In

How a TensorFlow program is structured, what the training loop actually repeats, and how to tell a model that is learning from one that is memorising.

We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end

Master activation functions in neural networks: ReLU, sigmoid, tanh, GELU, Swish, and Mish. Learn how activation functions enable deep learning models to learn

Graph neural networks (GNNs) rely heavily on architecture design and artificial hyperparameters, often resulting in expensive manual effort and poor perfor

Check out the new article: Neural networks made easy (Part 67): Using past experience to solve new tasks . Author: Dmitriy Gizlyk

NeurIPS Proceedings Search Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, David Sussillo Advances in Neural Information Processing Systems 32 (NeurIPS 2019) Abstract Recurrent neural networks (RNNs) are a widely used tool for modeling sequential data, yet they are often treated as inscrutable black boxes. Given a trained recurrent network, we would like to reverse engineer it--to obtain a

NeurIPS Proceedings Search Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, David Sussillo Advances in Neural Information Processing Systems 32 (NeurIPS 2019) Abstract Recurrent neural networks (RNNs) are a widely used tool for modeling sequential data, yet they are often treated as inscrutable black boxes. Given a trained recurrent network, we would like to reverse engineer it--to obtain a

Networks of coupled dynamical systems have been used to model biological oscillators1,2,3,4, Josephson junction arrays5,6, excitable media7, neural networks8,9,10, spatial games11, genetic control networks12 and many other self-organizing systems. Ordinarily, the connection topology is assumed to be either completely regular or completely random. But many biological, technological and social networks lie somewhere between these two extremes. Here we explore simple models of networks that can be tuned throug

Learn how the Perceptron Neural Network processes inputs, makes decisions, and powers AI. Explore its structure, working, and real-world applications