Deep Learning is everywhere; it is on your mobile, it runs behind your favorite search engine, decides what you see on your favorite social media platform, and the list is almost endless. Sejnowski tells the story of the deep learning revolution from an insider’s point of view. Although one can’t deny that the book is amazing, its title is a bit misleading. “The Birth of Computational Neuroscience and Its Relation to Deep Learning” would be rather a fitting one – though the publisher’s marketing department wouldn’t like it.

Sejnowski is one of the founding fathers of deep learning and also, he is on of the early pioneers of computational neuroscience, so this title can be read as his memoirs.

The story in a nutshell goes like this: Since the early 2010s, we can see the rapid development of various deep learning techniques, however the recent revolution started more than 70 years ago, with the works of McCulloch and Pitts. Early revolutionaries sidelined by the Perceptrons book written by Minsky and Papert in the late 60s, but they didn’t give it up. By the 80s, they organized the NIPS (now renamed es NeurIPS) conference, then a small, but vibrant community, which had 8000 attendees last year. Sejnowski’s story is very personal, he’s not afraid of expressing his dislike towards Chomsky, Minsky and others from the symbolic AI branch. He is against the general view that neural networks are tools of engineers, their resemblance to actual, physical neurons is superficial and deep learning is a technique, which has nothing to do with neuroscience. Rather, the author takes the opposite view and as a computational neuroscientist he emphasises the connections between the advancements of the field and the development of deep learning.

The first two parts of the book, which are describing the birth of deep learning and explaining its basic ideas, are excellent even if we take its personal tone into account. The third part is a bit sketchy, since it tries to be about the scientific and technological impacts of deep learning. Although its title aims to delimit its subject to scientific and technological impacts, it cannot restrict itself to merely these topics. Sejnowski writes about everything, from consciousness to education. Despite the flaws of its third part, this is a very informative monograph, which is strongly recommended for everyone interested in deep learning – let it be a manager, a student, or a data scientist.

Sources

The header image has been downloaded from the scikit-learn site.

The review is based on the Kindle and Audible editions of the book.