A Little Background

For the longest time, I was a Google Reader guy. I had a list of sites that I wanted to follow, and Reader was the way that I could quickly triage the news that I cared about. I’m almost embarrassed to admit it because RSS feeds are so last decade, but I like being in control of what I read and what I don’t read. And I don’t trust recommendations from Flipboard style applications or from my Facebook feed.

When Google Reader was unceremoniously shut down, I shuffled back and forth between different RSS readers until I found Feedly. Feedly makes a solid product, but I’ve never fully bought in. As a result, I no longer actively prune the sites on there (case in point: Feedly still pulls feeds from a number of watch related sites from the short-lived time when I thought that my life goal ought to be to own both a Lange & Sohne and Patek Philippe Perpetual Calendar Chronograph). The end result of all of this is that RSS is no longer the way that I get my news.

The way that I keep up-to-date on news these days (particularly on work-related topics) is by checking the General and Random channels on Lab41’s Slack. We have a large enough team these days that there are at least 2 or 3 gems in there every day. It has completely replaced my HackerNews habit. It got me thinking that others might be interested in the curated list of papers, articles, tutorials, and reviews that our team has put together. So, without further adieu, I present…

Across the Network

Welcome to first edition of Across the Network: Lab41’s weekly review of the most important happenings in the Artificial Intelligence universe.

Articles

WaveNet — Strictly speaking, this post is from the end of last week — but who’s counting anyways. The fine folks at Google DeepMind bring to us a generative model that produces raw audio waveforms. What does that mean? Well, the most practical application of this waveform generation is speech synthesis — when computers speak words to you. You know how you can tell that Siri is a computerized voice? With WaveNet…you can still tell it is a computer. But it sounds more like a human. How much more? /shrug.

Prosthetic Knowledge — Staying on the topic of generative models (in this case, a Generative Adversarial Network), the folks at MIT have built a neural network that can predict frames in a video from a single image. The layperson in me is usually not very impressed with these types of efforts (because the predicted frames are obviously not very real looking), but the geek in me can’t help be amazed at how well this network performs.

Diversity in Language Analysis — This is a really interesting article that discusses the datasets that are commonly used to train Natural Language Processing (NLP) models. The authors’ contention is that NLP models are always trained with “pristine” language. But if you train your model with data from the New York Times or Wall Street Journal, there are entire classes of English speakers whose speech and writing will be unintelligible. Seems reasonable. The authors suggest that NLP researchers should broaden their data sources in order to create more comprehensive models of the English language.

Evolution in Action — This link has nothing to do with AI. But I think that everyone should watch this video. The researchers behind MEGA-plate wanted to see if they could videotape evolution in action. So they set up a 4ft wide dish with agar jelly (bacteria like to eat this stuff) and segmented areas with different concentrations of antibiotics. Over the course of several days, they show in video how quickly the bacteria evolve to be resistant to the antibiotic. It is stunning.

Academic Papers

Stacked Approximated Regression Machines — The big news this week was that a paper that purported to have discovered a “simple deep learning approach,” was withdrawn from arXiv after some loud complaining in academic circles (and by loud complaining I mean shade being thrown in the quiet hallways of academic institutions). What did SARM claim? They claimed to get VGG-like performance using stacked layer-wise operations without back-propagation. I’m not actually sure about the details because the paper was pulled! If I find out, I’ll let you know.

Reinforcement Learning meets Starcraft — Finally, an AI that does something valuable! Playing Starcraft. True story — the first thing that my wife ever said to me (this was the second day of freshman year of college) was “Do you play Starcraft?” I didn’t. She didn’t talk to me again for 5 years. The folks from Facebook AI Research have just the answer for me though. High-level Starcraft play is all about micromanagement of resources. They wanted to see if an AI could do a good job. The answer — their solution is world-class, but can’t do everything a human can.

Starcraft 2 being played — I think

Resources

Data Science Masters — Interested in becoming a Data Scientist? The kind team at Luminant have put together a set of training resources that an aspiring Data Scientist can look at. It’s remarkably complete, and I found some things on the list that I hadn’t seen before. Definitely worth a quick peak, even if you can already explain the difference between Momentum and AdaGrad.

The Neural Network Zoo — I love this page. Every day, you read about a new neural network architecture. After all, every Computer Science graduate student in the world right now who wants to maximize their income is becoming a Deep Learning expert (haven’t you heard — there is a shortage of experts). This page intends to maintain a “complete” chart of all of the neural network architectures and simple descriptions about how they differ. I plan to use this site frequently.

Shameless Plug

Lab41 Reading Group — My colleague Abhi just wrote up an article explaining an academic paper that he read called SqueezeNet by Forrest Iandola et al. I suggest you take a look!

That’s all I’ve got. We hope to see you again next week on Across the Network.