The world’s leading publication for data science, AI, and ML professionals.

Strategies for Keeping Up with AI Research

7 tips to help you stay on top of it all

Original image by OpenClipart-Vectors on Pixabay
Original image by OpenClipart-Vectors on Pixabay

The Speed of AI

For anyone that is following the fields of Artificial Intelligence (AI), Deep Learning (DL) or Machine Learning (ML) it can seem that research is speeding past you like a race car sometimes. A quick search on arXiv.org shows that 2,683 new articles related to AI, DL, or ML were announced between April 1st and May 1st of this year (2020). The research community is constantly being flooded with new work (both good and bad) and keeping up can seem like an impossibility. This is only further exacerbated by the coverage that AI is receiving in the media. Whether it is the imminent arrival of self-driving cars, advancements in natural language processing, automated medical diagnosis, or the fear of the cybernetic robot uprising, AI has been getting plenty of attention.

There are, of course, many benefits to the myriad of exposure that AI is receiving; however, for those working in the field, there can also be some negative effects. First, the fire hose of information from new applications, better architectures, emerging sub-fields, etc…, can become a distraction. There is a countless number of rabbit holes to jump down at any moment. While it is important to explore new and emerging solutions that may provide better solutions, if we spend all of our time chasing shiny objects, then we will never make any progress of our own.

Second, the barrage of new research and the growing AI industry can create unwanted anxiety. The pressure to succeed and succeed quickly can shake the confidence of even the most experienced researchers and developers. It can seem like other researchers are publishing twice as fast or companies are developing twice as many applications. This can lead to poor design decisions and wasted time, which undermines our ability to apply AI effectively. Furthermore, this can make us susceptible to group thinking and limit our ability to approach problems with novel and creative solutions.

Ultimately, it is necessary to keep up with the research related to your application area or industry, if you want to stay relevant. The trick is finding the ways that don’t consume all of your time and work best for you. The rest of this article identifies strategies to do just that. It is also important that we allow the state-of-the-art research to inspire us rather than just influence us, but that will be a topic for a future article.

Strategies for Keeping Up

So the good news is there is a plethora of resources available to help us keep up with the research that is relevant to our interests. In this article, I share a number of the resources that I use and discuss some of my strategies for staying current with AI research. The items below are ordered based on my personal priority but everyone learns differently, so I encourage you to try them out and see what works well for you.

0. Be Realistic and Consistent

We all have a limited amount of time to spend on keeping up with AI, so be realistic and consistent. Dedicate a fixed amount of time that fits within your schedule. Recognizing your time limitations should help you to choose the appropriate strategies that will be the most effective for you. Work smart and hard.

1. Read Papers

So as obvious as it may sound it, it is very important to spend some of your time reading papers. This will provide you with the obvious benefit of learning about new and emerging work, but it will also strengthen your ability to distill and understand complex concepts. Reading technical papers is a skill that is crucial for engineering and science, and like all other skills, practice makes perfect. Bonus: critically reading papers is also a great way to come up with new research/algorithm ideas by really challenging everything the authors are presenting.

Here are some of my tips for reading papers.

1A. Stay organized.

So the first tip I have is to keep your papers organized. In my case, I have "ReadMe" folder on my cloud drive that has all of the papers that I would like to read. So, when you have a little bit of time you can go straight to your folder and start reading. Once you have read the paper you can either move it to somewhere else in your digital library or simply delete it.

1B. Read smart.

As I learned quickly during my Ph.D, reading papers can take a lot of time and if you don’t have a good strategy you will never get through all of the papers that you need to. Personally, I have found that a three-phase approach works very well for me. Each phase has a specific purpose and the subsequent stages build on the previous ones. This can greatly speed up the process, because you will often find that getting through phase 2 is sufficient in many cases.

This is my general approach.

  • Phase 1: Read the abstract and the conclusion.
  • Phase 2: Read through the entire paper, but skip any technical details that require lots of mental effort.
  • Phase 3: Read critically and challenge the authors assumptions and assertions.
  • Phase 4 (Bonus!): Code it up! This is obviously more than just reading, but ultimately the best way to truly understand new concepts.

I will be publishing a more detailed article on my approach in the near future. I would also encourage you to read several other strategies and find what works for you.

1C. Set up alerts.

Google Scholar is an amazing resource for many reasons, but the automated alerts are a great way to keep up with what’s new. Alerts can be created based on an author or a search string. Author alerts are great for following the heavy hitters in the field and search string alerts are good for finding new authors.

To create an author alert:

  1. Search for an author and click on their name to get to their Google Scholar profile.
  2. Select the "Follow" button in the top right corner of the profile.
  3. Enter your e-mail address and select the alert options.
Image by author
Image by author

To create a search string alert:

  1. Log into Google Scholar and select "Alerts" from the side menu.
  2. Click "Create Alert".
  3. Enter your e-mail address, the search string that should be queried and the total number of results that should be included.
Image by author
Image by author

Pro Tip: If you use GMail then you can take advantage of these fancy filtering tips to automagically organize the alerts in your in-box. You can read about the fancy filtering here.

2. Watch YouTube

YouTube is one of the most exciting places that I am starting to see really good content. There are a lot of really smart people that are publishing paper reviews, online lectures and tutorials. All of these are excellent sources for quickly coming up to speed on new topics and gaining intuition. I found that these videos have saved me time and made me aware of new topics that I wouldn’t have otherwise seen.

Rather than providing a long list of potential channels, I have included just three YouTube channels that I would recommend starting with. I have included a little blurb about each channel and I why I think it is worthwhile. After checking out these channels, I would encourage you to check out some more channels to find the ones that suit your needs.

  • Yannic Kilcher: Yannic posts really good paper reviews. He walks through newly published papers and provides most of the required background material necessary to get a good grasp on the papers contribution. His videos are straight-and-to-the-point which allows him to post a lot of videos, most of which are less than 30 minutes. Nowadays, if there is a new paper that I am interested in, I will do a quick search to see if Yannic has covered it yet.
  • Henry AI Labs: This channel is run by Connor Shorten and also provides a lot of paper reviews and explanation videos. Like Yannic, Connor is very knowledgeable and does an excellent job explaining AI concepts. The real golden nugget of this channel (for me at least) is the "AI Weekly Update" series. Each week Connor provides a high-level review of the latest news in AI research The update videos are less than 30 minutes and include a ton of content.
  • Two-Minute-Papers: This channel is run by Karoly Zsolnai-Feher and has already amassed over 600k subscribers. Karoly produces short (~5 minutes) videos that are very well produced. The videos are typically very high level, but very clearly illustrate the topics. These videos are easy to watch and can be addictive (in a good way).

Pro Tip: YouTube settings allow you play videos back at up-to 2x speed, which is extremely helpful for getting through AI content. I have found that 2x speed is tolerable, once you get used to it. For even finer control, you can also use plugins like Enhancer for YouTube.

3. Subscribe to News Letters and Subreddits

Another great resource is to use news letters and Subreddits to help filter out content. Admittedly, I don’t subscribe to many news letters, but I find The Batch by deeplearning.ai to be very informative. Andrew Ng is one of the biggest names in AI and for good reasons. Each week he provides a curated list of hot topics in AI, along with the relevant context and why this research might be important. I also personally, think that Andrew covers topics that are interesting for both researchers and application developers. I would highly recommend subscribing to this news letter, as well as find other news letters that match your interests.

I have been a big fan of Reddit for many years. I have always viewed Reddit as a community based filter for the internet. Reddit hosts a number of sub-communities can Subreddits, that are dedicated to a particular topic or theme. Members can post content that can be either up or down voted based on how well it is received. The hope being that the most useful and informative content is promoted to the top of the list. There are several different Subreddits related to AI and ML, but I would recommend starting out with Deep Learning. This community has just under 50k members and the Subreddit is very active.

As an aside, Reddit is not only a good source of information, but it is also a place to interact with like-minded AI enthusiasts. I think this can be especially useful for those that are new in the field and would like to get some feedback, or those who don’t interact regularly with AI experts. I don’t personally post in many of these Subreddits currently, but hopefully that will change in the near future.

4. Go Back to the Basics

I found that it is useful to spend some of my time reviewing the basics. Strengthening your foundations will make you more efficient at reading papers within your field. I also try to choose these topics based on what I have been reading. For example, say that you are trying to learn about speech recognition algorithms, then it would be worthwhile to review your basic signal processing and spectral analysis. This will give you an edge to better understand and perhaps improve on what you are reading. It will also help you to challenge assumptions that you may otherwise take at face value.

While it is impossible to cover everyone’s interests, I would recommend the following general categories as topics to consider.

  • Calculus
  • Linear Algebra
  • Optimization
  • Programming
  • Digital Signal Processing
  • Random Processes, Statistics, and Information theory
  • Domain Knowledge (depends on your field of interest)

There are a number of great free and paid resources online for learning these types of topics. I would recommend that you take a look at Coursera, edX, Standford Online and MIT OpenCourseware for quick start.

5. Read Technical Blogs

There is definitely no shortage of AI and ML blogs these days. Content can vary widely based on the site and purpose of the blog. I typically end up on most blogs through google searches when I am trying to find information on a specific topic. However, there are a few blogs that I browse regularly to look for new content. Here are the top three that I find informative, but as usual this will depend on your interests.

Pro Tip: You can use a news reader tool like Inoreader or Feedly to gather all of your blog content in one place.

6. Listen to Podcasts

Podcasts can be hit or miss depending on who you are. There are a number of different AI and ML related podcasts out there, many of which follow fairly similar formats. Most of them include interviews with researchers and developers, talk about emerging issues, and cover the fundamentals of the fields. I have found myself spending less time with podcasts lately, but I still think they are great resources and worth a review. Some of these podcasts can be very lengthy, so it is probably best to choose a few and to stick with those for a while.

Similar to the section on YouTube, rather than providing a long list of possibilities, I have provided a few recommendations below.

  • Linear Digressions: This podcast offers bite-sized doses of technical Machine Learning content. The episodes are short (less than 20 minutes) and cover a wide range of topics.
  • Talking Machines: This podcast features longer episodes (about an hour) and is less technical than Linear Digressions. The format is based on interviewing working AI practitioners and discussing relevant topics in a very practical sense.
  • Artificial Intelligence Podcast: This podcast is solely based on interviews and the episodes range from one to three hours. Full transparency, this podcast covers a broad range of topics, many of which are non-technical and even philosophical in nature. I personally find many of the episodes interesting and the many of the guests are very well respected researchers and developers in the AI community.

7. Monitor Social Media

Now, I will admit that this recommendation is a bit hypocritical, since I abstain almost entirely from social media. However, I recognize that it can be a great tool for tracking what’s happening with your favorite authors or generally what is happening in your field. Many researchers and developers will often Tweet about new papers or developments in industry. This can be used as another source of filtering out important information. So if social media is your thing that find ways to make it productive for you.

Conclusion

AI (and related fields) are experiencing a period unprecedented growth which fantastic for the field, but it can certainly be overwhelming. With so much happening all the time it is easy to get distracted and that can be a detriment to your work. Take some solace in knowing that you are not alone. The most important thing is to recognize what time you have and to use it as wisely as possible.

This article presented a handful of strategies that I use for keeping up research. Pick a few of the strategies from the list and give them shot. Don’t try to implement too much at once or it will quickly become overwhelming. See what works and what doesn’t, as you slowly find your way of learning. Ultimately you will need to find what works best for you and that may change over time. Keep exploring, keep learning, and grow your understanding.


Related Articles