[00:00:01] Hello again. This is Mark, and you're listening to the Learning Machine podcast. Today, we're going to talk about reviewing and remembering things and about one of my favorite tools in the world. So let's start this by looking at the brute force approach.
[00:00:16] One thing that's popular and works well, if you've got a test the next day is to just cram material, just spend a lot of time reading it, reviewing it, doing it repeatedly... and, you know, if it's an academic subject, maybe make flashcards, go through those flashcards over and over again until you know all of them. This is very popular and it does work for tests, but it's time consuming. And if you continue cramming and just going through flashcards by brute force after the test, you'll find it results in way too much study time in the intermediate term. And you'll probably get to a point where it just seems crazy because you know, the material that you're reviewing every single day or every single week or whatever interval you're reviewing it at and you're getting it right all the time, and why spend hours and hours doing it? So, the natural next step is to stop reviewing it and then eventually forget the material.
[00:01:16] This is because there's something called a forgetting curve. People don't forget information at a static rate. The more times you've reviewed something and the longer you've known something, the longer you will remember it without reviewing it. And in fact, the rate at which you forget things. Or, I should say, the time until you forget something tends to increase exponentially with more exposure. In other words, if you learn something today and you've been using it all day, you'll probably remember it tomorrow and you likely won't remember it in a week. However, if you've been using something for a week, then you probably will remember it a week later and might remember it a month later, but probably won't remember it a year later. And similarly, if there's something you've been remembering and using regularly for a full year, there's a good chance you'll remember it for a much longer amount of time.
[00:02:12] Now, in recent decades, people have come up with better and more efficient methods for reviewing and remembering things that have been learned previously. And these newer methods take advantage of the forgetting curve and the exponential nature of improvement with repeated review.
[00:02:28] One of the first was a German science journalist named Sebastian Leitner in the 1970s, and his system was just called the Leitner system or Leitner boxes. In its simplest form, we basically still have flash cards, but you have three boxes and the first box you review everyday. The second box you review less often, like say every three days, and then the next box you review even less often—for example, every two weeks. And when you review your flashcards, if you get one right that was in the first box, you promote it to the second box. And if you get one right that was in the second box, you promote it to the third box. If you fail to remember an item on a flash card, then you demote that card to the previous box and anything you fail in the first box, you review again immediately. With this system, you spend more time on the items that you've forgotten and less time on the items that you remember and don't need to review.
[00:03:30] And it's also very easy to extend. For example, you could add a fourth box that you only review once every two months or a fifth box that you only review once a year. And by using this system, you can actually keep a much larger set of flash cards that you're never giving up on. But you're not spending that much time on them either because you've learned them a while ago and you've been practicing them and you honestly don't need to spend much time on them. In many ways, Leitner boxes were the perfect system for the 1970s, and they're not that bad even for now. I've actually used them myself on a few occasions and found that they're good enough to get the job done.
[00:04:15] However, in the 1980s, after people started getting personal computers, some ambitious learners saw an opportunity to automate and improve the system further. The most famous by far was Piotr Woźniak, a Polish graduate student studying molecular biology and computer science in the mid 80s. He made a system called Super Memo, which basically did the same thing as the light in boxes, except that instead of having a whole bunch of boxes full of flash cards, all the flash cards were on a computer. And in a sense, super memo also put each card in its own box. In other words, it scheduled every single item individually, according to an algorithm.
[00:04:59] One item might be scheduled for review in 24 days and another in twenty nine days. And this time was also dependent on a user entered difficulty. So instead of simply keeping track of whether or if you could remember an item or not, you would differentiate between remembering with ease, remembering with difficulty and not remembering at all. Piotr Woźniak improved and revised the algorithm repeatedly over the years, and it's clear that he was highly motivated to improve his own personal performance as a student.
[00:05:34] However, Super Memo did become a company, and throughout the 90s more and more people started buying and using spaced repetition. But, it stayed highly niche until the mid to late 2000s. Then a new group of spaced repetition system enthusiasts appeared language learners. One of them was a developer named Damien Elmes, who was learning Japanese, and he created an open source competitor to Super Memo, called Anki, which means "memorize" and Japanese. It took time, but it gained more and more and more steam. And by 2008 or 2009, it pretty much dominated space repetition systems and it also dominated a lot of Chinese learning and Japanese learning blogs.
[00:06:23] And space repetition, in general, got further steam from a fantastic wired article in 2008 that was titled, "Want To Remember Everything You'll Ever Learn? Surrender to This Algorithm." It told the story of Peter was Nick's creation of Super Memo, as well as his fairly extreme exploits as a learner himself. And it drew a lot more interest to space repetition. By 2010, Anki had clearly broken out of the language, learning niche, and Roger Craig actually broke the all-time record for single day winnings on Jeopardy! after using Anki to memorize a vast number of facts. And, in the following years, Anki's reach only grew.
[00:07:06] At this point, there's a vast collection of free plug-ins for it, and people don't even have to create their own decks. There are already pre-made decks for most subjects people would want to study. However, Anki is not perfect. It has possibly the worst user interface of any piece of software I've spent considerable time using. And I say this as someone who's blogged about it positively and even contributed to the project myself about 10 years ago! It's a phenomenal tool that I highly recommend to any motivated learner, but it takes enough work to learn how to use the tool itself that unmotivated learners and semi-motivated learners just aren't really going to use it. And that leads to the rise of Quizlet, which actually was started before on Anki, but was much smaller in the early years and has become absolutely massive in the past five years.
[00:08:08] Quizlet was essentially a flashcard cramming app, created by another developer—Andrew Sutherland—in 2005, that didn't take off at first, but over time it ended up getting very popular. It's impossible to tell if it's more popular than on Anki since it's a web site, a very popular web site, and Anki is open source downloadable software. But, it's probably bigger than on key at this point.
[00:08:38] And the main reason is because it's very, very easy to use. You just go to the site typing a bunch of flashcards and then you're ready to go. You don't have to learn. Let's face repetition as you don't have to say how easy or hard a card was when you get it wrong. But due to the continued growth of spaced repetition, the Quizlet company eventually added spaced repetition support to its site in 2016.
[00:09:07] So that's what spaced repetition is. Now let's look at the experience of actually using it. Let's say you're studying a foreign language and you've got a deck with a lot of vocabulary words that you've already learned. You've just recently learned them and you don't want to forget them. Well, for any given word, it's usually going to go something like this: the first time you see it, maybe you'll get it right, maybe you won't. Then if you've got it right, it'll be shown to you again in a day or two. If you get it right that time, the next time is going to be maybe five days. Then after that, maybe a month or so. And within just getting this word right, eight times or nine times, your next scheduled review will be years away. That's how efficient, spaced repetition can be compared to just cramming or doing, you know, a weekly review or a monthly review.
[00:10:05] You really do get a lot if you take advantage of this exponential decay and in general, you're going to get about... well, it really depends on the type of card. Say you're doing something like recognizing a word in a foreign language, then you'll probably be remembering successfully 90 to 95 percent of the time. And those reviews will get very infrequent very quickly. On the other hand, if it's a harder task like going from your language to another, for example, you're an English speaker. You see the word memorize. You have to remember that that's Anki in Japanese. Then maybe you'll only be getting 85, 90 percent correct. And of course, if you have to write on key with kanji and you don't already read Chinese, then you're probably going to have an even steeper hill to climb. But it's still going to be a lot more efficient than if you had just done the cramming method. However, it's not a silver bullet. Spaced repetition is not the best way to learn everything. In fact, it's not the best way to learn everything related to language learning. It's pretty horrible for a lot of it. And unfortunately, due to the popularity of space repetition, most new language learning apps I see are essentially just spaced repetition, flashcard apps.
[00:11:27] These SRS flash card apps are very good for learning vocabulary and depending on what's included in the flashcards, they could be for grammar as well. But the problem for a language learner is that it's de-contextualized. So say if you just study vocabulary words, then they're going to be a lot of things that you'll miss like collocations. You won't know which words are normal to use with which other words. For example, in English, if someone asks, "How are you?" it would be completely normal to answer, "pretty good". It would also be fairly normal to answer, "absolutely fantastic". But it would be strange to answer that you're doing "absolutely good". There's no grammatical reason. It's just not something that English speakers tend to say.
[00:12:17] And there are many, many, many language features that are like this. There are also questions of word boundaries. For example, in English, the word "nose" refers to a person's nose or a dog's nose, but not every kind of animal. For example, an elephant in English doesn't have a nose. It has a trunk. In Japanese, the same word, 鼻 (hana), is for a person's nose and the elephant's nose. So the question is, what exactly does "nose" mean? Well, really tedious language teacher could explain this for every single word that you study, or even put this on the back of every flash card for every word that you're reviewing. But, it's not going to be efficient. You'll spend so much time worrying about edge cases for every single word that you're learning that you're actually not going to get anywhere.
[00:13:11] The solution in language learning is extensive reading. If you do a lot of reading and the material is easy enough that you can go at a decent speed, then you'll just get so much input that you'll have a feel for what words are used when and you'll know what the specific definition boundaries around though are. And of course, it's not just language learners that have this problem with spaced repetition systems—that everything is reviewed out of context.
[00:13:11] You might remember from when you were younger and taking math classes, there were certain techniques that you had to get good at or you had to be able to do at least on some tests, like say, use trigonometric identities and some information to figure out how tall something is or how far away it is or completing the square to derive the quadratic formula... something like that. Mastering these kinds of problem solving techniques has the same sort of issue revolving around context. If you just mechanically memorize each piece of a method, you won't necessarily be able to apply it when you're given an actual problem on a test or any other situation.
[00:13:41] On the other hand, if you put the entire problem into a flash card like say you just write out a math problem and that's the cue part of your flash card, and every time you see that card, you have to solve the whole problem. Well, that's going to take enough time that you won't be able to do those problems very often. You might want to do it for some really important things, but it can't be your go to study method.
[00:14:55] Similarly with programming, you can memorize a lot of things about a programming language, but you're not actually going to learn that language or learn how to program at all if you don't write programs and if you don't have the experience of making mistakes and then the experience of debugging those mistakes and fixing your programs.
[00:15:16] To summarize, there are two situations where spaced repetition works fantastically well. The first is when you're dealing with discrete chunks of information that don't rely upon each other. For example, learning how to write Chinese characters is like this, though there are some general things that you should know. For example, within any given character, you generally write from top to bottom and left to right. The way to write any given character does not depend upon knowing a different character.
[00:15:49] There might be a character that is a component inside of a more complex character, but you still don't have to know a different character in order to know how to write that one. And you don't have to care too much about the context because the way a given character is written doesn't change based on other characters near it, the way many features in language do, such as the feeling of a given phrase when surrounded by other phrases. Similarly, if your goal is to remember all of the capitals of all of the countries in the world, spaced repetition is a great way to do it. That's because the capital of Switzerland isn't dependent on what the capital of Australia is. And there are no other complications. It's just discrete chunks of information that you have to memorize. Knowing information about the periodic table of elements also falls into this category, as does a lot of what medical students study. Anything where it's just memorization, heavy spaced repetition is great.
[00:16:56] The second situation where spaced repetition works really well is when you're studying something more complex, but you can use spaced repetition as a scaffold to get through the initial phases a bit faster. Learning vocabulary is like this. Though there are the issues that pointed out earlier about context and collocations and word boundaries, spaced repetition can still be a huge win. And that's because you don't have to use it to learn and remember everything. You can use it to get a scaffold, to get started with whatever foreign language you're learning or medical vocabulary or whatever kind of vocabulary. And after having memorized, say, five hundred or a thousand or even five thousand words in this new area that you're studying, you can then do a lot of reading and use those words and encounter them in many different contexts and actually acquire a solid grasp of that topic.
[00:17:57] And the same is true in many other areas of study. I actually used it for learning CSS in 2012 when I got serious about learning how to style and layout web pages. And I didn't learn all of how CSS works through space repetition, but I did memorize a lot of properties and what they did and what the options for them were very quickly at the beginning and continued to practice those for several months until I had enough experience actually using those facts while building websites that I no longer needed the crutch. It was definitely a good use of my time to use spaced repetition to scaffold that beginning part of learning CSS.
[00:18:42] So that's pretty much the two areas where I found spaced repetition useful. One is for memorizing lots of unrelated facts. Two is for scaffolding the early parts of studying something that's more integrated. Now, there's one last thing I want to mention before we end this, and that is even knowing about spaced repetition can help you with topics that you can't really put in a flash card deck like, say, physical skills, learning how to do very specific things related to music or sports. Well, the same ideas still apply. Your forgetting curve is still going to be kind of exponential. So at the very beginning, you'll need to practice far, far more often. And then once you've gotten good at a skill, you can gradually decrease how often you practice it. And in fact, the amount of time you're likely to retain the skill will increase exponentially with practice. So, you should be practicing far more often at the beginning and far less often later. But don't stop suddenly go from something like daily practice to weekly practice to monthly to yearly. And then at a certain point you'll probably never forget.
[00:19:59] Hopefully this overview helps you find a way to use spaced repetition to dramatically improve your learning. And thank you for listening to The Learning Machine.