Monday, January 22, 2018

The Benjamin Franklin Method of Reading Programming Books

Let’s face it, programming books suck. Those general books on distributed systems or data science or whatever can be tomes for a lifetime, but, with few exceptions, there’s something about the books on how to write code in a language/framework/database/cupcake-maker, the ones with the animal covers and the cutesy sample apps, they just tend to be so forgettable, so trite, so….uneducational.

I think I’ve figured out why I don’t like them, and it’s not just that they teach skills rapidly approaching expiration. It’s their pedagogical approach. The teaching algorithm seems to be: write these programs where we’ve told you everything to do, and you’ll come out knowing this language/framework/database/cupcake-maker. Central to these books are the long code listings for the reader to reproduce. Here’s an example, from one of the better books in this category

class User < ApplicationRecord
  attr_accessor :remember_token
  before_save { self.email = email.downcase }
  validates :name,  presence: true, length: { maximum: 50 }
  VALID_EMAIL_REGEX = /\A[\w+\-.]+@[a-z\d\-.]+\.[a-z]+\z/i
  validates :email, presence: true, length: { maximum: 255 },
                    format: { with: VALID_EMAIL_REGEX },
                    uniqueness: { case_sensitive: false }
  has_secure_password
  validates :password, presence: true, length: { minimum: 6 }

  # …another 30 lines follows...
end

Traditionally, there are two ways to study a page like this:

  1. Type out every line of code 
  2. Copy+paste the code from their website, maybe play around and make small changes 

Approach #1 is a method that, like a lecture, causes the code to go from the author’s page to the reader’s screen without passing through the heads of either. The second is like trying to learn how to make a car by taking apart a seatbelt and stereo: you’re just toying with small pieces. Neither is a sound way to learn.

If you had an expert tutor, they wouldn’t teach you by handing you a page of code. Still, these books are what we have. How can we read them in a way that follows the principles of learning? Read on.

Mental Representations

According to K. Anders Ericsson in his book Peak, expertise is a process of building mental representations. We can see this because expert minds store knowledge in a compressed fashion. Musicians can memorize a page of music far faster than a page of random notes. Expert chess players told to memorize a board position will do much better than amateurs, but, when they make a mistake, they’ll misplace whole groups of pieces.

This is possible because music and chess positions have structure that makes them look very different from a page of random notes or a random permutation of pieces. Technically speaking, they have lower perplexity than random noise. So, even though there are 26 letters in the English alphabet, Claude Shannon showed that the information content of English is about 1 bit per letter: given a random prefix of a paragraph, people can guess the next letter about half the time.

This is why a programmer skilled in a technology can look at code using it and read through it like fiction, only pausing at the surprising bits, while the novice is laboring line-by-line. This is also why a smart code-completion tool can guess a long sequence of code from the first couple lines. With a better mental representation, understanding code is simply less work.

(How do these mental representations work? My officemate Zenna Tavares argues they are distribution-sensitive data structures.)

This is exactly what’s missing from the “just type out the code” approach: there’s nothing forcing your mind to represent the program as anything better than a sequence of characters. Yet being able to force your mind to do this would mean being able to learn concepts more rapidly. Here’s a 200 year-old idea for doing so.

The Benjamin Franklin Method

I don’t know what’s more impressive: that Benjamin Franklin became a luminary in everything from politics to physics, or that he did this without modern educational techniques such as schools, teachers, or StackOverflow. As part of this, he discovered a powerful method of self-study. I’ll let him speak for himself (or go read someone else’s summary).

About this time I met with an odd volume of the Spectator. It was the third. I had never before seen any of them. I bought it, read it over and over, and was much delighted with it. I thought the writing excellent, and wished, if possible, to imitate it. With this view I took some of the papers, and, making short hints of the sentiment in each sentence, laid them by a few days, and then, without looking at the book, try'd to compleat the papers again, by expressing each hinted sentiment at length, and as fully as it had been expressed before, in any suitable words that should come to hand. Then I compared my Spectator with the original, discovered some of my faults, and corrected them.

—Benjamin Franklin, Autobiography

This process is a little bit like being a human autoencoder. An autoencoder is a neural network that tries to produce output the same as its input, but passing through an intermediate layer which is too small to fully represent the data. In doing so, it’s forced to learn a more compact representation. Here, the neural net in question is that den of dendrons in your head.

K. Anders Ericsson likens it to how artists practice by trying to imitate some famous work. Mathematicians are taught to attempt to prove most theorems themselves when reading a book or paper --- even if they can’t, they’ll have an easier time compressing the proof to its basic insight. I used this process to get a better eye for graphical design; it was like LASIK.

But the basic version idea applied to programming books is particularly simple yet effective.

Here’s how it works:

Read your programming book as normal. When you get to a code sample, read it over

Then close the book.

Then try to type it up.

Simple, right? But try it and watch as you’re forced to learn some of the structure of the code.

It’s a lot like the way you may have already been doing it, just with more learning.

Acknowledgments

Thanks to Andrew Sheng and Billy Moses for comments on previous drafts of this post.

Liked this post?


Related Articles

14 comments:

  1. Very interesting read. Thank you for this information!

    ReplyDelete
  2. Great article. Ben Franklin strikes again. What a guy.

    ReplyDelete
  3. Yes indeed, very interesting.

    It's the application of the Confucius quote : "I hear and I forget. I see and I remember. I do and I understand.", specially the third part.

    ReplyDelete
  4. Your method is missing Benjamin's "short hints". You should write down such short hints as you read each section (paragraph, page, section, or chapter, depending on the student and the material).

    ReplyDelete
    Replies
    1. The point of the short hints + time delay is that it prevents you from using your memory of the exact details of the imitated piece, forcing you to instead develop your mental encoding. Simply writing down some notes about a chapter before you imitate the code samples doesn't give you these benefits.

      So, the short hints would need to be coupled with having a substantial delay between encountering the code sample and then doing it --- quite a lot of overhead for just reading a book. It's a much more heavyweight version of my approach.

      Delete
    2. A lot of overhead but your short hints with delay sounds like a good drill. You wouldn't swim with fists closed inside latex gloves all the time, but for a short time to get the dinner plate effect when you take them off...

      Delete
    3. Both ways are sort of like The Hard Way, but blindfolded.

      Delete
  5. In many ways, this is similar to the flashcard method of study. Instead of reading over a text again and again to try to it into the brain, flashcards force us to try to recall the material. Even if we don't recall it, we are better primed to learn it when we see the answer on the other side of the card. It's an old, simple method, but it really does work.

    ReplyDelete
  6. It remembers me how i been reading code books.
    After reading few code books your experience drives you to follow those rules.

    ReplyDelete
  7. Definitely related: https://medium.freecodecamp.org/the-benefits-of-typing-instead-of-copying-54ed734ad849

    ReplyDelete
  8. An interesting idea, I'll give this a try.
    Thanks for the nice write-up.

    ReplyDelete
  9. I thought this exact same thing when I was reading his autobiography couple months ago! It reminded me of why Haseeb Qureshi was so successful out of his coding bootcamp. If you're wiling to take a stab at a problem by yourself, and then take the time to compare what you have created to someone else's optimized solution, and finally recreate your own optimized solution, you'll end up a better programmer.

    ReplyDelete
  10. Great & smart approach

    ReplyDelete
  11. Did you say "traditionally, there are 2 ways to study a page like this"? Idk if others agree, but never used some of those 2 ways. Specially the second one. Actually, i find myself using 2 different ways to learn:
    1. Since i was a novice: i read the code, interprete it in my head as a whole and line by line too, trying to know what it does each of those lines, then try to program it, execute every time i add a single line to know if i'm predicting it right or not and compare them with the original line of code. If at some point the behaviour isn't expected, probably i will be learning something, or i made a mistake myself. In the best case, i've wrote a better way to express the same thing in a different and clean way. The later was specially true when i learned a bit of functional programming and was working with lists.
    2. This another way come in more recently, but i'm getting used to it. Basically is: just read the code and write a set of tests you think will pass and will fail, finally write that code and predict playing with the tests i wrote. However, this seems simple, but isn't because: How many tests? how big each one? also, how testable/coupled is that code?. For example, like the example you have there, you will have to decouple it, and test every single functionality alone, then test them in a integral way combining subsets of functionalities, and lastly, start typing that code while you refactor it. While you are typing, you have to predict which tests will pass, which tests will fail, which tests are missing, which tests are uneeded and which tests were bad. And obviously, run those tests a lot for every functionality you add. Of course, this method is harder to do, because some things are hard to tests or are untestable in some extreme cases. Also because you have to write code for code you didn't wrote yet and that is a change on the normal way of thinking. However, if you make this, this will force you to carefully pay attention to all each line, and also the whole scenario, and better, fixing all the smells in the example code too. And this way is actually, more fun. You will find you tend to code more time than reading, without feeling sleepy and learning more things than the ones the author wrote.

    ReplyDelete