
From Pencil to Something Else
If I ask you how much is 10 multiplied by 3 you’re probably able to quickly say: "30!". No need for any great effort. However, if I ask you how much is 3498 multiplied by 2345 you probably would have some trouble answering it from the top of your head. If instead I was to allow you to use pencil and paper you would have no problem and in a couple of minutes you would write down the numbers and get to the answer quickly.

One could say that pencil and paper are augmenting you, in a certain way. By augmenting I mean increasing the range of thoughts and operations you can perform, freeing you to focus on small parts of the problem at a time and eventually come up with the right answer.
Note that you are still using your mind and making a mental effort, however the burden on your short-term memory is greatly reduced by using an appropriate medium to aid on such effort.
Today, technology for "knowledge work" seems to be highly focused on outsourcing our mental capabilities: we google to find answers in seconds, we use calculators, everything is at out finger tips in the shape of an app designed to replace our mental tasks completely or almost completely.
On the other hand, when it comes to technologies that could work with us to augment our capabilities, I would argue that we don’t have as much richness and diversity as we do in the outsourcing department.
The point is not to criticize any of such technologies per se, one could even make the point that something like googling augments our capabilities in a certain way, but yet to put in focus the motivation behind their design, they are much more designed for passivity than they are for active engagement and thinking.
The goal of this essay is to argue for the importance of a different kind of medium for dynamic thinking that takes into account the diverse range of modes of human understanding, focusing on expanding our cognitive abilities.
From Plato to modern representations
There has been a tradition in philosophy, dating back to Plato, of associating technology or technique as a distinctive mark of mankind, something we use it to overcome our own fragile human condition.
The idea comes from the myth of Prometheus who, as myth has it, gave humanity control over fire (a metaphor for technique), so that humans could harness nature to overcome their own physical limitations.

The very fundament of the conception of technology base it as complimentary tool to our human capabilities. However, something happened throughout history , and technology seems to have gone down the path of replacement: speech, communication, calculations, physical prowess… We use machines to do it for us.

It is my belief that technologies aimed at helping humans to do knowledge work are the best example. They are designed for passivity, and the many capabilities we have are not integrated in such design.
I think this should be corrected, and a much bigger emphasis should be put in appropriate mediums for thinking that take into account meaningful contextual representations to augment our ability to think.
Contextual representations
By representation here I will specifically mean data structures and program-like structures that we can use to generate insight. This definition belongs to one of my favorite technologists Brett Victor, and is taken from his lecture: "The Humane Representation of Thought".
When we use pencil and paper, we can get a sense for the value of such representations. The number is a representation, the plus sign is another. These shape how we understand the world, and are heavily dependent on the media in which they are expressed.
For instance, it is very hard to meaningfully represent temporal dynamics using pencil and paper. One can write a "time" axis, but it would not be the same as having a medium that is actually dynamic, that represents behavior in time.
Pencil and paper as a medium for thought expression are limited in the kind of representations they allow just as any other medium.

Representations are mental tools we can use, manipulate and develop to build a complex understanding of the world. In essence, they are building blocks, the "legos" of our minds to build ideas.
However, as I said before, they are highly dependent on the medium they are expressed on. We can think of mediums like the pencil and paper examples as microscopes for our brains allowing us to see things we were not able to see before, by reducing the burden on our short term memory through the use of rich representations inserted in meaningful contexts.
Good thinking mediums have the potential to enhance us through facilitating access to richer representations that we can work with to gather insight, solve problems and think about things in detail.
Working with such mediums broadens the range of thoughts we can have, just like a microscope broadens the range of things we can see.

Such representations are shared across cultures (numbers and equations are widely used in our culture to do math, for example). These mental tools are important, they are the backbone of our ability to think complicated things.
Take this example for instance, imagine you are at home and you want to go to a certain restaurant. So you open your smartphone and you type on GoogleMaps the address to your restaurant of choice and the app shows you the way.
However, the app only goes so far. Now that you have the map in front of you, it is up to you to decode what those images and indications mean to get to your location. Because you know what they mean and you can quickly navigate these directional representations, you can get to your destination easily.
Can you imagine how hard it would be if instead of showing maps with images and arrows, the app was to show you only a verbal description of how to get to your destination?

It is so much more unnecessary mental work to decode the written version than it is to understand the map! Why? Because there are such things as good and bad representations. In this case, maps with arrows are good representations to guide people towards a certain location. Text in this case is not.
One can say that maps are an appropriate representation for direction information. However, if we were discussing history or philosophy, doing it with maps would probably not work as well. We would lose information that written language could offer to describe difficult, fine-grained abstract concepts. One will use maps for guiding people and use words to explain existentialism and the dreadful pain of existence.

The main point here is to understand that mediums and representations should be appropriate for the problem, task or context at hand and using the wrong medium or wrong representation will naturally hinder our ability to think about that particular subject.
When it comes to thinking about things, for some accident of history we have stayed focuses solely on symbolic representations as the main channel through which we represent thoughts and communicate ideas.
However, as technology evolved we now have the unique opportunity in history to reconsider our forced choices and evolve from these rectangle boxes we use to do all of our knowledge work.
Knowledge work does not have to be this constraining and physically limitating process of typing and clicking. It should integrate a more diverse and humane range of channels for absorbing and processing information.
Thinking Through Representations
Representations directly influence how we think. Language is probably the ultimate example of the power of representations. When we name something we gain control over it, it is empowering.
Representations allow us to internalize the things we wish to understand. When you think about mathematics for example, you probably think of this:

Complicated symbolic equations on a big board. This idea of math existing mostly on the realm of symbols is a byproduct of an evolution of mathematical notation, in other words, an evolution of mathematical representations. Take this minimal example:
- A number squared.
- x²
Just as in the case of the maps, it would be unthinkable to write down a description for x² every time we mean to use it, but there was a time when math was done through writing and not through the notation we are used to today.
Such an evolution was vital for mathematics because this simple and compact representation allowed us to effectively compress information, greatly enhancing the range of complexity mathematicians could tackle.
The examples I showed are all about taking complex structures and packaging them into something that displays their core intuition immediately.
How People Think
Brett Victor brilliantly argues in his presentation "A Humane Representation of Thought" that today’s mainstream mediums for thinking and solving problems, such as computers and tablets, are reductive, because they emphasize only a small subset of the range of modes of understanding that we possess. He comes from kind of a piagetian perspective, defending that we have the following modes of understanding:
- Visual
- Aural
- Tactile
- Kinesthetic
- Spatial
He also mentions other takes on the matter, like Jerome Bruner’s: enactive, iconic and symbolic. His argument defends that the technologies we use constrain the majority of these modes, with media accounting for only a subset focusing on visual and symbolic representations.
We are constrained by this low bandwidth channel for knowledge expression, which goes in contrast to the diverse range of thinking and understanding modes that we have. We are encaged by these screens that are not designed to incorporate the entire spectrum of thoughts we can think. We can type, click and draw and that is it! Better said by Victor:
It constrains the range of our intellectual experience
He defends the position of a dynamic medium, one that would span the entire set of human capabilities in a way that would free us rather that enslave us like these media consumption devices that we carry around all the time.
Although I think that AR and VR could, under the right circumstances (which are definitely not guaranteed), mitigate part of this problem, his critique is unique because it puts into question the nature of how we do knowledge work as well as the materials we use to do it.
Two great posts by Michael Nielsen discussing a subset of this potential dynamic medium are "Reinventing Explanation" and "Towards an exploratory medium for Mathematics" .
In these essays he tackles the need for such a medium under the lenses of meaningful interactive explanations (the former), and how one could start conceiving such medium for exploration of mathematical concepts (the later). On the later he showcases a prototype of an environment adequate for discovering and understanding a piece of mathematics called Singular Value Decomposition.
The points that comes across from Mr. Nielsen is that we need to rethink both how we understand what an explanation is as well as the "traditional" mediums we use to produce them.
From inadequacy to understanding
Ever since I was a kid, I felt a simultaneous fascination by theoretical topics like math and physics, and a dreadful feeling of inadequacy with regards to my true ability to pursue such topics.
Although I could always perform in school, I still felt that something was missing: a tool, a way to express what I was thinking in a natural and fast manner to allow me to play around with my thoughts without getting bored or frustrated.
This feeling pursued me my whole life, and things started to click for me on my masters when I got into deep Learning and programming with python. From that moment on, I was much more engaged, having this simple tool that could make my thoughts a reality in a speed that I never thought possible.
Although I was completely hooked and fascinated, the feeling did not go away. Even though I could write programs that would make my thoughts came true faster than any other tool I had tried before, it was (it is) still not fast enough.
Consider this simple example of trying to understand what a derivative is. Even better, let’s try to outline the workflow necessary for what it would take to understand that the derivative is the slope of the line tangent to a function at any give point.
I want to show how I would use my current tools to observe these behaviors and get to some satisfying conclusion about the relationship between a squared function and its derivative starting from the definition of derivative:

My first step would be to draw a function to get a sense for what is happening. After that, I would write a simple interactive tool with matplotlib and widgets to plot the behavior of the secant line that goes through two points on the function:
# %matplotlib widget
%matplotlib inline
import ipywidgets as widgets
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
sns.set()
def plotDerivative(a,h):
"""
Print the current widget value in short sentence
"""
x = np.arange(-5,10,1)
f = lambda i: i**2
pointA = (a,f(a))
pointH = (a+h,f(a+h))
m = (f(a+h) - f(a))/float(h)
print(f"m = {m}")
y = m*(x-a) + f(a)
plt.plot(x,f(x), color="blue")
plt.plot(x,y, color="red")
plt.scatter(pointA[0],pointA[1],c="purple")
plt.scatter(pointH[0],pointH[1],c="purple")
plt.xlim(-20,20)
plt.ylim(-20,30)
widgets.interact(plotDerivative,a=widgets.IntSlider(min=-10, max=10, step=1, value=5),h=widgets.IntSlider(min=0, max=5, step=1, value=1))

At this point, I have one question: Does the average time that it takes to generate a simple visual tool like this makes this approach worth it?
Well, there are much better ways to visualize this with code, but the point is to understand the derivative when you don’t already know what the derivative is (in this case as the slope of the line tangent to the function at any given point), with this approach I can see some meaningful behavior (the the line becomes tangent to the function as one point approaches the other) but at the cost of a few minutes to get the code right.
The issue here is that for such a low dimensional representation of this problem, the time that it takes to get here makes it a non-ideal thinking tool at least for this topic.
Of course I could have generated a better tool at the cost of more time coding or some approach that would tackle the core of derivatives more closely, however, from the perspective of someone trying to quickly visualize a relationship and get to some causality about what is going on, it does not make sense to use a tool that is not designed to accompany the fast pace of thinking.
I am trying to think about derivatives, and thinking happens fast, if I am to use a tool to think, the speed of the tool should match the speed with which I have ideas about the topic I am investigating.
However, I still think that today coding is probably one of the closest things we have to a thinking tool on the general sense when compared to other approaches, which begs the question of what could be better?
To exemplify a contrast, let’s take a look at a tool called desmos, particularly the graphing feature, which makes it easy to visualize any simple function in an interactive 2-dimensional grid.
The following is a quick video of the workflow for showing the resolution of the same problem as before but in desmos:
Quick and easy! It still requires some intuitive practice and it might take a minute, but it is much quicker than coding and offers more interactive features quickly, therefore making it a better medium for basic math graphing workflows than coding. I can just write the symbolic expression I know so well from my high-school math days, and get the graph plotted instantly!
Now, this was not supposed to be a lecture on derivatives or a lecture on why desmos would be better than programming, but to showcase how a more appropriate tool for a given topic can enhance our thinking about it.
A really nice math thinking workflow can be seen here by Grant Sanderson from the "Three Blue one Brown" channel:
You can see how he switches between desmos, programming and writing because each medium offers the right type of environment where one can manipulate the right type of representations in the appropriate way to get the meaningful insight one needs at the time one needs it.
The lesson to take from here is that not only there are appropriate representations, but the medium matters a lot and it should integrate properties like controlability, interactivity, immediate feedback and many other elements to make it an ideal thinking medium.
Final Thoughts on a Sketch of a Thinking Medium
Representing knowledge quickly to accompany the speed of our thoughts is all about working with environments that allow for quick and effective contextual compression of information.
The idea of interaction as a fundamental component of thought itself is not new. Douglas Engelbart is one of the foundational thinkers who introduced the idea of interactive Technology. In the last 10 years authors like Barbara Tversky, Michael Nielsen, Ken Perlim and Brett Victor have defended in some way or another for the need for appropriate thinking technologies.
Pencil and paper are great tools for thought compression but they are limited to a very low dimensional representation of the range of things we can think.
We know today that taking notes during a class enhances learning because as we take notes we engage with the material meaningfully. What we don’t know is the potential drastic improvements in learning that we could observe if we had more appropriate mediums to express our thoughts matching the speed with which we have them.
Just like the invention of the computer brought significant benefits to our ability to think about more complex processes that could not be done analytically, a more dynamic and humane medium for Thinking and solving problems would have similar positive effects.
As for the elements of such a thinking space, I recommend taking at look at Mr. Victor’s body of work at as well as Ken Perlim’s chalktalk. I would say that the fundamental elements are:
- direct manipulation
- interactivity
- immediate feedback
- ability to provide a holistic view of the system at all times
- in context manipulation
These theoretical building blocks should be at the core of the creation of thinking environments.
Rethink your Workspace
As an example of one thing you can do today is to rethink your workspace. Before, I thought of knowledge work as this sitting down activity that involves clicking, typing, sometimes drawing and taking handwritten notes, but after thinking deeply about this topic and heavily influenced by people like Brett Victor, Michael Nielsen, Barbara Tversky and Ken Perlim, I decided that the best thing for me would be to redesign my workspace to account for standing up and moving more.
That is not nearly the end of the story, because technology is the main core of the issue but it does help with engagement, as well as feeling that thinking is something we can embody! The philosopher Friedrich Nietzsche used to go on long mountain hikes to help him think. That was not something separated from his workflow, but integrated as part of his thinking process.
Now, I have whiteboards and my monitors can be extended to be used standing up and moving around (not just in the desk). I also want to integrate a desk touchscreen so that I can stand up but still be able to effectively interact digitally with my computer.
The main idea here is to create an incentive for movement rather than centralizing everything around the idea of sitting down.
Although this is a step forward, it does not tackle the main cognitive flaw of today’s thinking technologies. That will have to come from an entire collective of people thinking about better ways to build and communicate knowledge as well as engaging in society to make this a priority.
One of the dangers with this is that the logic of capital tends to be short sighted and bad choices tend to colonize the entire technological landscape, so it’s up to the people to realize that the little rectangle cages that we have been fed are not human enough to be the future of our thinking space.
Thanks and see you next time! 😉 .
If you liked this post follow me here on Medium, connect with me on Twitter and LinkedIn.
References
- Thought as Technology
- The Humane Representation of Thought
- Media for thinking the unthinkable
- Towards an exploratory medium for Mathematics
- Reinventing Explanation
- Making invisible understanding visible
- Dynabook
- Seeing spaces
- The Unreasonable effectiveness of Mathematics
- Communicating with interactive articles
- Welchalabs
- Petros Vrelis interactive Starry Night
- Interactive dynamics systems by Brett Victor
- Pleasure of learning
- Visual explanations improves learning
- People favor simplicity on explanations
- On diagrams and forms of knowledge representation
- Illustrations help focusing attention on explanative information
- Diagrams as a microcosm of cognition by Barbara Tversky
- Personal Dynamic Media
- Nielsen’s notes on chalk talk
- Ken Perlim’s ChalkTalk