To me, Robert Pirsig’s bestseller Zen and the Art of Motorcycle Maintenance (1974) was definitely one of the more engaging reads I’ve come across. The aim of this article is not to give a synopsis on the Metaphysics of Quality, but to distill some insights that might aid data scientists when getting discouraged with their project.
Zen and the Art of Motorcycle Maintenance
For those who have not read it – or those in need of a quick recap – the book interweaves the philosophical life journey of the author with the fabric of an All-American motorcycle road trip. It provides a thesis on the concept of quality, unifying the romantic and the classical perceptions of the world. As an illustration of those two viewpoints, the titular act of motorcycle maintenance serves as an extensive anecdote throughout the book. The romantic soul is primarily interested with how the motorcycle drives, looks and feels. A mechanical failure is nothing but a source of frustration. The classical soul, on the other hand, is concerned with the inner mechanisms of the machine, wants to know exactly what each part does. For them, a breakdown might actually pose an interesting challenge.
It is not hard to see the analogies with programming and Data Science. When faced with a malfunctioning piece of software or nonsensible results, many users will throw up their hands in despair. It takes a certain fortitude and analytical mindset to dive into the underlying data sets or error messages. The classical soul is at work here.
However, even one with a strictly classical view on the world is not immune to discouragement. Who never wants to toss their laptop in a dark corner after yet another incomprehensible error message, who stays motivated after running five different machine learning algorithms in vain?
There is no magic pill or deep philosophical insights that keeps you motivated at all time. Nonetheless, the more awareness you have on the causes that halt you in your tracks, the more likely you are to resolve the issue and get rolling again. Pirsig comprehensively outlines common reasons why people get stuck and lose Motivation. If it worked on motorcycles in the seventies, why wouldn’t it work in the modern era of data science?
Gumption traps
I have never attempted to disassemble a motorcycle into a thousand pieces and try to reconstruct it afterwards, but apparently it is no easy feat. Pirsig was not a professional mechanic either; for him it was a major learning experience. Like any human being, he frequently got lost, stuck, demotivated, bored, frustrated. He categorized all these feelings under the common denominator of gumption traps, and provided concrete suggestions on how to power through them.
Gumption is not necessarily a word we use in everyday life, so a definition might be in order. There are multiple (including terms such as shrewdness and initiative), but ‘common sense or resourcefulness’ seems to capture its meaning fairly well. The gumption trap, in turn, is an event or mindset that results in losing interest or confidence to complete a project. The ‘trap‘ part refers to the self-reinforcing feedback loop that kicks in. If you get discouraged, you will put less effort into the project, leading to disappointing outcomes, even more discouragement, resulting in… You get the picture.
The traps may be divided into external factors (setbacks) and internal ones (hang-ups). The latter is further decomposed into value traps, truth traps, and muscle traps. Pirsig discusses both common causes and solutions to all of them. Here, I try to translate them to specific applications in the domain of data science.
Setbacks
Pirsig mainly attributes external setbacks to a lack of knowledge. Not just any lack of knowledge; it is what strategy consultants refer to as ‘unknown unknowns’. It is hard enough to fill knowledge gaps that you are aware of, yet being ignorant that such gaps exist is arguably worse. In the latter case, setbacks may seem like acts of the gods, because you are simply unaware of the obstacles you face. It’s like navigating a reef without a map.
In data science projects, the lack of knowledge and understanding is often substantial. This is not a reflection on one’s intellectual capabilities: real-world projects are generally messy and poorly defined, being shaped along the way.
This is where the ‘scientist‘ in ‘data scientist’ should spring to life. Pirsig’s general advice is to be slow and meticulous moving forward. Don’t jump directly into coding, but plan ahead. Take the time to get a proper grasp on the project, the expectations, the potential obstacles. Frequently stop to reflect and analyse.
Concretely, the following steps may be of use to mitigate setbacks:
- Take notes. When making plans and talking to stakeholders, take some time to jot down your thoughts. Ensure that expectations are clearly aligned. Writing things out helps to formalize and structure the process.
- List requirements and objectives. Get a grasp on what conditions need to be met and what constitutes a successful project. Be as specific as possible. Using SMART criteria is often helpful.
- Define clear research questions. What insights should you obtain from the data? What capabilities should your new algorithm have? If you want to find the right answers, make sure you ask the right questions.
- Plan in advance. Break down the project into a logical sequence of comprehensive steps. What activities are needed for each step, how long should they take? What data and tools are needed to successfully complete the project? A clear pathway drastically increases chances of success.
- Frequently reflect. The previous steps are not set in stone once you set off. Projects are dynamic, objectives and questions may change, newly arising obstacles should be resolved. Take the time to reflect during your data science project, and don’t be afraid to alter course.
Naturally, there will always be unanticipated obstacles. Plans are never perfect, and some external factors will remain out of your control. Setbacks will inevitably arise, yet a careful and systematic approach will preserve focus on the project and reduce the risk of being thrown off your game. Remember that data science is rarely a straight road: drive slowly.
Hang-ups
Unlike setbacks, hang-ups stem from internal factors. Projects do not only get stuck due to external circumstances or unforeseen obstacles, but also due to insufficient information or the wrong problem angle. When working with data or code, it is easy to get completely caught up into a tiny aspect of the problem; a statistical outlier, a bug, a visual detail. Often the best fix is a quick break, reassessing whether you truly face a problem within the grand scheme of things. If it is, you might want to go back to the drawing board to tackle the problem step by step.
Pirsig divides hang-ups into three categories: value traps, truth traps, and muscle traps. The definitions and examples follow below.
Value traps
Value traps are hang-ups that occur within the mind. People are not machines: they want their work to be meaningful, to see their hypotheses confirmed, to book successes. When spending a lot of effort on a certain solution direction, it is hard to re-evaluate your frame of mind (i.e., your values). However, data science is a dynamic process, and one must be flexible in dealing with new facts and information. Also keep in mind: the solution you are invested in, might not be the best solution from the business perspective.
The typical answer to combat value traps is to take a step back when experiencing any of the points below, re-assess the problem and facts at hand, gather more information when needed. Often, getting out of your state of hyperfocus is all that is needed.
- Egotism: It is easier said than done to operate bias-free. Data might not support the conclusions you hoped or expected to see. The machine learning algorithm you thought was great does not provide the level of precision you desire. Your ego might kick in and convince you to ignore inconvenient facts, to see patterns that are not there. It’s a data-driven field though: be skeptical, let the data convince you.
- Anxiety: Let’s be honest, data science is not easy. You need knowledge in many areas, and will not always feel confident tackling a new project or challenge. Whether it’s Git, Bayesian statistics, PyTorch or Spark, detail the requirements from the onset and brush up your knowledge on the tasks at hand. There is nothing wrong with learning on-the-fly, but trust you have the core competences to get the job done.
- Boredom: Without seeing progress, boredom is bound to set in. Pirsig suggests to stop immediately at the first signs of boredom, yet that is rarely possible in practice. A coffee break or talking to a colleague might be enough to reinvigorate you. After a morning of fruitless debugging, perhaps you can spend the afternoon on coding a new feature instead. On the longer term, a switch to a different team or project may be a solution.
- Impatience: We all want to see results yesterday rather than now. When stagnating, abandoning a project seems appealing. Again, Pirsig’s suggested solution – indefinite project time – is generally not applicable. However, if you are always busy patching things up, always running from fix to fix, this might be a sign that there is insufficient time or support for the project. A realistic timeline and sufficient flexibility to reflect are prerequisites for succes.
Truth traps
Truth traps relate to either misunderstanding feedback received by the environment or a mismatch between questions and answers. As such, the ‘truth’ gets distorted, often causing frustration because there is seemingly no path to a solution.
- Misunderstanding of feedback: This can be feedback from an end-user, a stakeholder, your compiler, or simply an outcome. Incorrectly valuing feedback can cause you to feel lost or focus on the wrong fixes. Take the time to properly absorb feedback and ask for clarification if needed. In case of non-descript error messages: Google them. They may not be as cryptic as they appear at the surface.
- Relying on yes/no duality: Data analysis does not always provide a clear-cut yes or no answer. Unexpected findings and outliers often arise, and although insights should be presented in a comprehensive manner, it would be wrong to oversimply them. As Einstein put it:
"Everything should be made as simple as possible, but no simpler."
- Asking the wrong questions: If, despite your best efforts, you keep struggling to answer the question, you might want to reconsider whether the research question matches the context of the problem.
Muscle traps
Data science requires many skills, yet physical prowess is typically not listed among them. As you will see shortly, it has nothing to do with your bench press or 10 mile records. In general, muscle traps encompass the interaction between you, your computer and your work environment.
- Inadequate tools: To get the job done, it is essential to have the right tools. In data science, both hardware and software fall in the category of ‘tools’. Are you frequently waiting hours at end for your simulation runs to finish, does the right-side button of your mouse malfunction half of the time, is Panda’s slow data ingestion a constant source of frustration? You might want to re-evaluate the tools that you use. Make notes of the problems you encounter and look for alternatives.
- Poor working environment: Simple environmental factors such as the sun on your screen, the high office temperature or distracting music can severely hamper productivity. Having been working from home for nearly two years now, take your home office serious as well. Don’t spend your working days stooped over a 13" laptop at the kitchen table. Invest in a proper chair, a monitor, a keyboard. It will pay off in the long run.
- Muscular insensitivity: Basically, don’t smash your keyboard and don’t throw things at your monitor. It won’t fix the problem.
Final words
I realize that most of the solutions posed in this article seem obvious, trivial even. As humans, we are fallible though. We work for months with a docking station that does not charge. We get frustrated by the same error message over and over, without ever look up what it actually tells us. We keep overclocking our laptop day and night, although it is obvious we need a cluster for the job. We endlessly keep tweaking the neural network we spent so much time on, ignoring the red flags telling us it simply doesn’t work.
Pirsig’s core message is not to stop for a coffee or replace your mouse. His argument is to take a slow, conscious and deliberate approach to your project, to frequently zoom out and assess the big picture. As a data scientist, a clearly defined and well-informed action plan is essential for success. When feeling stuck or frustrated, a slow and systematic approach is needed to troubleshoot the problem and address the underlying obstacle.
Everyone will experience the gumption traps listed in this article from time to time. We don’t always have the right information or skills at hand, obstacles will pop up, and yes, even mundane feelings like boredom will kick in. That’s ok. As long as you have awareness of the traps you might walk into, and have a plan to get out of them, you are already halfway reaching your goals.
So, next time you feel frustrated with your project and are ready to throw in the towel, turn off the screen and make yourself a double espresso. It will make you a better data scientist.
Takeaways
- A gumption trap is a loss of enthusiasm or initiative, experienced when feeling stuck in a project. It tends to be self-reinforcing, as reduced enthusiasm lowers the chances of success and vice versa. In data science, underperforming algorithms and inexplicable data sets are typical drivers for gumption traps.
- Setbacks are exogenous gumption traps, often stemming from a lack of knowledge and awareness. A slow and systematic approach to your data science projects aids in timely identifying knowledge gaps, obstacles and bottlenecks.
- Hang-ups are endogenous gumption traps. Typically, they involve clinging to misleading facts or working with insufficient information or tools. Hang-ups can be divided into value traps, truth traps, and muscle traps.
- Value trap are primarily mental blocks, often involving a reluctance to abandon a certain hypothesis or solution approach. The more effort we put in, the less willing we are to accept inconvenient data. Sometimes it helps to take a break, zoom out and verify whether you are still solving the right problem.
- Truth traps involve misinterpreting feedback or trying to answer the wrong questions. Attempt to clarify ambiguous feedback (ranging from error messages to stakeholder comments) and re-evaluate whether answering the research questions resolves the problem at hand.
- Muscle traps entail working with insufficient tools or in an unproductive environment. Ensure you have the right software, hardware and office supplies for your job. Critically assess your bottlenecks and frustrations, and evaluate whether a different tool might resolve the issue.
- Ultimately, avoiding – and getting out of – gumption traps is about mindset and self-awareness. By deliberately taking a step back and slowing down, it is possible to adopt the classical mindset, systematically troubleshoot the problem, and remove to obstacle to move forward.
References
Pirsig, R.M. (1974). **** Zen And The Art of Motorcycle Maintenance: An Inquiry into Values. Vintage Publishing.
The Free Dictionary (n.d.). Gumption. https://www.thefreedictionary.com/gumption
Wikipedia (2021). Zen and the Art of Motorcycle Maintenance. https://en.wikipedia.org/wiki/Zen_and_the_Art_of_Motorcycle_Maintenance#Gumption_traps