Monday, July 12, 2021

Stay Anonymous and Give up Often

 In Scene 3 of Act 3 of Troilus and Cressida, Ulysses reframes Achilles’ past accomplishments in the context of human perception:

 

“To have done is to hang

Quite out of fashion, like a rusty mail

In monumental mock’ry.”

 

In other words, present accomplishments matter, and past victories are soon forgotten, or worse. While this is a (self-interested) attempt by Ulysses to motivate Achilles to get out of his tent, it also reflects an essential truth about how we see the world.

 

As limited information-processing machines, our attention is valuable and in short supply, and our survival depends on carefully allocating it. Because recent performance is more relevant for predicting future behavior than distant past behavior, it seems intuitively obvious that we would value recent accomplishments more highly. As an extension, in a larger social context we naturally focus our limited attention on the recent achievements and failures of those around us.

 

If this model of human behavior is accurate, then continual recognition as a relevant leader seemingly requires constant action – otherwise, the accomplishments of others will seemingly extinguish our public presence. Since this continual action is necessarily challenging, this model of relevancy rewards perseverance. Without perseverance, we would soon give up against the constant headwind of difficulty and fall back to the shadows of anonymity.

 

The value of perseverance is also lauded in the business world as crucial activities such as growing and sustaining businesses require a constant slog through a dense forest of obstacles – each one beckoning the weary entrepreneur to pause for a little while or try a different approach. Carefully committing to a direction and sticking with it until high-level information suggests otherwise (sometimes a challenging task of its own) is correspondingly an essential strategy to success in such an environment.

 

While persistence is seemingly as important to leadership as business, there might be an even higher-level or general-purpose value of this virtue.

 

As a small part of our work at my organization, we build and experiment with deep reinforcement learning models. Briefly, this combines a clever structure of an agent learning through interaction with an environment and large artificial neural networks. This generalized learning model can be applied to learn an extensive range of tasks, including playing Atari games, evaluating the design of prosthetics, and automating stock trading. Notably, most problems amenable to large volumes of trial and error and quick feedback can be attempted to be solved with deep reinforcement learning without too much effort today (although note the careful use of the word attempt).

 

A critical setting when training a reinforcement learning model is the exploration rate – how much time should be dedicated to trying random actions (exploration) compared to patiently executing activities known to produce a reward. In the beginning, a given reinforcement learning model has no information about actions in an environment that will result in a reward. In this case, sticking with a single action – chosen with limited or no useful information – would catastrophically hinder learning.

 

Clearly, the optimal approach is to try out many possible actions (to favor exploration). Then, as the agent learns more information about favorable actions, the optimal setting of this parameter shifts towards exploitation (limiting exploration). Unsurprisingly, gradually shifting from exploration to exploitation throughout learning can improve learning and performance and is a common strategy in reinforcement learning today.

 

While artificial neural networks (thankfully) don’t get stuck in tents, they can spend too much time chasing down actions that lead to suboptimal rewards, which underscores the importance of this setting.

 

Our human lives benefit from a similar process. When we’re not good at something – and are still learning - we benefit from failing fast and moving on to another approach. If we get stuck, our time is often better spent trying other methods or strategies.


Perseverance and hard work are essential values. However, maybe giving up - and knowing when to give up - is an equally critical leadership quality? If we don’t give up (and fail fast to move on to our next goal in situations with limited information), we might not reach peak efficiency in pursuit of our dreams. Because giving up and failing is a lot easier when we’re anonymous, maybe anonymity is - counterintuitively - a valuable and desirable supporting tool for creating an environment for iteration and exploration.

2 comments:

  1. Rob,

    Love your post! Knowing when to give up (some people say tactically retreat, haha) is definitely a key piece of leadership. This reminds me of the sunk cost fallacy, which is basically following through with a decision despite clear and convincing evidence that moving forward will not provide a return on the investment. For any I/O Pysch majors, I think this is congruent with Cialdini's principle of commitment and consistency, where people want to show that their actions are in line with their words. In other words, people will commit to an idea and stick with it because inconsistency is viewed as a negative personality trait. Thanks for sharing!

    ReplyDelete
  2. Fail fast is indeed the mantra at my company in the AI space. However, to fail fast we also need the ability to run experiments fast. Right now, it takes us months to do an experiment and hence the cost of failure is high and we are limited in the number of experiments we can do. We're working on the ability to run experiments in days as opposed to months so we can fail fast, run a lot more experiments and find that one golden nugget that is useful.

    Thank you for sharing Rob!

    ReplyDelete

Note: Only a member of this blog may post a comment.