Carpe Donut...

Each day is quite different and unpredictable. I know. I can drop the mic and call it a day right? Kidding aside, what I have learned along the way is the importance of keeping a few rituals.

Even if you aren't running a boutique data visualization company--things can go from ordinary to bat sh*t crazy in an eye blink.

Build the positive things in as non-negotiable. I need a few hours on a trail with my dog, time in the pool, or reading from  bike trainer. I also spend about 20 minutes coding before I close up the office (keep building skills) and reading something unrelated to work before turning off the bedside lamp. I am committed to completing Infinite Jest by David Foster Wallace.

These are the donuts. I recently heard a quote about worrying. Worrying is like praying for all of the things you don't want. Seize the Donut.

---

---

An appreciation for chaos in the data world organically created an interest in Bayesian probabilities. We don't know anything for certain. In fact probabilities are subjective--we seek a reasonable quantification of an idea or outcome. Naturally frequency data are brought to bear and combined with prior knowledge. An important tenet of Bayesian thinking demands an update to probabilities when new information becomes available across all nodes.

Think of the crazy adventure story for kids, Fortunately by Remy Charlip. Lots of probabilities to calculate.

​Fortunately, Ned was invited to a surprise party.
Unfortunately, the party was a thousand miles away.
Fortunately, a friend loaned Ned an airplane.
Unfortunately, the motor exploded.
Fortunately, there was a parachute in the airplane.
Unfortunately, there was a hole in the parachute.
Fortunately there was a haystack on the ground
Unfortunately, there is a pitchfork in the haystack 
Fortunately, he misses the pitchfork. 
Unfortunately, he misses the haystack. 
Fortunately, he falls in the ocean. 
Unfortunately, there are sharks in the ocean.

(and it goes on from here...)

Quanta magazine asks, "When faced with a difficult decision, should you go with your gut or carefully calculate the attendant risks?" in an insights puzzle titled When Probability Meets Real Life. It reminds me of the Monty Hall Problem introduced in early statistics courses. A contestant is shown 3 doors. There is a car behind one door and the remaining two doors each hide a goat. The conundrum is deciding whether Monty knows where the car is. He allows contestant to be shown one door before making a decision--to keep original door selection or make a different choice. The gotcha is this. If you switch your answer once shown the first door, your odds go from 50-50 to 2/3rds. Click the link above to see why if you are puzzled (and most of us were).

I work with large data sets. My focus is on non-proprietary data but I have access to proprietary data as well. Probabilities depend on subjective assumptions--whether acknowledged or not. Rarely are insights gleaned from a vacuum. We know or think we know something. The secret sauce is having data knowledge, subject matter expertise, and analytical thinking. We need to distinguish between an unknown probability and an inability to specify a mathematical probability.
​

Do we mean unknown through lack of skill in arguing from evidence, or unknown through lack of evidence? The first alone is admissible, for new evidence would give us a new probability, not a fuller knowledge of the old one; we have not discovered the probability of a statement on given evidence, by determining its probability in relation to quite different evidence...For it is not this probability that we have discovered, when the accession of new evidence makes it possible to frame a numerical estimate-- John Maynard Keynes, 1921, A Treatise on Probability

This works quite well at the point of care. If there is information in the medical record unknown to you as a physician your treatment decision depends on personal judgement and available evidence. You must ignore the existence of potential additional information--a type of willful ignorance.

Now if a last minute fax shares additional medical information, this isn't an updated version of the earlier risk/benefit probability. It is indeed a new probability.
​

Probability entails a delicate balancing act. On the one hand, we must recognize the infinite complexity of the individual case. On the other, we must simplify to achieve useful generalizations by ignoring much individual detail. To selectively view only the subset of information we deem relevant, out of all potential information, is to maintain a posture of willful ignorance. This is absolutely necessary in order to generate a mathematical probability statement, but is usually done implicitly and unconsciously. The challenges and opportunities of the future demand that it be done more explicitly and mindfully--Herbert I. Weisberg, Willful Ignorance The Mismeasure of Uncertainty.