In part 1, I discussed why we need to move away from MDE as an input variable to an output variable and make it a function of the available sample size. In this post, I’ll share what the rearranged formula looks like, along with a hacky Microsoft Excel solution.
As a reminder, these are the formulas for sample size where:
- α: the selected level of significance - β: the selected power - σ: the standard deviation - μ1 or p1: the baseline mean or proportion - μ2 or p2: the proposed/expected new mean or proportion (this is where our…
The Minimum Detectable Effect or MDE has always baffled me, and not because it’s a difficult concept to understand but rather because most people working in the field of experimentation have been using it counter-intuitively for so long.
For the record, I don’t think any of what I’m about to write is groundbreaking, but having worked in this field for many years, I’ve not seen anyone talk about it so I thought I would. Who knows, maybe you’ll disagree with me.
The MDE is one of several inputs used to calculate the sample size required for an experiment along with:
Sometimes we’re normal
Symmetrical around Mu
But like yin and yang,
Positive to negative
Sometimes we skew
Like a null hypothesis
you love to reject me.
And all because you believe,
That we’re just an example
Of two different samples
From two different populations,
But if we put aside our differences
We can create a better hypothesis
You’re seeing things
That just don’t exist
But the things that are real
You continue to miss.
This isn’t a mistake
Just look in the mirror
What you’re seeing is random
It’s a type 1 error
You say I’m needy, I’d say I’m…
It’s nice when everyone is in agreement about something. It’s even nicer when that agreement is around your way of thinking. But let’s face it, if everyone agreed with each other all the time, the world would be a pretty boring place.
Our uniqueness, whilst making us special, can often lead to conflict at work and we regularly find ourselves in situations where we’re going head to head with people because we don’t share their views and they don’t share ours.
These head to head battles of differing opinions are exhausting and if you regularly find yourself in them, either…
Things I’ve learnt and mistakes I’ve made
As we say goodbye to what has been quite possibly the most demanding year of our lives, well at least my life, I wanted to share a few things I’ve learnt as well as some mistakes I’ve made along the way when starting in my new role. I joined Gousto back in May 2020, during the middle of the first lockdown, and whilst the onboarding process was seamless, the experience was anything but. Don’t get me wrong, it was never going to be easy as I was building a new team and function…
This is quite possibly the easiest blog post I’ve ever written as well as my all-time favourite. Last night we had a virtual data team Christmas party where we played many games, one of which was a drawing competition where we had to draw a Christmas themed picture on Paint.
Now drawing anything on a computer usually requires a very special set of skills, software and equipment… none of which you’ll find within a team of data professionals, which is what made the outcome even better.
Although the drawings below look like they were made by a class of 7-year-olds…
Product Analytics lead at Gousto. Founder of CRAP Talks. Twitter: @dodonerd