Welcome everyone to the first edition of my newsletter! There are so many newsletters out there and you've chosen to read mine. Will it be filled with interesting insights or tedious navelgazing? Let's find out together.
Lately I've been ruminating about how much value is really created by analyzing data. Analyzing data is what I have become accustomed to doing to be valuable, and now thinking about what's next for me I am forced to consider whether it is a thing I should continue emphasizing.
I have often argued that one of the most valuable types of data at a company is from A/B tests. Here is how a typical A/B test goes: Someone has an idea for changing the product and they build a prototype of that idea. They set up a test and randomly try out this idea for some fraction of the users for a few days or weeks. Then they estimate whether their idea provided benefits according to some criteria, and decide whether the idea is good enough to keep.
We can roughly break that up into three phases: