Articles
In 2013 and in particular in 2018 and 2019 I spent many, many hours creating various models (in Excel, sometimes also with Simulacion 4.0 ) in order to estimate total market size and competitor size in terms of quantities shipped in various product categories, such as plug-in humidifiers, temperature-control equipment, and—most notably (and most extensively)—ground-penetrating radar for concrete scanning and geotechnical investigations.
Here’s a few lessons and guidelines distilled from that experience, in no particular order:
- “One is none”. Just one model, is no model. Each model is a lens through which you look at a ground truth that you will never know for sure.
- Try bottom-up, top-down, high-level, low-level models, and test them against each other. Use some of the models to cross-check the rest of them.
- Be consistent with your assumptions across all models.
- Comment your models as you go along, as if you were commenting source code. Even better: write it out like a Jupyter notebook, explaining the rationale behind each model step. Make it possible for others to follow your reasoning and poke holes at it.
- Models = software. Version your software and keep a Changelog.
- Model how the product portfolio of a player makes up its revenues. Remember: not all revenues come from product sales.
- Model what the product portfolio of a player reveals about its target segments; cross-check that the outcome isn’t out of whack.
- Use publicly-available data and scale metrics with ratios; such ratios are often consistent within an industry with low business-model diversity.
- Don’t trust off-the-shelf market reports that you can buy from market-report companies. Use such reports as directional and relative input, if at all. If provided with a sample, do not trust anything in that document.
- If you have many inputs and assumptions, run a Monte Carlo. At the very least, examine scenarios. Nominal estimates are garbage.
- Use your gut feeling and ask if the results make sense. Use others’ gut feelings and do the same. Attach a probability to each estimate based on the feedback, and calculate an expected result.
- If possible, generate weighted averages of the predictions of different models (create ensembles). Jitter the weights; see if the ranking of outcomes changes dramatically.
- Check if your results “break reality elsewhere”; ask yourself: “if these results are true, what else must also be true?”
And always keep in mind:
“All models are wrong, but some are useful.” — George Box
