Data-driven off a cliff - Anti-patterns in evidence-based decision making

Thu, 20 Oct 2016 18:30 - 21:00

Ebisu Garden Tower 32F Indeed Tokyo Tech Office

Yebisu Garden Place Tower 32F, 20-3 Ebisu 4-chome, Shibuya-ku, Tokyo 150-6032

kuma Kenji Hayashida Shunsuke Kai 中村純 Norbert Nemes Dimitar Dinchev Shweta Verma E. Tera Yoshiaki Yamada Trung H. Tran Kuang  Ye T.Hirayama Jason Ball Leonard Chin Joeper Serrano Kyouhei Ohno + 106 more participants

Registration for this event is closed.

Free admission
Indeed will provide food, beer, and other refreshments

Event Schedule:
19:00 Door opens & food and mingling - Please eat while it's fresh!
19:30 Tech Talk (1hr)
20:30 Q&A and further mingling and beer
21:00 Door closes Event Schedule

Metrics and analytics are crucial for building successful products at scale. They're a powerful weapon. You can hit your target, or you can blow off your own foot. You can accidentally or intentionally misuse them in all sorts of ways, hence the cliche about lies, damned lies, and statistics.

In this talk, I'll explore some of the ways you can use data to drive you off the cliff. I'll talk about anti-patterns in evidence-based decision-making. Sometimes it's a lack of math or statistics. Sometimes not understanding your domain or how a data-driven approach should be applied. Sometimes it's not appreciating the limits of what data. And sometimes it's that empiricism is unnatural, inhuman, and inhumane. You can build an exquisitely bad decision on a rock-solid foundation of data, and I'll teach you how.

In spite of that, data-driven decision making is not quite completely awful. Fallacies, biases, misconceptions, and over-confidence are the pernicious diseases that lead to many bad decisions, but you can cure them, or at least treat them. It's still possible to make good decisions, like your excellent decision to come to this talk. In spite of all the flaws, there is still good inside the data.

I'll talk about why and how Indeed measures despite the pitfalls, and how data-driven experimentation has been a critical factor in our dramatic growth. I'll talk about some of the missteps we've made in using data and what we learned from them. I'll abuse concepts from physics, pomology, and quantum hermeneutics to help you reason about metrics and achieve better results. You'll leave this talk smarter than you came in, and with evidence to prove it*.

Ketan Gangatirkar is a software engineering manager at Indeed. For the last 7 years, he's been helping people get jobs. He used to think he had great ideas, but being wrong all the time cured him of that. Now all he wants is evidence. He was analytical before it was cool. It's not cool, and it never has been, but he's still hoping. Someday. Someday

About this community
@IndeedEng TOKYO

@IndeedEng TOKYO

@IndeedEng is a technical presentation of our approaches to building a successful large-scale consumer application.

Join community