Lessons from scaling Uber and Opendoor | Brian Tolkin (Head of Product at Opendoor, ex-Uber)
Executive Summary
Brian Tolkin discusses product reviews, low-volume experimentation, judgment and an early uberPOOL marketplace lesson. Selected third-party transcript coverage; company stories are his accounts, unverified here. This selection omits Zillow and the closing recommendations. Untimed editorial topics are not video chapters.
Key Takeaways
Define the purpose of a product review
Tolkin says a review may inform an audience and create accountability, but its primary aim should be making the product better. A frightening review can work against that aim. State what the team should gain from the discussion instead of treating it as a performance before executives.
Keep the discussion small and share the artifact
He generally tries to keep the conversation under ten people while distributing the document more widely. The artifact can help the whole team and new hires understand prior decisions. Broad access to context does not require everyone to join the live discussion.
Use a template to improve problem framing
Tolkin describes context, problem, possible solution, risks and success measurement in a product review. He also distinguishes early ideas from nearly-ready launches. The template helps people think, but rigid adherence is less important than clarifying the customer's job and the stage of work.
Check experimental power before promising an answer
At Opendoor, lower transaction volumes make some A/B tests harder. Tolkin asks what effect a test could detect and how long it would take. A long experiment may be worthwhile, but pretending it will answer the question sooner creates false confidence.
Seek other evidence when a standard test cannot work
He discusses customer conversations, observational approaches and other ways to increase conviction. These are not interchangeable with a well-powered randomized test. The point is to name the limitation and improve the decision rather than force a statistically empty result.
Use judgment with a feedback loop
If testing is not feasible, Tolkin says consequential low-confidence decisions need more customer input or another perspective. When a team ships on judgment, it should still watch signals such as support issues or adoption. Intuition is not a reason to stop checking whether the choice was right.
Find the constraint a marketplace actually needs
Tolkin describes an early uberPOOL commuter pilot that lacked enough matching liquidity. Limiting the beta by company did not create a viable pool of rides. His lesson is to identify the operating constraint rather than defend a neatly designed pilot; the historical account is unverified here.
The Conversation Breakdown
Define the purpose of a product review
⏱ Untimed topicTolkin says a review may inform an audience and create accountability, but its primary aim should be making the product better. A frightening review can work against that aim. State what the team should gain from the discussion instead of treating it as a performance before executives.
Keep the discussion small and share the artifact
⏱ Untimed topicHe generally tries to keep the conversation under ten people while distributing the document more widely. The artifact can help the whole team and new hires understand prior decisions. Broad access to context does not require everyone to join the live discussion.
Use a template to improve problem framing
⏱ Untimed topicTolkin describes context, problem, possible solution, risks and success measurement in a product review. He also distinguishes early ideas from nearly-ready launches. The template helps people think, but rigid adherence is less important than clarifying the customer's job and the stage of work.
Check experimental power before promising an answer
⏱ Untimed topicAt Opendoor, lower transaction volumes make some A/B tests harder. Tolkin asks what effect a test could detect and how long it would take. A long experiment may be worthwhile, but pretending it will answer the question sooner creates false confidence.
Seek other evidence when a standard test cannot work
⏱ Untimed topicHe discusses customer conversations, observational approaches and other ways to increase conviction. These are not interchangeable with a well-powered randomized test. The point is to name the limitation and improve the decision rather than force a statistically empty result.
Use judgment with a feedback loop
⏱ Untimed topicIf testing is not feasible, Tolkin says consequential low-confidence decisions need more customer input or another perspective. When a team ships on judgment, it should still watch signals such as support issues or adoption. Intuition is not a reason to stop checking whether the choice was right.
Find the constraint a marketplace actually needs
⏱ Untimed topicTolkin describes an early uberPOOL commuter pilot that lacked enough matching liquidity. Limiting the beta by company did not create a viable pool of rides. His lesson is to identify the operating constraint rather than defend a neatly designed pilot; the historical account is unverified here.
Core Frameworks & Ideas
Review purpose and artifact
Keep the live group small enough to think together, state the goal and share a document that preserves context.
Power before promise
Check detectable effect and runtime before treating an A/B test as a near-term source of an answer.
Judgment plus feedback
When a standard experiment is infeasible, increase conviction through other evidence and arrange a way to learn after shipping.
Actionable Next Steps
- ✓State whether your next product review is for information, accountability or improving the idea.
- ✓Keep the discussion group small and share the review artifact more widely.
- ✓Check effect size and runtime before promising a test result.
- ✓Name the feedback signals you will watch after a judgment-based launch.
Notable Quotes
“In terms of things I’ve learned, I think being really intentional about what the goals are, I think it’s okay to say that there are two goals, a goal of accountability and inform to an audience.”
“And then the last bit I would say is if you’re not going to get significance, if there’s no other techniques at your disposal, then sometimes you just got to trust your intuition and ship it.”
“And we very quickly realized that back to what the kernel of truth is here is liquidity is the only thing that matters.”
The One Thing to Remember
A review should improve the product, and a decision should be honest about what the available evidence can tell you.