Brent Tworetzky

Removing obstacles to building products that matter

What drives a product manager’s soul? More than anything, strategic product managers yearn to solve important problems for their users and businesses - to make an impact with their products. Build great products to change the world! Seems simple, but it’s really really hard.

As I think about setting 2017 goals, I keep coming back to basics – how can we all do better next year and have deeper impact? What gets in our way?

This year the XO Group product team met several core goals, beat some exciting stretch initiatives, and failed at a few basics. In fact, when polling New York City’s product execs at November’s New York Product Conference , most had 2016 and 2017 goals on their minds. Let’s look at what gets in the way of product impact so we can do more for our users in 2017!

Bad goals get in the way of product impact This is not an article about how to set great goals. Goal setting is an article by itself, and today many teams and institutions use a variant of the SMART framework (specific, measurable, achievable, relevant, time-bound), shared almost fifty years ago in Management Review , to come up with good goals for the period ahead. XO Group product teams balance overall topline company goals (e.g. user and member growth, revenue growth, user NPS) with squad-specific goals (e.g. page load time improvement, increased number of guests RSVPing per wedding, increased number of wedding venue tours, etc.)

Many XO Group product teams created SMART goals that set them up for success, but a few tripped up on their initial goals and direction that slowed those teams’ impact this year. Here are some goal challenges we’ve battled at XO that also frequently appear at other great NYC tech companies:

Wrong goals get in the way of product impact At XO Group and across CPO discussions, we don’t always think hard enough or get the right buy in that our product management goals are the most valuable. We sometimes measure user funnel behavior without getting to a user outcome. We sometimes set a goal without understanding if we can operationalize the results. (Can customer support handle new user behavior?) It’s important to think through what happens if we achieve these goals. Is the goal is midway point, a signal of success, or success itself? One XO product team set a BHAG (big hairy audacious goal) to deepen our relationships with our user through new tools that don’t exist anywhere. The BHAG team was so focused on hitting its goal, which they did, that they missed seeing how BHAG risks sending users to a digital no man’s land due to user flow problems. Towards the end of the year, as we approached the BHAG goal, we realized our information architecture challenge and changed user pathing … though the BHAG product itself needed to change as well.

At the New York Product Conference , Oscar Salazar , founder of Pager, Uber, and other great startups, stressed the point of investing enough time and though on setting the right goals and metrics. He used in-app chat as an example - would success be longer chats or shorter ones, more or fewer? Some teams assume that more time spent in an app shows success, but if the app’s goal is for a user to achieve a task (get appropriate medical help, in Pager’s case), more time could be a signal of a longer-than-necessary process. Pager took several months to get to the right goal and metric; getting this metric right greatly improved user success and happiness. The Knot mobile app launched a chat experience this year, and the team used chat length as an early engagement metric. In our case this metric made sense, but we vigorously debated the topic.

Too many goals get in the way of product impact With too many goals, teams can prioritize incorrectly or suffer decision fatigue, ultimately not moving any of the many metrics in a meaningful way. Unfortunately one of my teams spun in circles this year attacking too many problems.

Teams commonly unintentionally combine goals and key performance indicators (KPIs) with other monitored metrics, giving the appearance (and operational reality) of too many goals. Be careful not to confuse the two! Are KPIs and other metrics communicated equally (e.g. in status emails), investigated with equal urgency, given equal air time in team meetings, or managed to equally? Teams should elevate and focus on KPIs to avoid too many goals.

Side note : KPIs and metrics are two different things. We can measure many, many things (thank you MixPanel and Google Analytics!) though we only truly have a few critical goals. Best practice is to focus on KPIs but also consider other metrics for either system health monitoring (page load time, specific funnel step performance) or for investigating user behavior.

With too many goals, stakeholders start to gloss over what’s really important and may struggle to understand and best support a team. When a team’s weekly email shows a dozen metrics, my CEO has emailed me more than once asking what’s actually going on.

Unclear goals get in the way of product impact For some of our XO innovation projects (such as our wedding concierge chatbot), goals were too vague for longer than necessary. Exploration projects do need free space—up to a point. After a while without a clear goal, innovation projects can turn into science projects with little accountability.

— — — — — — — — — — — — — — — —

Even with good goals, great product managers and teams often predictably struggle achieving the outcomes and impact they set out for. Some common challenges that block product teams from achieving their goals include misaligned team culture, too much process focus, unchecked cognitive biases, and challenged visibility.

Misaligned team culture can get in the way of product impact Strategic product management gives teams goals and the autonomy to achieve those goals. The setup works well for well-aligned teams committed to outcomes and results, but not all strategic product teams fit that description. Is the team committed to results, and does the whole team agree on an outcome philosophy?

Is the team committed to results?

At XO all product managers and designers are trained on best practice tools and processes, yet we still take too long sometimes to validate that a product really solves a problem. We sometimes spend design and engineering efforts on products that won’t move the needle enough. We sometimes iterate around a weak solution “to do the full process” rather than forcing a signal quickly and bolding changing direction to better reach our goal. Have you heard yourself or someone else ask for “one more AB test” to see if a mediocre product can suddenly deliver a tremendous outcome? Does the team move fast enough? One easy test: does your team celebrate output or outcomes? A big release or a big milestone? Teams should sign up for this stress together – you’ll fail and harm relationships if not everyone agrees. Does your team want to do what it takes to win? If uncomfortable with becoming 100% outcome oriented immediately, can the team together move in this direction one sprint or month at a time? Infuse the rigor of estimating how a product will perform and measure how well your products actually work post launch. Does your team always measure a feature or product’s individual impact? Sometimes you’ll need to spend extra effort going back to measure, or implementing a holdback test, to gain that learning, and you’ll need to get your team to buy into spending that energy. Bonus points – learn as much as you can from your failures to increase team learning to make better decisions in the future! Of course, have rules about how flexible your team can move to achieve goals. Even when moving quickly, use best practice and good ethics. Does the whole team agree on an outcome philosophy?

As a product leader for a team with such autonomy, you still need to get the team to commit to outcomes over outputs, a results oriented culture, and comfort with quick failure and change. For example, does the team agree on when to iterate on a product versus reinventing the product versus throwing the product away? If not, it can be uncomfortable for a product manager to convince a team to change direction before it is ready. Especially if the team members are conflict averse. Too much focus on process gets in the way of product impact Best practice product management follows a well-tested process, and we spend a lot of energy training our teams on this process so they follow it well. (Learn agile! Prototype! AB test!) That said, great product managers also know how to get early signals during product discovery and adjust quickly to get to stronger results. Less tenured or confident product managers cling to the process, and complete it fully, before getting a signal that a product achieves its goal. Some parts of product exploration and development can’t be rushed, though with experience and an outcome mindset product managers learn how to accelerate and change team velocity mid-product development.

I sometimes see or hear about product managers leaning on process as a crutch for inserting more urgency and rigor. I’ve been guilty of letting process comfort slow my teams down several times before. Process should support the goal, not the other way around!

Jon Stross , co-founder of Greenhouse and the original product manager of BabyCenter two decades ago, mentioned at the New York Product Conference that tools have gotten so much better over the last twenty years, but it’s still just as easy to get lost in the trees and not see the forest. He encourages product managers to add rigor of checking in regularly about if a team is solving problems measurably, with urgency.

By appropriately focusing on the product management goal more than the process, and accepting the discomfort of having to learn quickly and decide quickly, most teams can move faster in a smart way. This rigor is hard, so team members should hold each other accountable, and PMs can lean on their managers for help.

Cognitive biases get in the way of product impact Cognitive biases challenge many parts of product development. Some key biases I see product managers falling into include

Anchoring bias – getting stuck and over-optimizing around an initial signal Choice support bias – exaggeratingly remembering how right we’ve been in the past to give confidence in our current hypotheses Confirmation bias – falsely perceiving signals to support our hypotheses Hindsight bias – thinking a finding was obvious, even though it wasn’t. Related to choice-supportive bias of remembering feeling more confident than we were actually were when we made a hypothesis Loss aversion – feeling irrationally uncomfortable throwing something away Ownership effect – overvaluing an idea or product because it’s our own Understanding and overcoming bias is it’s own field of psychology – I haven’t seen a great thorough article about bias in product management yet. Just being aware that biases exist, and seeking to double check our thinking (or have someone else check us) can help a lot

Lack of visibility gets in the way of product impact Ideally teams have clear visibility into KPI progress and can use metrics as a north star. Unfortunately, sometimes tracking breaks down.

When we don’t know about broken analytics, teams could go in any direction. When metrics are *really* broken, teams can more easily sniff out illogical results. When metrics break but retain some logic, teams can follow a false north star for weeks or months. Don’t let this happen to you!

When we know our KPI tracking is broken, teams have to make the hard choice of top goal imperfectly measured vs. secondary goal perfectly measured until regaining clean tracking, and there is no easy answer here. Unfortunately, teams sometimes turn to easy-to-track relatively unimportant metrics, especially when the team has too many goals. When tracking broke for an XO team team, they spent three months using proxy metrics and lowered page load time and tech debt, and unfortunately neither moved a primary or secondary KPI even though they seemed like good ideas.

— — — — — — — — — — — — — — — —

As I look at this year’s performance across XO Group’s twenty product squads, those that performed best had a clear measurable goal, shared outcome urgency as a team, and felt empowered to maneuver to hit their goals. I’m grateful to my old boss Vish Makhijani , CEO of Udacity and former COO of Zynga, for drilling that rigor into me. The XO teams that performed worse either had weak goals, as mentioned above, were too wedded (pun!) to the process to move quickly enough, or failed to explicitly get all team members onboard an outcome-oriented culture.

In 2017, we hope to set stronger product management goals and move with urgency to achieve them. Our users are waiting!

---

This article is cross-posted to my Medium blog .

Brent Tworetzky

New York, New York, United States

5K followers 500+ connections

About

Experience

Peloton Interactive

Recommendations received

Arvind Kadaba

Patty Izquierdo

View Brent’s full profile

Other similar profiles

Umaimah Mendhro

Charlie Newark-French

Mark Collier

Sean K Mehra

Christopher Martin

Kevin Mayer

Hooman Radfar

Amit Shafrir

Charles Seely

Greg Isenberg

Jay Jamison

Prakash Ramamurthy

Ben Sardella

Matt D.

Walter Noot

Josh Kerr

🎯 Darin K. Newbold

Pascal Finette

Dan Kim

Christian Duffus

Explore collaborative articles

Add new skills with these courses

Become a Marketing Entrepreneur

Building a Winning Enterprise Marketing Strategy

Customer Experience (CX) Strategy: How to Design an Organizational Approach

Brent Tworetzky

New York, New York, United States 5K followers 500+ connections

About

Experience

Peloton Interactive

Recommendations received

Arvind Kadaba

Patty Izquierdo

View Brent’s full profile

Other similar profiles

Umaimah Mendhro

Charlie Newark-French

Mark Collier

Sean K Mehra

Christopher Martin

Kevin Mayer

Hooman Radfar

Amit Shafrir

Charles Seely

Greg Isenberg

Jay Jamison

Prakash Ramamurthy

Ben Sardella

Matt D.

Walter Noot

Josh Kerr

🎯 Darin K. Newbold

Pascal Finette

Dan Kim

Christian Duffus

Explore more posts

Explore collaborative articles

Add new skills with these courses

Become a Marketing Entrepreneur

Building a Winning Enterprise Marketing Strategy

Customer Experience (CX) Strategy: How to Design an Organizational Approach

New York, New York, United States

5K followers 500+ connections