Creating alignment between product management, user experience, and engineering in a product team can be challenging. When your product depends on innovative technology, including research scientists in this mix may create additional complications.
During my five years in IBM’s chatbot platform Watson Assistant, I have led various product missions with multiple AI research teams. In this article, I will explain a few different modes of collaboration we have experienced between the product and research scientist teams. Some collaborations were extremely productive after we successfully created a united team. In some other cases, we suffered from cross-discipline conflicts that produced suboptimal results.
Background: How do our product teams work?
Our product team organizes deliveries around missions, outcomes, and squads:
- A mission is a delivery that focuses on a few measurable outcomes.
- A squad is a cross-functional team that owns a mission, and is led by a product manager, user experience, and an engineering manager.
We staff every squad with all the skills it needs for a particular mission. Ideally, each squad owns a single mission, and each team member is assigned to a single squad.
For example, we may have a mission to "increase engagement for contextual entity feature by 20%". This squad for this mission may require designers for a new user experience, UI development skills to implement these designs, changes to our machine learning infrastructure (for example, to reduce training times), and an improved machine learning model (for example, to increase algorithmic performance without impacting training time).
Collaboration challenges when we need AI innovation
Creating a squad around a mission with product team members is a routine operation. Of course, we do struggle with the typical Product Development challenges: There are always priorities, roadmaps, operational and staffing issues. As we execute, we occasionally struggle with cross-discipline disagreements between engineering, product management, and design.
Additional challenges emerge when product teams collaborate with research scientists to solve hard AI problems, especially for an existing and successful product with a lot of customers.
Researchers are usually not part of the product organization, but members of the Research organization. The collaboration between the research and product organizations is always planned and formalized with a document that describes the goals, plans, and commitments.
Asking researchers to "just join the team" does not always work. People who belong to different organizations usually have different cultures, daily challenges, business commitments, and career growth goals.
Product teams develop rigorous engineering practices and team ceremonies to manage regular cross-disciplinary conflicts and avoid upset customers. There are always big milestones, important customers, and operational challenges to worry about. Product managers and designers worry about the A/B tests, NPS, conversion, and engagement metrics. Engineering teams sift through every pull request with the fear of a defect or production incident slipping to production.
Research teams, on the other hand, attempt to solve seemingly impossible problems. They create various prototypes, often work on multiple open-ended projects that require experimentation. Customer pressures, production incidents and business goals are not routine challenges for them. However, they must catch up with technology trends, publish and secure sponsorship for future research projects.
Three modes of collaboration
Although the ideal collaboration is creating one team from all disciplines motivated to deliver an outcome, the differences outlined above make this difficult. Team members’ or management’s vision for an ideal collaboration can sometimes be overwritten by business constraints and personalities.
Mode 1: Controlled collaboration through dependencies
The fastest way to kick off a collaboration is keeping the product and research teams separate and managing collaboration through dependencies. However, the teams often pay for this initial speed later.
In this mode, the research team works mostly independently from the product team, after an initial kick-off workshop. Product management describes the business goals, and technical leads agree on a high-level technical path and schedule. Then, two teams work separately, except for regular checkpoint meetings between technical leads and managers. Code artifacts may be kept separate, and the product team periodically adopts these research artifacts.
Teams inexperienced with cross-discipline collaborations tend to default to this approach, due to its "fast start" nature. Isolated teams seemingly make quick progress during the early stages. However, this speed is often an illusion and teams pay the price many times over as misalignments and misunderstandings emerge.
This collaboration mode also isolates researchers from the customer needs. Often, the desired user experience, operational constraints, and business requirements impose requirements on the AI algorithm. A disconnected research team cannot get real-time updates on the A/B tests, user research findings, and updates from product management. The new AI algorithm ends up not meeting what the customers need, even if it beats all the research benchmarks.
This type of collaboration may produce good results, if the components developed by the two teams already exist, or if they have well-defined boundaries and API contracts. This mode also tends to emerge to cope with dysfunctions between the teams and organizations, if they exist. The isolation allows both sides to make progress, but also defer dealing with conflicts to a later time.
Mode 2: Partial collaboration in a single squad with operational limitations
A tighter collaboration is possible by creating a single squad, but with rules and restrictions on how the research and product teams will collaborate on the mission. This mode can work well for customer-facing missions when the research team needs or prefers some degree of independence.
Operations are a frequent element of negotiation for such collaborations. The research team may not have the skills or interest in deployments, maintenance, and pager duty. Similarly, source-control boundaries that mimic team boundaries tend to form, with rules that dictate who can deliver, review or merge code for different components within the same application boundaries.
Who joins the squad ceremonies can also be intentionally limited. There may be only one or two research leads who always join the scrums and playbacks, keeping the rest removed from these activities. It is also common to have some (or all) of the research team to have less than 100% allocation on such product missions. The reasons for these rules are often other concurrent product or research collaborations, conferences, and time needed for publications.
The prerequisite for this mode to work well is to have at least one team member from engineering and one from research to be "all over the place" without any limitations, and assigned 100% to the mission with no other side gigs. Ideally, these engineering and research leads act as technical leads for all engineers and researchers combined. They lead end-to-end product features and even participate in operations. Having two such exemplary leads who both have engineering and academic/AI chops significantly helps to bridge the product and research team.
Finding talent with such breadth of skills and motivation can be hard. In addition to systemic hiring, an organization needs to have a disciplined approach to develop such talent in-house. An organizational culture that hires the right kind of people incentivizes learning and "moving around" is crucial to develop and keep such talent.
Mode 3: Full collaboration in a single squad
This mode is about having one squad where everybody is 100% assigned to the mission, and nothing but the mission, with no special rules or restrictions on either team.
This mode works great for customer-facing missions. Every team member focuses on the same outcome, joins the same meetings, sees the same experiment results, and worries about the same user personas.
This mode also works well when the innovation needed is bounded by a well-defined domain. Open-ended research challenges and "finding the algorithm that will drive the product direction" do not work well when research and product teams are focused on a customer-driven outcome.
When it works, this collaboration mode is also the most gratifying for the individual team members. Research teams love seeing their research results helping actual customers. Some of them appreciate the new skills they gain as they work with engineers to deploy and scale their solutions. Product team members greatly enjoy being part of innovative research and gain more appreciation of how research teams work.
However, _"_one squad, one mission" does not always happen, even in standard product teams. It is not uncommon to ask an engineer or a designer to "give a hand to another mission for a few sprints" to hit a deadline. Researchers have similar conflicts. The time needed to do "research stuff", write papers, attend conferences, or participate in other collaborations.
In addition, if there are personal or incentive conflicts between the product and research leadership, early phases of the "one squad, one mission" can be chaotic.
What should you do?
As ideal and fun as it is, working as a single outcome-oriented cross-functional squad (or its old school name, "feature teams" ) is not always easy. There can be many organizational and personal obstacles to successfully collaborate as a single team in product teams.
If your product teams are not experienced with working in cross-functional teams, starting with "single squad, single mission" may be even harder. Although cross-functional teams are not a new concept for product teams, creating boundaries around components and disciplines is still common in many product organizations. Having some team members who are experienced working in cross-functional squads will help to succeed with this collaboration mode as you add researchers to the mix.
Finally, just giving the teams the time to figure out how they should collaborate is sometimes the right strategy. In the early days of my experience, we were able to predetermine and execute how we wanted to collaborate with AI research teams about 60% of the time. For the remaining times, our collaboration methodology emerged over time with experimentation (and after some conflicts and failures) and was shaped mostly by constraints and personalities.
However, our success rate with research teams to execute "single squad, single mission" approach has increased significantly, as we have gained more experience, and as more people in product and research teams have started to see the light after several success stories.
