During the last few years, we have seen that nearly all technology based products and services must learn to leverage AI in order to compete effectively in the marketplace. We recommend adopting Lean/Agile principles and practices for this purpose. In this manner, organizations can continuously improve time to market and realize exceptional value early and often. In this post we’ll describe the top challenges that we see in the field as well as tips and tricks for applying Agile to get the best of AI. These are all patterns that we’ve applied successfully at AgileSparks in our work with a diverse set of clients. What are the top challenges that we’ve seen in the field regarding AI in the organization?

• Working as a silo separated from the rest of the teams, often feeling that their role in the product development is not clear (“you don’t understand the nature of our work”, “we have to research a long time before you can start”, etc.).
• Lack of alignment between the AI work and the rest of the organization due to separate goals & backlogs.
• Lower engagement of the AI people with the rest of the people in the organization.
• Infrequent feedback and learning due to working with big requirements / long research.
• Lack of transparency regarding the AI work – not clear what is being worked on and how it is progressing.
• Not sufficiently leveraging the AI group abilities due to low and late involvement of the AI group in the backlog refinement.

Why is it important to address the above patterns? The above challenges contribute to inefficiencies in the flow of value in the organization due to delays and waste in the work. In this post, we’re going to discuss how to incorporate AI people and work within the product development life cycle in order to overcome these deficiencies. We will differentiate between Data Scientist (aka Algorithm Developers) and Data Analyst roles. Note that in some organizations these roles are done by the same people. How should Data Scientists collaborate within the life cycle of product development? We’ll start first with the Data Scientists. Data Scientists write algorithms and build statistical models. They arrange sets of data using multiple tools in parallel and build automation systems and frameworks. We’ve found that the following approaches help Data Scientists to better collaborate with the rest of the organization throughout the business & product development lifecycle:

• The lead Data Scientist in the organization participates in defining the vision & roadmap.
• Data Scientists are members of the Program level (multiple teams working in collaboration) and participate in the Program events.
• Data Scientists are part of the backlog refinement process.
• The Data Scientists’ research & business features should be sliced smartly (vertically instead of horizontally) to achieve small valuable batches that will be continuously integrated and feedbacked. The small batches might be actual working models or validated learning that indicates whether we’re hearing in the right direction or not.
• Data Scientists collaborate with each other in a dedicated Agile Team leveraging Lean/Agile mindset and practices (similarly to other Agile Teams) and sharing the same synchronization and cadence as other Agile teams.
• Member of the AI Community of Practice (CoP)

How should Data Analysts collaborate within the life cycle of product development? Data Analysts design and maintain data systems and databases, using statistical tools to interpret data sets, and prepare reports to present trends, patterns, and predictions based on relevant findings. At AgileSparks we’ve found that the following approaches help Data Analysts to better collaborate with the rest of the organization throughout the business & product development lifecycle:

• Data Analysts are part of the backlog refining process to ensure that data considerations are discussed and applied for all backlog items.
• Most of the Data Analysts work is part of the functional features included in the user stories that are implemented by the team.
• Data Analysts are members of the development Agile teams, sharing the same goals and backlog. They participate as full Agile team members.
• Member of the AI Community of Practice (CoP).

Summary From our experience, by implementing the above approaches, organizations will gain the following benefits:

• The AI group will be aligned with the business purpose.
• The AI group will become more engaged with the purpose and work of the rest of the organization.
• The organization and the AI group will gain transparency regarding the AI work and progress.
• The AI group will be more effective and efficient bringing real value faster by working with small valuable batches and continuously learning & improving.