High uncertainty, lack of in-house capability and the quest for a highly accurate model. These are just three of the main challenges in implementing a machine learning project.
I remember my boss asking me, “So… how are we supposed to derive insights from this unstructured data set?”. The rules of a typical business intelligence solution no longer apply. The high uncertainty of machine learning projects is harder to manage due to lack of in-house capabilities.
Honestly, no one in our organization understands how to implement artificial intelligence technology beyond just theories. No one has put together a solid solution suitable for production, not even a pilot project that business users can try out. To make matters worse, there’s always the nagging question: how can we be sure that the machine learning model is “accurate enough”? How do we know when we’re truly ready for production?
Agile project management has a framework to handle highly uncertain projects such as machine learning. Scrum is a framework that’s fit for projects with many unknowns. As Gunther Verheyen said:
Agility is the state of high responsiveness, speed, and adaptiveness while controlling risks. It serves to better deal with the unpredictability so typical of the work of software development and to the markets that organizations operate within.
Agile development with Scrum is the light at the end of the tunnel. The ether to which our machine learning tech will operate. So, how are we doing it? Here’s what we have figured out so far:
1. Start with small to zero investment
License costs for an AI solution are no joke. For instance, an enterprise natural language processing solution hosted in the cloud can cost as much as $200K. It makes more sense then to try a proof of value (PoV) with small to zero investment to test the waters.
2. Shift to agile driven contracting
When you’re done with the PoV and finally demonstrated that your chosen AI will cater to your business needs, its time to talk to procurement. Traditional time and material nor fixed priced contracts will not work.
Agile driven contracting is an agile project management process that emphasizes on the delivery of value, not just the solution. The idea is, you pay only when the value is delivered. Wikipedia perfectly sums it up:
In Agile contracts, the supplier and the customer together define their common assumptions in terms of the business value, implementation risks, expenses (effort) and costs. On the basis of these assumptions, an indicative fixed price scope is agreed upon which is not yet contractually binding.
3. Design thinking for requirements gathering
The Stanford Design School (d.school), now known as the Hasso Plattner Institute of Design started the Design Thinking process with just three steps: Understand, Improve and Apply. This framework emanates the values of agile project management as it emphasizes the importance of empathy in doing good project design. This means that rather than zooming into the functionality of a machine learning solution right away, it is best to start with knowing the current customer pain-point or challenge. This will drive which features to build and what the user experience should be like.
4. Scrum for project implementation
At the heart of our implementation is Scrum. Contrasting it to the traditional and predictive project management models, Scrum is effective in handling big unknowns, risks and changes typical of machine learning projects. Its streamlined process only includes four major events: Sprint Planning, Development, Sprint Review and Sprint Retrospective. It is perfect for cutting the bureaucracy and skipping right to building something which our business users can use asap. Our sprint runs in two-week increments with a build ready to use at the end of the 10th day.
5. Build the machine learning model with business
Agile and machine learning is a match made in heaven. By using scrum, we emphasize the importance the criticality of business participation to project success. Instead of being heavily reliant on the technical team, we asked subject matter experts from operations to build the machine learning model with the geeks. Together, the vision that we defined in the design thinking sessions is translated into machine and data science problems that the model should be able to reflect.
On my next article, I will talk about the other five things that we did to use agile to manage our first machine learning project. This zooms in right to development and how to effectively do a validated learning approach to make sure the increment is still on the right track.
Remember, doing a machine learning project will never work using predictive and traditional approach. Also, if you decide to use agile for implementation, but your contracting model is still fixed priced or straight time and material, chances are this will be a primary cause of the delay. This is most especially true if you have to change requirements midway, which is inevitable for machine learning projects.