Building models that impact millions of people is the greatest perk I am fortunate to have as part of the Data Science group at Wix. There is no doubt that a good model can scale, automate and optimize products and boost business KPIs, improve users’ satisfaction, and reduce manual labor.
Photo by Alina Grubnyak on Unsplash
In this post, I will share with you the fundamentals that helped me to initiate successful projects across Wix. The steps I will share are also applicable to non-related AI projects and will help you to reduce risk and increase trust, while collaborating with a range of business units and roles.
The 5 steps for initiating a successful project
Why do data science projects fail?
As many experienced data scientists already know, in the real world where business impact is the key and having the most accurate model isn’t enough and there are many reasons why projects fail, here are the major ones:
Lack of a stakeholder - not having a business/product owner (i.e., a stakeholder) for a project is leading data scientists to build all sorts of interesting but not important or relevant problems.
Over-committing - not estimating technical limitations correctly with respect to model deployment, data quality, availability and processing due to lack of infrastructure and common best practices.
Over-complicating - building a complex State-Of-The-Art (SOTA) performing model takes time and requires research that can delay integration and loss of product relevancy. In most cases, a simpler model is sufficient and can generate a similar business impact.
Bad narrative - The tension between people causes overall bad experiences around data science projects, such as being too long, costly, risky, and uncertain, which can close the door to new projects.
What is a successful data science project?
When you are leading a project with an external business unit, you should ensure that the organization will be impacted by data science in the present and future time.
Present business impact - the project is proven to show a significant improvement in a meaningful metric the organization seeks to optimize defined by a business owner.
Future opportunities - building trust, a foundation, and know-how, to keep impacting the business with more projects.
Communication is All You Need
The lack of communication between technical and business people is a well known issue, but in data science projects it can scale because they are more abstract, dynamic and require more education.
The business side might consider data science projects as risky and costly, but today the availability of stable infrastructure, methodologies, and past projects’ experience reduces the risk and the cost dramatically.
The rise of GenerativeAI models reduces the need for training models for NLP tasks like summarization and text classification, and can be built out-of-the-box with correct prompting.
On the other hand, data scientists don’t always understand the customers’ constraints, roadmap, KPIs, and prioritization. This might lead to incorrect project recommendations that might not positively impact the business.
So, after understanding what a successful project is and that communication is our key concept in achieving it, let's next take a deep-dive into the steps that will help us achieve it!
Step 1 - Setting the Right Mindset
Before jumping around meetings with external business units, it is super important to set up the right mindset that will help you gain a good impression, trust and alignment.
“Can do” approach - This might be the first time the people you meet are talking with a data scientist. You should act as an ambassador and know that this impression will determine current and future projects. You should be optimistic. Remember that a very simple model can create a huge impact.
Have a marathon mindset - Because you are initiating a new project, things might take time. That's OK! Don’t beat yourself up or take it personally - be patient. You’re playing the long-term game.
You have one shot - despite the marathon mindset, you should remember that failed projects due to bad practices, communications, and executions can close the door for future projects. You don’t want to lose this opportunity and take it lightly.
Practical steps:
Internal alignment - Talk to your managers and other data scientists that have been leading projects externally to learn what were the key factors for success.
Practice a peaceful mindset in meetings - If you are (like me) always looking for the next thing to do, and get impatient when things take too long, try to take some steps back on different occasions such as meetings to practice patience.
Open for business - Start talking to relevant business units you want to collaborate with and collect intelligence, let them know you are open for business.
Step 2 - Tech Stack Awareness - Limitations & Opportunities
Before starting any AI initiative, you should have your infrastructure and methodologies straight. You have to make sure you have a full working cycle of deploying models which are scalable and accessible to data and prediction. In case you don’t have one, I highly encourage you to at least have some cycle you feel comfortable and aware of its limitations.
At Wix’s Data Science group, our amazing engineering team developed an ML Platform that allows data scientists to deploy models to production with CI/CD, eliminating the need for any DevOps and other technical hassles.
Practical steps:
Learn tech limitation - Learn the limitation of your infra to not overpromise.
Fill the technical gaps - If you are missing critical infrastructure, you should make sure your group prioritizes and develops even a very basic one.
Set internal expectations within your team and engineers as to what kind of projects you can commit to technically
Step 3 - Find a Hero - a Motivated Strategic Fighter
Everyone wants to be part of the AI revolution, but when seeking your hero, it has to be someone who knows the business unit well with respect to goals and priorities. On top of that, the hero should have some technical capabilities and who is aware of the data sources and the development cycle for feature releases.
You should invest in educating your heroes, giving them as much knowledge and resources about AI to allow them to understand the potential and the risk. On the other hand, learn from them about their business, main KPIs, and short and long-term goals.
The hero is going to be your partner, you should be super open and honest with each other.
Practical steps:
Find a hero (or they will find you)
AI education - Prepare simple DS materials and relevant projects to go over
Learn the business from your hero
Step 4 - Run Fast & Play Safe - Prefer “Low-Hanging Fruits” Over “Moonshots”
When meeting with your hero and other people in the business unit, you will probably face some innovative ideas which are not so practical. These “moonshot” projects hold high potential value, but are risky.
Data scientists (mainly juniors) sometimes want to do cool and complex AI models / research, while your stakeholder is only interested in the impact and resources. This tension can be solved if you will first deliver a simple (though valuable enough) version and later invest the time in more complex research.
Because it's your first project, you should keep it safe, and look for projects with high value and low risk, what we call “low-hanging fruit” projects. Notice that business-wise “moonshots” and “low-hanging fruit” hold similar potential value.
Remember Occam’s Razor principle, that we will always prefer a simple and secure model (“low-hanging”) than a complex one (“moonshot”). Keep in mind that building simpler models will allow your hero to accumulate success, build trust, reduce uncertainty, increase confidence about the project and advocate it better.
Don't over-promise, be transparent with the risk and ask for help from your team.
Practical steps:
Map potential projects, both low-hanging and strategic, and make sure they are relevant to the business's main KPI.
Play with the data - Get access to the data and play with it. You most likely will uncover some pitfalls or opportunities.
Prove potential - Come up with evidence of the potential of the “low-hanging fruit” projects. It can be by showing engagement/traffic, articles about similar projects in the industry, or a domain expert opinion.
Choose a low-hanging project and build it very quickly and dirty, the data and model should be very straightforward and shouldn't take too long to develop. Keep in mind that no one has committed yet to integrate and use this model.
Step 5 - Bidirectional Prioritization - “Bottom-Up” and “Top-Down”
Soon after you find your business partner for the project and have mapped relevant projects, you will very quickly be aware of all the reasons why there weren't any AI projects until now.
One of the main reasons is the lack of resources and prioritization. Therefore, you can’t get any commitment that someone will eventually use the model.
AI models that are being “Written for the drawer” are common and unfortunate phenomena, where a model you were working on for a long time is not being consumed and therefore you made no impact.
To make your models ״see the light״ and mitigate the risk of wasting precious time, you should go in two directions at the same time:
The "top-down" approach - talk with managers and think about projects that are aligned with the roadmap and company focus and needs, making sure you are not wasting time.
The "bottom-up" approach - start playing with the data and show feasibility with a simple baseline to reduce risk and increase trust (step 4 - “low hanging fruit” project).
Combining both approaches complement each other. In order to prioritize the project it should be relevant to their strategic business goals at the right time frame and, on the other hand, you want to make sure you are working on the relevant projects that will “see the light”.
Once the stakeholder commits to the project, The final step is to write a “project scoping” document that outlines all the requirements, SLA, KPI, and timelines. It should serve as a contract between you and the business unit, providing clarity for both sides. Since there can be a lot of uncertainty around data science projects, the scoping doc allows everyone to be on the same page and improves communication.
Practical steps:
Meet with a key business decision maker, showcase the projects and consult with them. Show them your "low-hanging fruit" demo to convince them that you are ready to deliver results!
Get priority - Get management agreement to integrate with the model and evaluate its performance. Based on this commitment, you should decide about model deployment or searching for another “low-hanging fruit” based on the feedback you received so far.
Scope the project - collaborate with the stakeholder to write a scoping document that wraps everything up and reduces any uncertainty around the project.
Conclusions & Final Words
In this post, I shared 5 steps for initiating a successful data science project that creates business impact and opens the door for new projects. It all starts and ends with good communication which needs to be mastered as you go.
The 5 steps can be divided into two phases: preparation and Ideation. The preparation phase handles all steps required to (1) set the right mindset, be aligned with your data science values, and look at a project as a long-term relationship. (2) To understand the tech limitations and opportunities so you know when not to over promise, and (3) meet with people and find a hero who will partner with you to advocate for AI projects together.
The ideation phase takes care of (4) mapping and defining potential projects and eventually choosing a “low-hanging” worth working on. Then you should (5) get priorities and commitment to integrate the project by talking with managers and showing them concrete progress and impact already made with the "low-hanging" project. Finally, collaborate with the stakeholder to write the scoping document that ensures everyone is onboard with the project.
That's it, you are now equipped with the right mindset and tools to start your journey of impacting your company with AI. Make sure to always be learning, continue advocating for your work, stay humble, and enjoy the ride!
This post was written by Lior Sidi
More of Wix Engineering's updates and insights:
Join our Telegram channel
Visit us on GitHub
Subscribe to our YouTube channel