Many AI experts suggest the current rapid rise of AI, and the ubiquitous use of machine learning, is at least partially borne from the abundance of data that has become available. Plenty of machine learning theories and practices have been around for decades, but to use these techniques effectively, large amounts of data were needed to “train” a machine. For much of that time, coming up with that amount of data was difficult, slowing the progress of AI research. Today, that need has become less burdensome.
So what does it take to train a machine?
If you want a machine to be able to recognize images (i.e., patterns in shape, color, contrast, etc.), train it using a wide variety of pictures — that’s data.
If you want a machine to understand something that is spoken (i.e., translating sounds and words to text), train it using a large stable of audio recordings — that’s data too.
And if you want a machine to understand what is written, train it on many, many different phrases and sentences — I think you see my point.
As IPsoft’s Senior Cognitive Trainer, I spend most days teaching humans how to use Amelia, our cognitive Artificial Intelligence (AI) platform. Learners seem to be most frequently surprised by how much data variety is required for AI to work effectively. But if you think about Amelia as a new employee who needs to be trained in a new skill, this need for data should become clearer.
Training Amelia Like a Human — With a Twist
Imagine Amelia has been hired to work in a specific role, so we want to teach her specific skills. Amelia needs training just like a human who is taking on a new job. Unlike a human, Amelia doesn’t need to sit in a room with 20 other new employees to go over the logistics of the role, and then sit in another room for days or weeks and learn the in-and-outs of her new responsibilities.
However, Amelia does need to learn where and in what context a customer might need assistance, as well as how a customer might respond to various questions to complete a process and achieve their goal. And for that, again, you need data.
Data is the mother of AI. Without data, AI and machine learning couldn’t exist.
The machine learning algorithms used to train Amelia are built into her system. It’s relatively easy to learn the steps required to create a model that recognizes a customer’s needs and responses to questions. The challenge is ensuring that the data teaches Amelia to understand how a specific audience will articulate their needs without programming her directly.
When Amelia is trained using a set of training data, she is not programmed. She is an AI platform after all. Instead, she learns from the different patterns of variation that exist in the training data and uses this knowledge to understand new inputs from the customer. Amelia can then assist the customer with specific goals and extract relevant information from responses to help them achieve their intended tasks.
As an example, imagine Amelia is acting as the reservation agent for a luxury hotel. A new customer starts a conversation with her to make a hotel reservation:
“Hey, I need to get a room from tomorrow through Friday and I just need one bed cause I’m travelling alone.”
From this one customer statement, and using several machine learning models, Amelia can understand and obtain all of the following information:
The customer wants to make a hotel reservation with:
- Check-in date: Tomorrow
- Check-out date: Friday
- Room type: Single (as the customer only need one bed)
- Number of guests: one (because the guest is travelling alone)
This is the power of machine learning at work, but it’s not something that just happens. It takes data — not reams and reams of data (i.e., just keep adding more and more and maybe the machine will figure it all out). It’s about the quality and variety of data.
Data Variety is Key to Training AI
Having variety in the training data is similar to how we learn as humans. Imagine you are taking a math class. Would you study only the same example problem over and over again: 2 + 2 = 4? All you would learn is that one simple equation. It would be more effective to study many variations on addition (i.e., 3 + 6 = 9, 100 + 44 = 144, etc.). Over time, you would include other mathematical operations like subtraction, multiplication, division, etc. In a sense, this is analogous to how you train a machine.
My recommendation to new AI learners is to start training Amelia on one skill (e.g., make a hotel reservation). Obtain or create examples of how someone might ask for help with this task, and be sure there is as much variety in sentence structure and word choice as possible given the audience that Amelia will serve. Use this data to teach Amelia, almost like you would use different example equations to learn math.
When you are satisfied with Amelia’s performance, either through testing or experience, add a second skill by training her using varied examples for that skill, analogous to including subtraction in your math studies.
As Amelia progresses in her learning, you may need to go back and revise some of the data to account for new information that has been included in her training. Think of this as if you were being taught absolute value for the first time.
You originally learned that:
- 3 + (-3) = 0, but now
- 3 + |-3| = 6
That’s a completely different result given a small modification to the data. New information has changed how you perceive the equation, just like new data can change how Amelia perceives a customer’s intention.
For Amelia, the process of training a machine learning model to understand a customer’s needs is easy. To ensure that Amelia learns and understands effectively, the data used to train her should be:
- Linguistically varied
- Reviewed and potentially revised as she is taught new skills
Data is the mother of AI. Without data, AI and machine learning couldn’t exist. So be kind to your data — your AI depends on it.