The Role of AI in B2B Marketing

Using machine learning and behavioral modeling to generate leads

B2B marketing is easy. All you need to do is find the perfect prospect at precisely the right time in their own educational process. Once you do that, just get exactly the right information into their hands that shows how you’ve got the perfect solution to thoroughly address their pain points. Sit back and watch your business grow like a weed on steroids. Nothing to it.

But, if B2B marketing is so simple, why do some organizations seem to succeed at it while others struggle? Turns out, identifying the perfect prospect at precisely the right time is similar to, but a lot harder than spotting a red umbrella in a sea of black ones. Prospects worth pursuing often can only be identified by subtle cues, patterns of behavior that can only be differentiated by careful analysis of just the right data.

Some companies respond by throwing money at the problem — throwing up their hands and just pursuing all the umbrellas out there. Others get really good at identifying the red umbrellas.

I work for a small software company called Projector PSA that builds Professional Services Automation software targeted at helping consulting companies run their businesses. We compete against rivals that are larger than we are by several orders of magnitude. Because we can’t blindly throw money at problems, we pride ourselves in finding smarter, more efficient ways of doing things. Many of these approaches are rooted in making effective use of the vast amount of data available to us, such as how we’ve used semantic analysis to perform competitive analysis, predictive analytics to improve revenue projections, and neural networks to identify effective project managers.

Today, I thought I’d talk about how we’ve started using artificial intelligence in our B2B marketing efforts. Specifically, how we use machine learning algorithms and behavioral modeling to fuel part of our lead generation engine.

Marketing Funnel

Before diving deeper into the machine learning aspects, I need to briefly describe our overall marketing model. Like many businesses, we think about our marketing process as a funnel that corresponds to a visitor’s progression along their educational journey. They may just be looking for basic information about best practices in running a professional services firm. Our job at this stage is to turn this general interest into awareness that a class of product called Professional Services Automation (PSA) software exists and that it may be of use. Or, they may already know the acronym PSA and are actively comparing different products. Here, our remit is to show how our product is different from others on the market. This funnel (simplified for clarity) looks a bit like this:

Different bits of content on our web site are written to target visitors at each of these educational stages. For instance, we may have a blog post about best practices in measuring utilization targeted at people in the Awareness stage. Or, we may have an eBook that provides a primer on PSA software for visitors looking to get a more in-depth Education on the PSA market. Finally, we have an interactive performance scorecard that allows readers to compare the performance of our product against other products — a piece of content that visitors in the Differentiation stage find useful.

Of all the people who move through the Awareness, Education, Interest, and Differentiation stages, only a small fraction become an identifiable lead by signing up for a free trial or contacting our sales team. It’s at this transition of becoming an SQL, or Sales Qualified Lead, that we gather information about the visitor, their contact information, and their company name.

What, then, of all the other visitors who never identify themselves, who never become an SQL? Are they lost forever?

Research and Proactive Outreach

It turns out that for these anonymous visitors, which we call visitor non-converts because they never converted to a lead, we have one critical piece of information. Each visitor to any web site has an IP address, which is a unique internet address from where the visitor accessed our site.

Many of these IP addresses point back to Internet Service Providers (ISPs) such as Verizon or Comcast, which are a dead-end for us. However, if someone works at, say, Acme Consulting and accessed our site from work, with a technique called reverse DNS lookup, we can often translate that IP address to the company Acme Consulting.

From there, we can point our research team at identifying people at Acme who fit the profile of our typical buyer (say, their COO or VP of Professional Services) and contacting them individually. We don’t necessarily need to reach out to the exact person who actually visited our site, but just knowing that Acme has some interest in a product like ours is a good trigger for us to reach out.

This research and proactive outreach process, if done right, actually should be fairly labor intensive.

Our team needs to make sure we’re reaching out to the right people and figuring out the right targeted, personalized touchpoints for each person. Touchpoints that are not just templatized email spam, and may just as likely be a phone call, a comment on a blog post, a retweet of an interesting perspective, or an introduction to a relevant colleague. Touchpoints that require an understanding of the market, the environment in which the recipient is working, and the pain points that the prospect is likely facing. Most of all, touchpoints that actually provide value to the recipient.

The problem is we may easily have tens of thousands of visitors to our web site each month. Far too many for just the research phase, much less for the labor-intensive personalized outreach.

Behavioral Analysis

This is where, in my opinion, things get interesting…where the data science comes into play. Because we don’t have any hope of doing even the most basic research on each of the tens of thousands of IP addresses each month, we need to figure out how to prioritize. We can use automated means to filter out known ISPs, already converted SQLs, current clients, and other visitors to narrow down the set of visitor non-converts, but that still leaves us with huge numbers that need to be further prioritized. Enter behavioral analysis.

We can tell an awful lot from things like web logs about how a visitor behaved on our site. As mentioned, we break the content on our site out into different categories, such as our blog, our product pages, pages about our company, press releases, job descriptions, etc.

We find that certain pages (such as our pricing page) are good indicators of buying intent on their own. However, it’s often combinations of behaviors that are more interesting.

A person who looks at a smaller number of pages, but at a wide variety of content may indicate higher interest. If a visitor looks at a lot of pages but only at one type, say, our job descriptions, that person is more likely a hiring candidate than a potential customer. Someone who reads only our blog, but doesn’t look at our product pages may be more interested in educating themselves about best practices in running a professional services firm rather than finding a product to help them do so. A visitor who looks at nothing but our product pages and screenshots may be trying to reverse engineer our application and not be at all interested in actually using it.

Our web logs also are able to tell us how a visitor first reached our site — whether they searched for something in Google or clicked through from a partner’s site. They can show how long the visitor spent on each page, whether they signed up for premium content (such as our eBook), and whether they came back multiple times for more information. By tracing each visitor through the data that we collect on our web site, we can do a fair amount of analysis about that person’s behavior and infer some things about their interests.

In the past, traditional marketing automation tools provided some capabilities to turn all this behavior into actionable information through a mechanism called lead scoring. Users would configure the tool to understand that reading a blog post might be worth one point, viewing a product page might be worth two, and spending more than 30 seconds on the pricing page was worth five. This was a bit of a brute-force chain saw approach to a problem that required the precision of a scalpel. It required the lead scoring model to be set up manually based on some fairly arbitrary assumptions on the relative worth of each action, could rarely be applied retroactively to historical data, and often correlated poorly with real-world results.

Machine Learning

Finally, we get to the artificial intelligence and machine learning. We wanted to avoid the arbitrary nature of the manually compiled lead scoring mechanisms. To do this, we decided to teach a machine learning classification model to determine the right weightings of all the behavioral inputs to try to predict propensity to convert.

For each visitor, both successful SQL conversions as well as visitor non-converts, we took all the behavioral data and turned it into a set of flags, ratios, numbers, and other metrics. This will be different for each situation, but for us, this consisted of factors such as total number of pages viewed, fractions of pages that were of each type (blogs, product pages, job descriptions, etc.), average duration on different types of pages, etc. Most importantly, we captured our key target attribute, a flag that indicated whether the visitor converted to an SQL or not:

With all of this data crunched, we turned it into a vector for each visitor and fed all of our visitor data into several machine learning categorization algorithms. After comparing their outputs, we finally settled on a particular neural network model in the end. (I won’t go into the details of the process we used to compare and select from among the different algorithms here.)

This dataset, about 40,000 visitors, served as both training and test data. As we gathered more data going forward, we could make decisions whether to test the new visitors using this initial training model or to use the additional data to continue to train the algorithm.

Either way, what we were after in the end was the model’s confusion matrix. The confusion matrix shows that of all of all the visitors who actually did not convert, the algorithm predicted 99.6% of them correctly. Of the visitors who did convert, it predicted 80.9% correctly. However, other than acknowledging that the algorithm was getting a decent hit rate, these correct predictions, the true negative and true positive cases, actually aren’t all that interesting to us.

What we’re really interested in are the wrong predictions, specifically the 0.4% that the algorithm thought should have converted but didn’t. These false positives are effectively visitor non-converts (visitors who didn’t convert into a lead) who “acted” more like SQLs (visitors who did convert into leads). These are the visitors that are worth taking a closer look at and, if we just take the binary 0/1 prediction at its word, this 0.4% alone turns into over a hundred potential prospects worth chasing.

What’s even more interesting is that we don’t have to just take the binary 0/1 prediction at its word. We can look at all the visitor non-converts (the 0.4% as well as the 99.6%) and look at the p(1) probability provided by the classification model. That is, the probability between 0 and 1 that the visitor should have converted as predicted by the algorithm.

This gives us finer grained control over which visitor non-converts we chase. If our research and outreach teams are totally slammed, we may only chase prospects with a 0.8 probability or greater (a small fraction of the 0.4%). On the other hand, if they’ve got lots of capacity, we may chase prospects with a 0.4 probability or greater (thus dipping a bit into the 99.6% quadrant).

The point is that this approach doesn’t give us just a binary sense of should/should not have converted, it prioritizes our entire list for us based on the probability of predicted conversion.

Conclusions

I’d love to say that this process quadrupled our lead generation rate overnight. I’d love to say that our entire marketing team went on vacation to Bora Bora for a month because their work for the next year was done. But, I can’t. Not because we didn’t get positive results (we did), but because we’re just at the very start of this process…it’s early days yet. We’re starting to get a handle on the prospect identification piece and are working on getting all the processes, infrastructure, and training in place for the research and proactive outreach efforts.

What I can say is that in our initial very small proof of concept, we reached out to four companies that we identified and converted one into an SQL. That’s too small a sample size to reliably predict actual conversion rates long term. But, it was enough for us to expand our initial proof of concept into a larger initiative.

This sort of proof of concept, even if it hadn’t resulted in any positive conversions, is exactly the type of thinking we love to explore as a company to try to gain that little competitive edge.

To look at things a little differently. To try to do things more efficiently. To get better at identifying the red umbrellas.