Over the past year, we’ve witnessed the development of a new and important movement in the networking community: Collective Intelligence. While the meaning of Collective Intelligence has evolved, its emergence indicates an increasingly apparent need for useful data collected from a variety of vantage points, that is efficiently analyzed to produce actionable insights. The concept has been important enough that Gartner has dedicated an entire report to it even in its early stages, in Innovation Insight for Collective Intelligence Benchmarking, Vivek Bhalla and Will Cappelli, September 26, 2016, which is available to Gartner subscribers.
Because Collective Intelligence is still a developing concept that is increasingly shaping how monitoring solutions are being built, it’s important to be clear what we mean by it. In this post, we’ll discuss Gartner’s definition of Collective Intelligence and further clarify how it affects network monitoring.
Gartner on Collective Intelligence Benchmarking
The Gartner report coined the term Collective Intelligence Benchmarking (CIB), describing it as “a quantitative approach to understanding user experience from a mass-scale perspective from the outset.” CIB takes two forms: externally generated (“utilizing shared datasets from multiple third-party end users”) and internally generated (“aggregating CIB data internally within one’s own organization”).
In sum, CIB is aggregating and analyzing large data sets to understand baselines and benchmark performance of services and applications. This definition is quite broad and in many ways parallels the concept of Big Data. To further clarify the concept of Collective Intelligence, and in order to be more precise about its use in performance monitoring and the innovations that come out of it, we’ve put together our take on this trend.
Our Take on Collective Intelligence
At ThousandEyes, we believe that Collective Intelligence creates a community across which actionable data is shared. It aggregates data from multiple customers, environments or users, analyzing and sharing this data so that all of these parties can mutually benefit from each others’ vantage points. The key is that Collective Intelligence aggregates previously siloed data, allowing people and organizations to benefit from gathered data that was previously inaccessible to them. By pooling data from many different perspectives, we’re able to quickly see patterns and collect insights in the aggregate — with the entire picture in view, Collective Intelligence answers the question, “Is it just me, or is everyone else seeing the same issue?”
Collective Intelligence is also about much more than benchmarking. While benchmarking focuses on comparing performance over time or across services and applications, there are many other types of data analysis that can be useful to a network or IT team. One example comes from our recent release of Internet Outage Detection, which gathers and analyzes data across different customers and organizations to share insights about performance issues occurring on shared Internet infrastructure or common applications. This form of Collective Intelligence is more about event correlation than it is about benchmarking.
Collective Intelligence can also take form within a single organization, aggregating data across a collective of users who are using the same internal network infrastructure. Data from many endpoints is pooled, so that issues can be quickly detected in an aggregate view, whether they affect one person or the entire organization. We plan to execute on this vision for Endpoint Agent in the future.
We believe this more precise definition of Collective Intelligence is useful because it represents a real innovation in the way data is aggregated and shared. The vision of Collective Intelligence is now possible in large part because of the emergence of software-as-a-service (SaaS), which allows data to be collected and pooled across many different customers and users. Collective Intelligence is different from what was previously available — many monitoring solutions in the past may have been able to collect and aggregate large data sets, but were never able to share insights across different customers, environments or users.
The emergence of Collective Intelligence is an important sign of greater things to come. It signals a move away from siloed data and data dumps, and a move toward increased sharing of performance data across customers, environments, networks and users. Not only does it require a unique data set pooled from many different vantage points, but it also requires analysis using intelligent algorithms to rapidly produce truly actionable insights. We at ThousandEyes look forward to executing on this vision of Collective Intelligence.