Case #4: Big data analytics for the future

What is this technology area all about?
Big Data Analytics is an area that focuses on applying techniques from statistics, mathematics and computer science (in particular artificial intelligence) to create actionable insight from very large data sets, acquired from various sources. It also includes the tools and techniques such as natural language processing that are necessary to transform open source intelligence into structured data suitable for Big Data Analytics.

How could this technology be applied?
Big Data Analytics is already transforming all aspects of how both governments and industry manage their business. In particular, we believe it will have enormous impact on the area of threat intelligence, allowing defenders to anticipate threats and act proactively, rather than reacting to current threats and ongoing attacks.

How could this technology be applied to the defense sector?
The defence and security community has for a long time understood the value of intelligence. What BDA brings to this community is the ability to analyze data – both open source and other – at a new scale, and to leverage artificial intelligence/machine learning to create predictive threat intelligence. In the cyber-threat intelligence space, these technologies will be necessary to keep up with the dramatic increase in volume and sophistication of the attackers. For geopolitics, the technology can be key in tracking terrorist organizations like ISIL.

How does this technology match up against the Defense Innovation Initiative?
Big Data has been singled out as areas of great potential to the Defense Innovation Initiative. Big Data Analytics and in particular its applications in threat intelligence are key in giving the US and its allies an advantage in intelligence in the coming decade. As information technology is becoming increasingly accessible to adversaries of all kinds, even more resources need to be spent to keep that advantage. The technology could be used for example to provide intelligence against hybrid threats.

What Technology Readiness Level are we talking about?
Big Data Analytics products exist all across the upper TRL scale today, from 5 to 9. Research (TRL 1–4) is continuing to provide new technology which will further improve the capabilities of BDA.

What are the thresholds against further advancement?
During the last 4–5 years we have seen the first full-scale systems for Big-Data-Analytics-based Threat Intelligence. Cloud computing and advances in distributed algorithms and storage has reached a level where these systems are feasible to build, but for the next 10x improvement in capability, further advances in database technology, analytics, and information visualization are necessary. For open-source-based threat intelligence, further advances in natural language processing, machine translation, and unsupervised machine learning are also very important.

Read more case studies.