NetApp’s strategy to address the Internet of Things (IoT) is Data Fabric.

Matt Watts, NetApp’s Director of Technology and Strategy for EMEA, writes about how Data Fabric forms part of a ‘Layer Cake’ approach for the IoT…

“As the reality of the Internet of Things edges closer, organisations will need to move past strategic and theoretical discussions to something more substantive,” explains Watts. “While every mention of IoT seems to start with the game of who can cite the highest possible number of connected IoT devices, the focal point should really be how to work with the vast amounts of new data IoT devices are poised to generate. One well-known car company built up about 14 petabytes of storage over several years. But in just the last few months, its driverless car program created an additional 4.5 petabytes.

“This unbridled data growth will make an already challenging IT problem – organising the data to make it valuable – even greater. Advanced data management and organisation is essential to effectively assimilating and using the flood of data that will come from IoT. However, given the scale and scope of this data, we need new thinking and approaches beyond traditional solutions.”

The IoT Layer Cake

Data Fabric is NetApp’s vision for the future of data management. It enables organisations to manage their data seamlessly as it flows to wherever they need it most. In relation to the IoT, NetApp views this is a ‘Layer Cake’ approach with a set of integrated functions to meet the challenge of IoT. There are four fundamental layers in this cake, and they are:

1. Partnerships and Solutions
2. Common APIs
3. Data Fabric
4. Data Platforms / Data Lakes

These four elements each provide a critical part of a holistic solution that will empower organisations to use IoT data and provide IT departments with a strategic infrastructure that is not cost prohibitive or too limited to support huge incoming data flows.

1. Partnerships and Solutions

Explaining the partnerships and solutions element, Watts explains that this is aimed at “providing customers with complete, fully integrated approaches that solve common problems or needs. At this time, NetApp is currently working with two key partners on IoT solutions. Virdata, a Technicolour company, provides the ability to capture any type of data from any type of device and deliver real-time insights and analytics. The Virdata service offering is developed in NetApp FlexPod for OpenStack.

“The other partner, IBM Cloud (SoftLayer) provides the compute platforms, cloud infrastructure and turnkey big data solutions for deploying Virdata’s service offering. These two partnerships provide strong examples of how NetApp can provide complete solutions for specific applications or scenarios.”

2. Common APIs

Common APIs are a vital element in making the layered approach work across a broad base of IT infrastructure, products and platforms. Watts identifies how “APIs are the preferred approach for interaction with key components of the layer cake, and individual developers or organisations will have their favourites, making broad API support a critical function.”

As we are still in the early stages of IoT, Watts asserts: “[…] we just don’t know all the answers at this time, so ensuring that there is a focus on API support is essential to the layer cake’s long-term validity. Some existing APIs will obviously play an important role. OpenStack, for example, will be a very important set of APIs for delivering value from IoT data. OpenStack is already included in the NetApp IoT layer cake approach.”

3. Data Fabric

 NetApp’s Data Fabric offers a single, standardised data management and storage solution which works across different architectures and platforms to provide a common method of making data available without placing it in silos or limiting flexibility. It links the data platform, data lakes, APIs and key software products that will be used at a higher level to create IoT applications. It therefore represents a consistent interface that makes it easy to provide IoT data to specific workloads or applications.

Watts argues that the requirement for a solution like Data Fabric is “magnified when the huge amount of data from IoT starts to become part of an organisation’s overall data asset. In addition, using Data Fabric allows IT organisations to store data on the most appropriate and cost-effective type of physical storage […]. Data Fabric does this with a ‘single pane of glass’ management console for all data, regardless of what platform it is stored in or where it might be located.”

4. Data Platform & Data Lakes

Watts continues: “The data platform layer is designed to provide an enhanced approach to Platform as a Service (PaaS) for IoT. The platform is made up of autonomous components including capabilities such as compute, object, store, networking, orchestration and other functions. The key to IoT support is that these components are designed to support massive scale out to ensure necessary throughput.

“The data platform also provides simplified integration with key software products and environments. A good example of this is the NetApp NFS Connector, which allows analytics to run natively on NFS data with tools such as Hadoop. The data platform also supports enterprise software solutions from vendors such as Oracle, Cisco, VMware, Red Hat, Microsoft and CA.”

Meanwhile, data lakes are a set of global “object pools” that offer broad access to large amounts of data. They use a flat architecture to store vast amounts of raw data and it this type of requirement that will become common in the IoT. Watts explains: “Each data element in the lake has a unique identifier and an extended set of metadata tags. The use of Hadoop with data lakes is already increasing as IT organisations are looking at higher levels of data scaling and accessibility.”

Data Fabric to Drive IoT Innovation

NetApp’s Data Fabric solution is ideal for creating a flexible, scalable and adaptable method of managing and protecting data, both now and in an IoT centred future. Watts concludes: “With IoT data coming from so many devices and endpoints, Data Fabric’s ability to centrally store and deliver data to any infrastructure component or workload becomes increasingly important. In addition, as the size and number of data lakes grow exponentially in an IoT-driven world, Data Fabric is an excellent solution for managing and protecting these huge data stores.

“The age of IoT requires fundamental change to older, traditional data storage and management approaches. NetApp has developed a solution that bridges current and future requirements, allowing your organisation to create value from IoT without storage bottlenecks or limitations.” 

(Adapted from original blog article written by Matt Watts, Director, Technology and Strategy, EMEA, published on 21^st February 2016: https://community.netapp.com/t5/Partners/Enabling-the-Benefits-of-IoT-through-a-Data-Fabric/ba-p/116103)