The Internet of Things (IoT) promises to change how we interact with our world in nearly every way by enabling us to access data anywhere in the world.  Smart energy, or smart grid technology as is it also known, is one of the quieter aspects of IoT, given its machine-to-machine (M2M) implementation.  Advances in solar technology, for example, enable individuals to generate power beyond their needs and put it back onto the grid for others to use.  Similarly, large arrays of solar panels can provide an effective alternative source of power. 


Managing solar systems requires the collection, aggregation, and analysis of tremendous amounts of data.  Consider that each meter on the smart grid needs to be checked regularly and continuously to monitor the amount of power it is consuming or generating (when solar is present at the location).  The magnitude of data being generated is on the order of TBs/month and needs to be consistently tracked, both for billing accuracy and to enable useful availability forecasts so utility companies can reliably meet shifting demand.


Such "Big Data" problems change how we approach the management of solar and other smart energy systems.  Architectures are required that can provide high performance processing with the ability to manage large amounts of analog data.  In terms of processing capacity, real-time analytics are needed to use collected data to full advantage.  Often these algorithms are compute-intensive, and a great variety of algorithms may be in use at any one time to support advanced capabilities such as early warning detection of disturbances and potential blackouts.  In addition, to gain insight into seasonal output and behaviors of users, past data records will need to be retrieved for analysis, increasing the capacity requirements of the system.

Figure 1.png

Figure 1: The 3 Tiers of Big Data applications (source: National Instruments)


Big Data applications can be segmented into three tiers (see Figure 1).  At the edge, Tier 1 sensors collect data.  Tier 2 networked nodes convert this analog data to a digital format, process early analytics, and forward data to Tier 3 servers in the IT infrastructure for more advanced analysis.  Ideally, end-to-end solutions are required that can seamlessly interface data between the tiers.


National Instruments (NI) offers Intel-based platforms to enable organizations to address the five “V”s — volume, variety, velocity, value, and visibility — of Big Data applications.  Because accuracy and timely monitoring are critical, especially when data are voluminous and geographically dispersed, sophisticated solutions are required.


To assist with Tier 2 processing, NI offers its CompactRIO and PXI systems controlled by either a PC or custom embedded control module.  The NI CompactRIO is a rugged, programmable automation controller based on the Intel® Core™ processor with reconfigurable chassis, user-programmable FPGA, and hot-swappable I/O modules.  Its flexible architecture provides the performance required for advanced control applications, high-speed data transfer and logging, and processor-intensive analysis.


Figure 2a.pngFigure 2b.png

Figure 2: NI’s CompactRIO and PXI Systems, with LabVIEW software, provide data acquisition and analysis for Big Data applications like solar and smart energy (Source: NI)


The NI PXI system is an open platform for test, measurement, and control.  With specialized synchronization buses, high performance processing, and real-time measurement capabilities, the NI PXI system offers high bandwidth with low latency.  The system can be driven by a PC or by embedded control modules based on the Intel® Atom™ and Intel® Core™ processors.  I/O modules are available from over 70 vendors, providing the flexibility to support I/O from high-resolution DC to 6 GHz RF.


Both platforms are supported by a variety of software tools, including NI’s graphical LabVIEW system design software.  Using C or VHDL, LabVIEW abstracts the complexity of programming FPGA resources for real-time processing of digitized signals at the point of capture.  LabVIEW also targets the Intel architecture controller to perform visualization as well as in-motion and early life analytics.  Key features of LabVIEW include its high level of integration with NI’s hardware platforms, unique graphical programming environment, data analysis and visualization capabilities, and built-in application-specific libraries for software functionality and hardware interfaces.


The Intel® architecture is ideally suited for meeting the requirements of Big Data applications.  With their multi-core capabilities, the Intel Atom and Core processors provide the performance needed for real-time analytics in a flexible and power efficient manner.  Intel® Virtualization Technology (Intel® VT), combined with integrated advanced graphics engines, enables complex visualization of data as well as supports multiple OSes running on the same hardware platform.  Data management is also simplified through close collaboration between NI and Intel, as well as a wide range of other industry leaders, resulting in a vast ecosystem of solutions, software, and tools that enable robust Big Data processing across all three tiers.


Another aspect of Big Data systems is that they commonly aggregate many data channels, thus management features such as discovery, status, update, security, diagnostics, and calibration are required to reduce risk when integrating the NI CompactRIO and PXI system with Tier 3 IT infrastructure.  The Intel architecture helps facilitate integration through reliability, availability, serviceability, and manageability (RASM) features in both hardware and software.  These RASM features improve overall efficiency by enabling advanced functionality including remote monitoring, separation of critical tasks from other applications through virtualization, trusted execution to prevent unauthorized software from running, secure boot technology, accelerated cryptographic capabilities, and memory interfaces with error correcting code (ECC) for greater data reliability.


The magnitude of Big Data applications requires engineers to approach data acquisition and analysis in new ways.  With the right hardware and software tools, solar and other smart energy applications can leverage the vast amount of data available to achieve the highest efficiency.


Learn More

Contact Featured Alliance Member:

Solutions in this blog:


Related topics:


National Instruments is an Associate member of the Intel® Intelligent Systems Alliance.



Nicholas Cravotta

Roving Reporter (Intel® Contractor), Intel® Intelligent Systems Alliance