Skip navigation
2012

Digital video has become commonplace in a wide variety of embedded devices and promises to be the cornerstone for a new generation of applications based on image analysis. Designers are using electronic images of people, places, and things to extract information for applications that bridge the physical and virtual worlds. Manufacturing, biometrics, surveillance, quality control, security, medical, and commerce are just a few of the market areas that incorporate video processing algorithms to analyze real-time images. For example, in an earlier Roving Reporter post, I covered the Intel Audience Impression Metric Suite for digital signage which provides video and face detection algorithms that can dynamically modify content depending on the viewer’s demographics. Similarly, industrial production line inspection stations now include multichannel, high-speed video analysis equipment to record and verify product configuration and accurate alignment.

 

As these video applications grow in complexity, designers are turning to dual cameras to add another dimension or to increase the precision. Two image sensors are typical for applications such as 3D stereoscopic video, robotics, black box car driver recorder designs, accurate 3D analytics for security/surveillance and many other applications.  Although today’s image signal processors are capable of processing the data that two image sensors output, they do not often have the port configurations to support multiple sensors. In addition, as image resolutions move above 720p, camera vendors have replaced the traditional parallel CMOS bus with unique serial buses with different widths, speeds, and protocols. With these challenges in mind, Lattice Semiconductor devised the MachXO2 family of Programmable Logic Devices (PLDs) which, along with an inexpensive frame buffer, can combine two image sensors, synchronize them, merge the data, and output a format compatible with single input port configurations (See figure 1). The MachXO2 dual sensor interface design can output two images in a top / bottom format or a left / right configuration, depending on the format the signal processing software expects. Both the Intel® Atom™ and 2nd generation Intel® Core™ processors have undergone significant updates for graphics processing and would be excellent Image Signal Processor (ISP) candidates for these new multi-sensor applications.

 

img44139.png

 

The new Intel® Atom™ platforms (codenamed Cedar Trail) feature an integrated Intel® Graphics Media Accelerator 3600/3650 graphics engine to enhance 3D performance for media applications such as high definition 1080p video playback and streaming at a fraction of the power consumption of previous generations. Intel streaming single instruction, multiple data extensions can also be used to accelerate software processing of complex arithmetic and video decoding tasks. The platform delivers multiple digital display and output options including LVDS, HDMI, VGA, and DisplayPort to support a variety of presentation formats.  The dedicated media engine combined with the integrated memory controller provides enhanced performance and system responsiveness, including an improvement in graphics performance up to 2X compared to the previous generation platform. Theses signal processing and display features are well suited for multi sensor embedded market applications.

 

The 2nd generation Intel® Core™ architecture features an integrated graphics processor optimized for media analysis applications plus dedicated hardware for high speed video processing. The graphics processor incorporates fixed function hardware in the signal processing channel as an array of parallel execution units for rapid encoding and decoding of high definition video in order to maximize the throughput per watt and to replace programmable functions.  The video processing section includes advanced logic for removing noise, sharpening, scaling, and color processing of video signals. The 2nd generation Intel® Core™ architecture also features a unified power management design where the graphics processor has a separate power plane and clocking so it can run at a different voltage than the CPU depending on the workload. The 2nd generation Intel® Core™ architecture also incorporates the Advanced Vector Extensions (AVX) instruction set optimized for audio, image, and video processing. With the AVX extended performance, designers can eliminate external hardware-based digital signal processing silicon to reduce the component count and lower overall power requirements.

 

The integrated graphics features of both the 2nd generation Intel® Core™ architecture and the new Intel® Atom™ platforms give designers a choice in video processing capabilities combined with low power and a variable number of CPU cores to match the requirements of a variety of multiple image sensor applications. If you are starting a new dual camera image analysis project and you have questions, please share your concerns with fellow followers of the Intel® Embedded Community.  You can also keep up with technical articles and product announcements at the Embedded Computing Design archives on Image Analysis.

 

To view other community content on Sensing and Analytics, see “Sensing and Analytics - Top Picks

security_analytics.jpg

Warren Webb

OpenSystems Media®, by special arrangement with Intel® Embedded Alliance

 

Lattice Semiconductor is a General member of the by Intel® Embedded Alliance.

 

High-performance vector and matrix processing is a central requirement for applications like radar detection, video analytics, medical imaging, and factory automation. In the past, developers often used specialized digital signal processors (DSPs) or field-programmable gate arrays (FPGAs) in these applications. Today, multi-core Intel® architecture (IA) processors with Intel® Advanced Vector Extensions (Intel® AVX) offer a more flexible alternative.

 

Intel AVX, an advanced form of Intel® Streaming SIMD Extensions (Intel® SSE), doubles peak floating-point performance by widening the floating-point data path from 128 bits to 256 bits. When combined with the multi-core performance of processors such as the 2nd generation Intel® Core processors – which offer up to four cores at 3.4 GHz – these instructions provide impressive signal processing performance. For a detailed look into the new instructions, see Intel® Advanced Vector Extensions: Next-Generation Vector Processing.

 

Developing and porting applications with Intel AVX is made easier by broad support from the Intel® Embedded Alliance. This organization’s 160-plus members collaborate closely with Intel to create optimized hardware, software, tools, and services that give OEMs a head start on their designs. Table 1 shows some of Intel’s and Alliance members’ tool and software support for Intel AVX.

 

OS and Hypervisor Support

• Various versions of Microsoft* Windows*, including   Windows* XP Embedded and Windows* Embedded Standard 7
• QNX* Neutrino*
• Wind River VxWorks*
• LynuxWorks* LynxSecure 5.0
• Linux*

Tool Support

• Microsoft Visual Studio 2010
• Intel® C++ Compiler
• Intel® VTune™   Amplifier XE
• GNU tools including GCC, Binutils, and GDB
Yasm Modular   Assembler
• Netwide Assembler (NASM)
AltiVec.h header   file (PowerPC* to Intel® AVX conversion)

Vector Signal and Image Processing Libraries (VSIPL)

• GE Intelligent Platforms AXISLib-VSIPL
• Curtiss Wright Controls Embedded Computing Continuum Vector
• Mercury Computer Systems MultiCore   Plus MathPack
• CodeSourcery VSIPL++
• N.A. Software VSIPL
• RunTime Computing Solutions VSI/Pro

Other Software Libraries and Components

• Intel® Integrated Performance Primitives (Intel® IPP)
• Intel® Math Kernel Library (Intel® MKL)
• Intel® Thread Building Block
• Intel® Array Building Blocks 1.0 (beta)
• GNU C Library (glibc) 2.11

Table 1. Partial list of the software support for Intel® Advanced Vector Extensions.

 

Several  of the companies listed in the table above also provide signal processing hardware. As noted in the blog Intel AVX-enabled processors excel in sensing and analytics, much of this hardware targets military and aerospace markets. For a deeper dive into the hardware targeting mil/aero – and for the applications of Intel AVX in this space – check out the article SWaP-able Solutions Take Off in Mil/Aero.

 

Other applications that can leverage Intel AVX include digital signage and surveillance. Video analytics plays a key role in both markets. In the case of digital signage, anonymous video analytics (AVA) can be used to analyze the viewing audience and tailor signage content accordingly – see the blog Audience measurement optimizes digital signage applications for an overview.

 

Surveillance applications have opportunities for a broader range of analytics, such as left-package detection, that can improve security and reduce the burden on operators. The challenge here is the fact that many organizations have already installed analog camera systems. As the article

Upgrade Analog Surveillance with HD and Analytics explains, this challenge can be overcome by deploying hybrid surveillance systems.

 

A similar set of opportunities and challenges is found in the factory automation market. As noted in the blog Advanced CPU Architectures Boost Production Testing, high-speed digital signal processing and video analysis functions are commonly found in production test systems. Implementing these functions with Intel AVX can simplify your designs and cut the bill of materials (BoM). More broadly, signal processing tasks can be found throughout the factory for sensing and control. The article Replace Real-Time Hardware with Multi-Core Software has a great perspective on how to HMI and control tasks using IA processors (see Figure 2).

IntervalZero_figure_3_cnc.png

Figure 2. Multi-core IA processors can handle both HMI and control tasks.

 

security_analytics.pngThe links I’ve listed here only scratch the surface of what the Alliance has to offer.  To learn more about advanced signal processing, see intel.com/go/embedded-analytics.

 

 

GE Intelligent Platforms, Microsoft Corporation, QNX Software Systems, Ltd., and Wind River Systems are Associate members of the Intel® Embedded Alliance. Curtiss Wright Controls Embedded Computing and LynuxWorks are Affiliate members of the Alliance. CodeSourcery and Mercury Computer Systems are General members of the Alliance.

 

Kenton Williston

Roving Reporter (Intel Contractor), Intel® Embedded Alliance

Editor-In-Chief, Embedded Innovator magazine

Follow me on Twitter: @kentonwilliston

Embedded markets of nearly every stripe are clamoring for a new generation of devices that can intelligently connect to other devices, to the enterprise, and to cloud services. Customers worldwide are discovering that Internet connectivity can help boost revenues, lower costs, and enable entirely new business models. In this blog, we’ll examine emerging connectivity requirements and design solutions for various markets, including retail, medical, automotive, manufacturing, and energy.

 

First, let’s review two key terms: cloud computing and machine-to-machine (M2M). Pinning down an exact definition for either term is a bit tricky – everyone seems to have different ideas of what the terms mean. At the highest level, however, we can define cloud computing as the use of virtual servers available over the Internet.  Intel has been very active in this area in recent years, as has the the Intel® Embedded Alliance, whose 160-plus members collaborate closely with Intel to create optimized hardware, software, tools, and services that give OEMs a head start on their designs. If you want to know more about the topic, the webinar Cloud Computing for Network Equipment by Premier members Emerson Network Power and RadiSys is a good place to start.

 

As the name implies, M2M is all about connecting machines to one another. Although definitions vary, this typically means connecting field devices to the enterprise, often through a cellular network. In many cases, M2M applications leverage cloud computing, both to provide an interface between field devices and the enterprise, and to provide specific services within the cloud. For example, the Eurotech Everyware Device Cloud, illustrated in Figure 1, provides the following services (Eurotech is an Associate member of the Alliance):

  • Web-based dashboards      that display device data in real time
  • Management reports      that monitor trends
  • Integration with      popular services such as Salesforce, Twitter, Facebook, and Google apps
  • Notifications      automatically sent to designated recipients via SMS, email, Twitter, or      automated phone calls
  • Simple integration      into SAP, Oracle, and customer-developed applications
  • Mobile support to      provide relevant data to specified users on the go

End_to_end_solution.png

Figure 1. The Eurotech Everyware Device Cloud.

 

M2M technology is being embraced in a wide range of applications, including retail, medical, manufacturing, and energy. For an overview of its uses, check out our M2M Experts Round Table blog with Eurotech, Premier member Kontron, Affiliate member ILS Technology, and General member GoS Networks. Or dig deeper with the white paper Connecting the Dots in M2M.

 

One of the key challenges of M2M is the complexity of deploying these systems. In light of this challenge, Intel recently collaborated with Kontron, ILS Technology, Wind River, and top cellular carriers to create a M2M Smart Services Developer Kit that brings together field hardware and software, cloud services, and cellular connectivity. (Wind River is an Associate member of the Alliance.) You can read all about the kit – shown in Figure 2 – in the article Fast-Tracking M2M with a Standards-based Bundle and the white paper Simplifying M2M. Additional details are also available in the webinar Speeding M2M Solutions To Markets.

 

m2m kit.png

Figure 2. The M2M Smart Services Developer Kit.

 

Now let’s zoom in a bit to see how Internet connectivity is impacting a few key markets. Digital signage makes a particularly good case study. Early digital signs mainly provided information flow in one direction. In contrast, the latest offerings incorporate anonymous video analytics (AVA) to report viewership trends to the back office, enabling advertisers to tweak their displays for greater impact.  By adding two-way connectivity, signs can also support interactive features such as sales, subscriptions, and other real-time transactions. To see examples of how Emerson and Associate member Norco are supporting these applications, read the blog Connectivity and remote management key digital signage explosion and the e-book Digital Signage Gets Smart.

 

Connectivity is also in high demand for in-vehicle infotainment systems. Consumers increasingly expect their cars to offer services such as live traffic and streaming radio, and to connect to portable computing, cellular, or entertainment devices. A major challenge in this space is implementing a platform that can keep up with the rapid changes in consumer electronics. This is where the flexibility of Intel-based platforms can provide a significant advantage. To see example solutions from Norco, Associate member congatec, and General member OpenSynergy, see the blog Mobile computers spawn vehicle infotainment systems.

 

Medical applications are also clamoring for connectivity as the world moves to electronic records and hospitals seek to differentiate themselves by offering more sophisticated bedside entertainment. However, medical applications also present tough security requirements. Thus, security and remote management capabilities are paramount for connected medical devices. To see how you can implement these features, see A Prescription For a Secure Hospital by Emerson Network Power.

 

Finally, let’s take a look at automation markets, including industrial automation, building automation, and home automation. While these are notably diverse markets, one characteristic they share is a desire for connectivity – including connectivity both to the cloud and to local automation buses – combined with intuitive touch-screen interfaces. Intel® Atom™ processors and the Qt* software platform are a good match for these requirements, as shown in the article Touch-Screen Automation, Simplified by General member M31 S.p.A. Together these technologies enable rapid development of automation gateways (Figure 3) with sophisticated capabilities.

 

m31-3.png

Figure 3. Example architecture and applications with an automation gateway.

 

connectivity.pngThe links I’ve listed here are a great start.  For more on extending the Internet to embedded devices, see intel.com/go/embedded-connectivity.

 

 

 

Emerson, Kontron, and Radisys are Premier members of the Intel® Embedded Alliance. congatec, Eurotech, Norco, and Wind River are Associate members of the Alliance. GoS Networks, ILS Technology, and OpenSynergy are General members.

 

 

 

Kenton Williston

Roving Reporter (Intel Contractor), Intel® Embedded Alliance

Editor-In-Chief, Embedded Innovator magazine

Follow me on Twitter: @kentonwilliston

Communications and networking equipment often employ multi-architecture designs that combine network processors (NPUs), digital signal processors (DSPs), and general purpose processors (GPUs). From a workload perspective, this approach makes sense because telecom equipment handles highly diverse workloads – namely applications, control plane, packet processing, and signal processing. However, the multi-architecture approach makes development more complicated and increases the difficulty of scaling designs to suit different markets.

 

This problem can be solved by consolidating telecom workloads onto a single architecture. Such consolidation is possible thanks to the leading performance of multi-core Intel® Xeon® processors, along with advances in virtualization and other software technologies. Together, these technologies enable up to 4:1 workload consolidation, making it possible to run application, control, packet, and signal processing on a single platform. Figure 1 shows an example of how this consolidation can be used to greatly simplify base station designs. Other applications include:

  • Unified threat management (UTM) and other security applications
  • IP Media Servers, such as those used in Enterprise PBX systems
  • Deep Packet Inspection (DPI) in LTE infrastructure and other applications

 

base station example.png

Figure 1. Consolidating workloads onto a single architecture greatly simplifies designs.

 

If you are new to this concept, the white paper Consolidating Communications and Networking Workloads onto one Architecture is a great place to get started. It shows how telecom equipment suppliers can reduce development effort, power consumption, and time to market – all while lowering their customers’ CapEx and OpEx. Among other things, the paper highlights real-world packet-processing benchmarks using solutions from members of the Intel® Embedded Alliance. (See Figure 2.) The Alliance’s 160-plus members collaborate closely with Intel to create optimized hardware, software, tools, and systems integration services that give OEMs a head start on their designs.

 

packet performance.png

Figure 2. Packet processing on Intel® Xeon® processors.

 

As the above chart suggests, the increasing packet processing performance of Intel Xeon processors is a critical ingredient for workload consolidation. Developers can unleash this performance with the Intel® Data Plane Development Kit (Intel® DPDK). As shown in Figure 3, the Intel DPDK provides optimized packet processing libraries for Linux*. For details, I recommend the white paper High-Performance Multi-Core Networking Software Design Options by Wind River, an Associate member of the Alliance. This white paper covers standard Linux* SMP, Linux with Intel DPDK, and the Wind River Network Acceleration Platform.

 

dpdk.png

Figure 3. The Intel® Data Plane Development Kit (Intel® DPDK).

 

A key challenge in workload consolidation is the fact that different workloads may require different operating systems (OSs). This challenge can be overcome with virtualization, a technology that enables operation of multiple OSs on a single hardware platform (Figure 4). Intel Xeon processors support virtualization through Intel® Virtualization Technology (Intel® VT), a set of technologies that provide hardware acceleration for virtualization.

 

norco+figure+1-alternative.png

Figure 4. Virtualization enables multiple OSs to run on the same hardware.

 

Keeping this background in mind, let’s look at how some Alliance members are using Intel Xeon processors to enable workload consolidation.

 

Unified threat management (UTM) appliances are one of the hottest segments of the security market. These devices integrate a full spectrum of security technology – such as intrusion detection, firewall, and anti-virus – to help customers reduce security costs. Bringing all of these workloads onto a multi-core Intel Xeon processor makes it easier to scale UTM designs. For example, Figure 5 shows how packet processing performance tracks linearly with the number of cores used. This figure comes from the excellent article, Scalable Performance for Unified Threat Management, written by Advantech, a Premier member of the Alliance.

 

fastpath.png

Figure 5. IP forwarding performance for a 2.4 GHZ Intel® Xeon® processor E5645 running 6WINDGate* from 6WIND. The performance varies based on the number of fast path protocols running in the system.

 

One of the challenges of combining security workloads is the fact that different security functions often require different OSs. As noted above, virtualization is the key to bringing such functions together. To see how you can leverage virtualization to build multi-function security solutions, check out the article Versatile Network Security Devices by NORCO, an Associate member of the Alliance.

 

Deep packet inspection (DPI) is another hot market, and one that is full of challenges. In LTE applications, for example, DPI solutions must cope with exploding data capacity and increasingly complex packet inspection. Developers can meet this need by combining Intel® Xeon® processors with 6WINDGate*, a specialized packet processing software product from Affiliate Alliance member 6WIND.

 

Figure 6 shows the inner workings of the 6WINDGate. This software was recently benchmarked on an AdvancedTCA* (ATCA) blade from Premier Alliance member Emerson Network Power. For a basic fast path configuration including VLAN, IP forwarding, GTP-U tunneling, flow accounting, and QoS conditioning, the Intel® Xeon® processor E5645 on this blade can process 2.5 million packets per second (Mpps) per core with an average packet size of 512 bytes. This equals roughly 10Gbps of packet ingress and 10Gbps of packet egress per core. To translate these numbers into real-world performance, Emerson’s carrier grade Centellis 2000 2-slot ATCA platform could deliver up to 80Gbps of throughput in a 3U form factor. For more details, see the article Deep Packet Inspection in LTE Networks.

 

6wind+figure+2.png

Figure 6. 6WINDGate* DPI platform for Intel® Xeon® processor-based PCEF.

 

Real-time IP media processing typically runs on specialized DSPs, but Intel Xeon processors can deliver carrier-class signal processing using software instead, even under heavy load. The Convedia* Software Media Server from Premier Alliance member RadiSys uses this software-based approach to implement an IP media processing platform that can scale to thousands of ports on a single 1U rackmount server, or tens of thousands of ports on a fault-resilient bladed chassis. Using commercial server hardware, this Linux-based SIP server can consolidate the functions of announcement and recording servers, audio and video conference bridges, interactive voice response units (IVR/VRU), messaging equipment, and speech platforms. In addition, a special “co-residency” capability allows the integration of VoIP all-in-one telecommunication products. A single Convedia Software Media Server deployment can support a broad range of enterprise VoIP applications, including IP PBX, IP Contact Centers, VoiceXML-based IVR, Unified Messaging, or voice/video enterprise-wide conferencing.

 

workload_consolidation.pngThe links I’ve listed here only scratch the surface of what the Alliance has to offer. For more on building flexible networking solutions, see intel.com/go/embedded-consolidation.

 

 

 

Kenton Williston

Roving Reporter (Intel Contractor), Intel® Embedded Alliance

Editor-In-Chief, Embedded Innovator magazine

Follow me on Twitter: @kentonwilliston

Filter Blog

By date: By tag: