Deep Learning Analytics at the Edge
In 2018 deep learning analytics were nearly all processed either on a server or in the cloud, not at the edge. However, 2019 will be the year of the embedded deep learning application specific integrated circuit (ASIC) system on a chip (SOC).
Due to the power requirements of current GPU hardware, deep-learning analytics have typically had to run on servers. However, the transition of deep learning out to the edge has already begun. NVIDIA offers the Jetson embedded computing platform that allows edge-based inferencing. However, the Jetson platform is an all-purpose GPU not specifically designed for video surveillance cameras. Intel’s Myriad X VPU is the third generation VPU from Movidius and features the Neural Compute Engine – a dedicated hardware accelerator for deep neural network inferences. Deep-learning analytics are also being deployed exclusively in the cloud using Video Analytics as a Service (VAaaS) solutions with just the simple addition of a gateway edge device.
When deep learning-enabled cameras were first launched in 2016, very few AI chipset options were available. NVIDIA’s Jetson and Movidius’s Myriad were often used in deep learning-enabled camera product demonstrations. However, high prices and high-power consumption of these chips meant the early specific AI chipsets has limited adoption in cameras. IHS Markit expects that in the next few years, the SOCs designed for network cameras will be capable of performing the basic processing required for deep learning analytics to run on the camera, without the need for additional processing power. The ASIC SOCs will be beneficial for large scale production aimed at the price sensitive mass market. ASIC SOCs with lower power consumption and a more compact design are being developed. Both established semiconductor giants and smaller startups are developing ASICs for use in deep learning-enabled cameras increasing competition in this area.
Currently both Ambarella and HiSilicon, a subsidiary of Huawei, are developing ASIC SOCs for network cameras. Ambarella has already released the CV2S SOC, however, this chipset is presently too high priced and overly powerful for mass market video surveillance requirements. It is likely to be used for autonomous vehicles. But, currently in development, and due to be released in early 2019, the CV22s includes CVflow architecture that provides the DNN (Deep Neural Network) processing required for deep learning analytics. Similar to Ambarella, HiSilicon is developing the Hi3559A SOC with a CNN accelerator to allow the processing deep learning analytics; Whilst Qualcomm will soon be releasing the QCS605.
The level of inference is something that can be changed with tradeoffs in features, accuracy, frames rates, and resolution, but IHS Markit expects that 0.2 Deep Learning Tera-operations per Second (DL TOPS) is enough for a basic classification network with low frame rates. IHS Markit expects that by 2022, 50% of network cameras shipped globally will include a deep learning accelerator that can provide between 0.5 to 2 DL TOPS. This will allow cameras to do basic object detection and classification leading some to describe it as ‘the motion detection of tomorrow’ hinting it will become a standard feature. Additionally, it is not expected that deep learning accelerators will add any significant cost to the price of the SOCs.
It is expected that the development of edge-to-core processing will become significantly more common in the coming years. As such more powerful edge devices will help distribute the required workload. It is not expected that these edge devices will replace the need for central server or cloud processing, but instead will complement each other.