Rahat Jain
Managing Director at IDIS India
In recent years, the terms ‘intelligent’ and ‘artificial intelligence (AI)’ have been applied to many different types of security systems, but with little apparent agreement when it comes to a precise definition of what AI is. This is despite the fact that 71% of security professionals report that AI video analytics already provides value to their operations or that they expect it to in the future, according to the IFSEC Global Video Surveillance Report 2020, which analyzed feedback from over 700 security respondents globally.
So, it’s important to understand that not all solutions labelled as ‘intelligent’ or ‘powered by AI’ are designed to the same standard or deliver equal value.
Many of the early iterations of video analytics rely on Binary Large Object (BLOB) technology. This is found on most modern IP cameras, which is why they are commonly referred to as ‘blob type’ analytics. These are formulated to detect an event such as a virtual line cross; they detect and track objects as ‘motion blobs’ and distinguish them from smaller binary objects. For many applications these are still useful.
But video analytics capability has moved considerably beyond this. Today security departments can take advantage of deep learning that leverages neural networks made up of multiple layers of algorithms and advanced processing. This is now driving what is widely accepted to mean true intelligent video analytics.
Deep learning engines are ‘trained’ using vast datasets of images and video footage of people, objects and vehicles. They can ‘look for’ size, shape, speed and directional information, and they continue to learn while in use. To an extent, deep learning replicates the way neurons work in the brain – it can analyze and prioritize input from video data to decide which inputs are of value, and it will notify security operatives accordingly.
Deep learning’s real value comes from being able to detect suspicious activity or unusual events and eliminate those smaller binary objects that are just ‘noise’ and apply rules that meet with specific applications and operational requirements. In addition, deep learning should enable users to use metadata to search multiple camera streams to find the most accurate matches for persons or vehicles of interest within minutes.
But again, some caution is needed. Deep learning video offerings can still disappoint, generally as a result of having been launched too early, before engines were fully trained and able to recognize objects reliably and accurately.
Systems integrators need to exercise caution regarding claims and jargon. They need to understand which offerings and which functions will genuinely add value for their customers, and help them to increase productivity, provide useful business intelligence, and ensure they deliver RoI, long term.
Overcoming the Common Challenges Caused by False Alarms
The 2020 IFSEC Global Video Surveillance report cited reducing false alarms as the #1 reason for adopting AI – and for good reason.
Traditional blob type analytics cameras are prone to being triggered by environmental factors such as heavy rain, snow or moving foliage, and struggle to distinguish a human presence, which may present a threat from harmless animal activity.
For users, this can result in time being wasted investigating the cause of alarms, and the larger the site, or more overstretched the system operator, the worse that problem can be.
ALARM OVERLOAD
Alarm overload is a common problem. Operators can quickly become desensitized by false alarms and can start missing genuine threats, or even be tempted to shut off the system.
Alarm receiving centers and virtual guarding firms typically increase charges for more frequent call outs and they may even withdraw monitoring services from problematic sites until cameras are re-configured or replaced. This can result in organizations needing to draft in additional security officers, to maintain protection, or risk leaving gaps in security.
Over time, many organizations find it unfeasible to maintain systems that are prone to false alarms. The solution? By moving to deep learning-based analytics, customers can attain improved situational awareness, with highly accurate AI-assisted notifications for intrusion, object, loitering, and unusual event detection. Security operators will be better able to manage everyday events, and respond to more serious threats and emergencies. In short, safety and security are enhanced by better detection and verification.
ELIMINATING OPERATOR FATIGUE AND INCREASING EFFICIENCY
Unlike human brains deep learning engines don’t get tired. They can constantly monitor multiple camera streams in search of suspicious behavior, maintaining performance levels even in the busiest scenes such as retail malls, logistics centers, higher education settings and outdoor spaces. Relying on human operators to monitor multiple cameras means hiring enough staff to cope and allowing for regular breaks to ensure they stay alert.
Using AI-assisted notifications free-up operators from having to constantly monitor multiple camera streams and video walls. Instead, they can respond quickly and flexibly, and not just from the control room. They can configure alarms to be received to client software, and on mobile devices such as smartphones and tablets, giving the ability to verify and respond to events on the move.
Improving the ability of security managers to oversee security operations away from the control room – by giving them more accurate information along with powerful VMS functionality and tools – lets them better manage incidents on the ground and direct their teams.
The latest generation of AI-assisted tools can transform the work of security teams. Today’s truly smart video technology can allow security provision to be better focused, with officers being re-deployed to more important tasks that add greater value to their roles, for example giving them time to engage with the people they are helping to safeguard rather than remaining unseen in the control room. And strategically, heads of security can interpret and use accurate real-time and historical data to drive more informed decision-making, to better mitigate risk across their enterprise.
Speeding Up Investigations with The Power of Metadata
Deep learning and intelligent video analytics capture metadata even when analytics rules are not applied, meaning that users can benefit from advanced searches across single or multiple camera streams, including across large and dispersed estates.
Deep learning engines can classify people and vehicles including the numbers of vehicles and people in each scene, as well as the colors and shapes. In cases where time is of critical importance such as where suspects need to be found and apprehended, this ability can make a crucial difference. For example, based on eyewitness reports concerning the number of people, the color of their clothing, and type and color of any vehicles involved, operators can search for relevant video footage more quickly. Further, they can track the target’s movements and pinpoint their last known positions from the vast amounts of video data, and do so in minutes rather than hours or even days.
Deep learning engines also extract and record the appearance characteristics of people across video-streams, providing another powerful tool to speed up investigations, and improve the detection capability that can allow security teams to reduce losses and prevent crime.
There are immediate and obvious applications such as finding missing children in large retail malls, or re-uniting friends and family in major sporting and leisure venues. But there are many other applications where person match functionality can be incredibly valuable.
For instance, in retail, loss prevention managers can be alerted by their Point-of-Sale (PoS) and inventory systems to suspicious activity such as ‘sweet hearting’ or stock level discrepancies. In office and corporate enterprise environments, the insider threat is an ever-present risk. By using access control data, security teams can be alerted by an employee acting suspiciously, for example entering facilities out of hours or trying to access restricted areas. When the image of a person of interest is selected, deep learning engines rapidly will scan vast amounts of video data by user-specified time and date to present and collate the closest matches.
This can help loss prevention managers to prevent further shrinkage or let corporate security teams monitor employee behavior and work with HR departments to intervene when a security breach is suspected.
Users can also search video based on the rules they have configured – such as virtual line cross to detect intruders, suspicious loitering of persons and vehicles in specific locations, or unusual events such as vehicles parked illegally – all of which can be precursors to criminal activity. Such activity might occur days, weeks, or months before a crime is committed, and captured video can quicky provide essential forensic evidence.
Business Intelligence
In the past the accuracy of video analytics didn’t always meet user expectations. Today the evolution of processing power and advances in deep learning algorithms, means other departments, not just security, can reap the benefits and gain reliable intelligence to make informed decisions. This gives security integrators the opportunity to use their expert knowledge of video tech to the advantage of a wider set of stakeholders. It opens up the prospect of new projects that offer a high return on investment.
People counting and heat mapping, whether to manage flow or control occupancy, have long been valuable tools for many organizations. Applications include assessing footfall in retail environments for marketing purposes; counting individuals into and out of secure areas for security and safety; managing occupancy for licensing and safety purposes at locations such as nightclubs, theaters, and music venues; or gathering data to help make staff deployment more efficient.
Deep learning advances also enable more accurate surveillance of large public spaces such as retail malls, transportation hubs and campuses, meaning that people counting and heat-mapping can be deployed for both indoor and outdoor applications.
The latest generation people counting combined with heat-mapping can also measure dwell time. This helps generate reports based on trend analysis, and as a result allows more efficient management of sites to meet a range of requirements.
There are obvious advantages for retailers in the ability to correlate video data with Point-of-Sale (PoS) information, allowing them to generate reports which deliver essential branch-by-branch information, and key intelligence such as insights into sales conversions, peaks and troughs, bottlenecks etc. With this more accurate information retailers can look to deliver greater RoX (Return-on-Experience) and convert browsing into purchases, for example with AI-assisted notifications being used to reduce queues at cash registers and changing rooms, to better place promotions, to change and optimize store layouts, and to direct staff using both real-time notifications and historical data to improve the customer experience.
Workspace Optimization
Workspace optimization is becoming more important than ever due to the rise in flexible and remote working, and the pressure for buildings to be made more energy efficient. Looking ahead, facilities managers will need better insights into the ways their sites are used.
AI people counting, heat mapping, and occupancy monitoring will give FM teams the accurate data they need to better understand facility and workspace utilization. This can enable improvements such as adjusting heating, cooling and lighting in real time or using historical data to understand peak flow times, allowing HVAC systems to be re-set to reduce energy consumption.
COVID-19 has already led to a huge shift to home working, and while most companies and indeed employees have missed interacting with colleagues, meeting face-to-face with existing customers and building new relationships with prospects to increase sales, looking ahead it’s clear there will be more demand for flexible working.
AI intelligence will give FM managers the insights they need to manage departments and individual teams, to see which workspaces are becoming overcrowded (hot desk areas or meeting rooms for example), and which spaces remain underutilized.
It’s a varied picture and organizations need accurate data to understand properly what’s going on. For instance, many businesses have found that their back-office staff are more than able to work from home, while R&D and sales teams have underperformed with reduced innovation and motivation. It’s for good reason that employee experience and well being are rising to the top of the C-Suite agenda.
As a result, FM managers will come under increased pressure to provide the right workspaces for teams to be able to work seamlessly and efficiently. This will be more important for attracting and retain the best talent.
In the medium longer term, AI tools will help facilities and estates managers see opportunities to re-design their buildings and floors and decrease traditional office space. Some corporations are already considering a shift to a ‘huband-spoke’ workplace model, in order to reduce wasted hours commuting and to cut carbon emissions. As these organizations look to open more easily accessible and affordable satellite offices away from major city centers or large campuses, AI video insights will allow them to make more informed decisions.
AI-Powered COVID Solutions
At the time of writing, the COVID-19 pandemic is still causing unprecedented ramifications worldwide, including major economic and social disruption. Looking to the future, even when the virus is brought under control societies will remain on high alert for emerging infection full stop after risks.
In commercial settings, asking frontline staff such as security officers and retail staff to enforce government and sector specific hygiene compliance is problematic. Not only are frontline staff already at higher risk of infection, but these additional duties may also put them at increased risk of verbal or even physical abuse. Most people have responded well to the measures needed to prevent the spread of COVID-19 with both patience and good humor, but for others the stresses of the pandemic have sometimes resulted in increased aggression. Mask wearing has been one such trigger. A simple common-sense protective measure for most, but one that is deeply felt as a partisan issue by the few.
At the same time social distancing and occupancy monitoring can impact on the customer or employee experience, and lead to increased infection rates.
The good news is that many of these tasks can now be affordably automated with accurate AI-assisted video analytics that can be configured to issue verbal warnings like the safety and security announcements passengers used at airports and other transportation hubs.
Highly accurate mask detection technology can detect people wearing a mask or wearing one improperly so that it does not adequately cover the mouth and nose. This allows users to configure and trigger event alarms and issue verbal reminders accordingly.
Other functions include people counting and occupancy monitoring to check people coming in and out of premises and show the status of occupancy in real-time. For retailers, dashboards displaying wait times can be positioned at store entrances using a simple 3-step traffic light system to automate the admissions and flow in and out.
Alerts can be used for social distancing violations which allow users to set sensitivity levels depending on local, state or sector specific requirements. Users can also ensure they are alerted to overcrowding or bottlenecks and issue verbal reminders or respond in person when necessary.
Not only do these deep learning analytics support safe back-to-work strategies and undoubtedly help prevent the spread of the infection, they will also help reduce the burden on staff and help meet compliance.
As offices re-open, organizations will want to avoid further disruption to business continuity. They will be rightly wary of viral outbreaks that might not only result in staff off sick and needing to isolate, or expensive deep cleaning services being required, but for big brands the impact of negative publicity too. And AI video solutions that held today will continue to deliver long-term value in terms of business intelligence.
Growing Pressure to Pivot and Adapt
I f COVID-19 has taught us anything it is that businesses need to be able to pivot and adapt quickly. Flexible and scalable AI solutions will be a popular choice, offering backward and forward compatibility, leveraging existing investments in cameras that can be used for security and safety as well as business intelligence purposes, as well as integrating with third party systems, platform, and correlating with other data.
End users will to be keen to see product roadmaps that will allow them to easily add more AI-assisted analytics as they become available without incurring unpredictable increases in licensing fees.
They will be keen to learn of upcoming analytics functions in the pipeline, and new tools that will allow them to combat common challenges such as slips, trips, and falls, which are ever-present workplace risks, especially in sectors such a healthcare, assisted living, retail, and manufacturing. Looking ahead, it also seems unlikely that there will be a return to former ways of doing things – for organizations or for wider society – because these have been revealed as inherently vulnerable.
So, flexibility and the ability to adapt quickly will become priorities. This makes it vital for systems integrators demonstrating new AI tech to also show the true long-term value of technology that is scalable and flexible.
In offering these solutions to customers, systems integrators now have some big opportunities to help them adapt and prosper long term.
