A key feature of connected lighting systems is their ability to collect data from sensors and share that data with other lighting and building systems. While occupancy sensors have been widely adopted as an energy-saving strategy in buildings, well-described methods for testing the performance of such sensors are few and relatively unsophisticated. As a result, it’s been difficult to predict the performance of occupancy sensors in buildings, and in practice they often fail to meet expectations in terms of energy savings. The reasons for poor performance range from fundamental technology limitations, to misconfiguration, to poor placement in the room — where they’re sometimes removed or disabled by dissatisfied occupants because of such problems as false positives, false negatives, or failure to commission the device in a manner suitable for the specific application.
To that end, Pacific Northwest National Laboratory (PNNL) conducted a literature review of well-described test methods for characterizing some occupancy sensors’ performance, and of research articles containing ad-hoc test methods, in order to identify and consolidate the best techniques for characterizing sensor performance in indoor spaces, as well as identify any test-method gaps that might need to be filled in order to evaluate emerging technologies and products. Based on this review, we came up with recommendations for test methods to characterize occupancy sensors that target the indoor environment. This article presents those recommendations, along with the background and reasons why we think they’re needed.
The status quo
The terms “occupancy,” “presence,” and “motion” are widely used, at times interchangeably. “Occupancy” and “presence” are typically used to imply very similar occurrences — namely, that a defined space is occupied by some object of interest (e.g., humans, animals, cars, bikes), or that an object of interest is present in that space. Motion is simply the change in physical position of an object of interest.
Most occupancy sensors deployed in buildings today utilize readily available and inexpensive passive infrared (PIR) sensors to detect motion, which thus becomes a proxy for occupancy. This is a common characteristic of sensors in general; instead of directly detecting or measuring their intended parameter, most detect or measure something related to it. The absolute status of a specific parameter is often referred to as the “ground truth,” and sensor errors — whether systematic, random, false positives, or the result of some erroneous inference — are failures to detect that ground truth.
In our review, we used the term “occupancy” to cover all ways to detect an object of interest in a space — such as whether it has entered a space, left a space, or is persisting in a space. Presence is generally considered the most fundamental aspect of occupancy and is the aspect most commonly referred to in energy-saving strategies; i.e., when no one is present in a room, the lights should be turned off and the thermostat turned down.
Previous research efforts have attempted to systematically categorize properties of occupancy. We are partial to a 2010 technical report by Thiago Teixeira that established five “low-level spatio-temporal properties” for describing the performance of occupancy sensors (presence, count, location, track, and identity), and have also drawn on prior methods to establish five states of presence (major motion, minor motion, hand motion, stillness, and absence).
![The five spatio-temporal properties investigated in the reviewed studies containing ad-hoc test methods. Most sensors or sensing systems that utilized IR technology are susceptible to the presence-detection loss characteristic of PIR sensors when a stimulus remains still. This compromises evaluation of other spatio-temporal properties beyond presence when presence detection is lost, as the ability to detect each higher-level property is dependent on successful detection of the lower-level property or properties. [Image credit: Illustration courtesy of Pacific Northwest National Laboratory (PNNL).]](https://img.ledsmagazine.com/files/base/ebm/leds/image/2020/10/2010LED_ins_pnnl.5f7b72abe5cd2.png?auto=format,compress&fit=max&q=45&h=257&height=257&w=250&width=250 "The five spatio-temporal properties investigated in the reviewed studies containing ad-hoc test methods. Most sensors or sensing systems that utilized IR technology are susceptible to the presence-detection loss characteristic of PIR sensors when a stimulus remains still. This compromises evaluation of other spatio-temporal properties beyond presence when presence detection is lost, as the ability to detect each higher-level property is dependent on successful detection of the lower-level property or properties. [Image credit: Illustration courtesy of Pacific Northwest National Laboratory (PNNL).]")
No well-described test method was identified that could evaluate all five temporal properties and stimulate all five states of presence. We found only two existing test methods that fully (or nearly so) describe how to characterize the ability of sensors to evaluate at least one of the spatio-temporal properties of occupancy: “Occupancy Sensors: Motion Sensors for Lighting Control” by Maniccia and Davis of the Lighting Research Center (LRC), and NEMA Standards Publication WD 7-2011 (R2016; Occupancy Motion Sensors Standard). Neither has been standardized or significantly adopted by the market, and both struggle to capture the performance impacts of obstructions (i.e., walls, barriers). Furthermore, both test methods have significant limitations with regard to their ability to characterize sensors that detect anything beyond motion, or to evaluate technologies other than PIR (such as ultrasonic, which relies on reflections off walls in a room to perform well).
In search of more-capable test procedures, we reviewed and analyzed journal articles and other reports that were published within the last 20 years and that included a test method for characterizing the performance of one or more types of occupancy sensors. A total of 33 studies containing ad-hoc test methods for characterizing occupancy sensor performance were examined. The test methods employed in each study are categorized according to which spatio-temporal properties they evaluated, and key or unique test conditions are noted. Given the energy-savings emphasis on the ability to better characterize presence, additional attention is paid to how that ability is evaluated. Specifically, test conditions for presence are categorized according to which of the five possible states of presence they simulate.
Collectively, the reviewed studies employed a total of six unique sensor technologies: visible light; IR; industrial, scientific, and medical band radio frequency; other radio frequency; mechanical; and air quality. Of the studies that evaluated sensor systems (i.e., systems comprising more than one device), some used arrays of identical or similar devices, while others used devices based on different sensing technologies. Twenty-seven of the studies evaluated a sensor or sensing system that used a single technology, with only six studies exploring combinations of more than one sensor technology. Infrared was the most frequently evaluated technology (13 studies).
Thirty-two studies contained unique test conditions that went beyond those found in the LRC and NEMA methods, both of which were designed to evaluate a single sensing device installed in an open room with no doors, furniture, or obstructions, and with a single human subject or single robotic arm to create stimuli. Sixteen of the studies evaluated a sensor system comprising more than one sensing device, eight used doorways or hallways, nine used an enclosed room, 10 used furniture obstructions, and 10 used other obstructions. Twenty of the studies contained test conditions that used multiple human subjects or robots at the same time to stimulate the sensor or sensing system.
We categorized all of the reviewed studies according to the five spatio-temporal properties that were evaluated, as shown in the graph above. The ability to detect only presence was evaluated most frequently (in nine studies). Count was evaluated in eight studies, location in five studies, track in nine studies, and identity in two studies. All studies of sensors or sensing systems that utilized IR technology were susceptible to the presence-detection loss characteristic (false negative) of PIR sensors when a stimulus remains still. In such instances, evaluation of other spatio-temporal properties beyond presence is, by definition, compromised when presence detection is lost, because the ability to detect each higher-level property is dependent on successful detection of the lower-level property or properties. The detection of spatio-temporal properties beyond presence was compromised by presence-detection loss in six of the 33 studies.
A path forward
Most of the reviewed literature contained ad-hoc test methods for characterizing a specific occupancy sensor technology. A wide variety of unique test conditions was identified. Given the growing number of technologies being developed and adapted — alone and in combination — for the detection of occupancy, and their increasing success in discerning spatio-temporal properties beyond presence, the development of test methods that are more capable, replicable, and fully described appears to be needed. So we suggest the following test method requirements for characterizing occupancy sensors targeting the indoor environment:
- Evaluate the five spatio-temporal properties — presence, count, location, track, and identity.
- Evaluate the five states of presence — major motion, minor motion, hand motion, stillness, and absence.
- Define test conditions that establish the five spatio-temporal properties of occupancy and the five states of presence using human subjects or using well-defined and well-calibrated means that do not require unique, one-of-a-kind equipment.
- Facilitate configuration flexibility for multi-device sensing systems.
- Utilize a room of a defined size.
- Utilize obstruction simulations that can be easily reproduced and do not require unique, one-of-a-kind equipment.
- Utilize sources of false stimuli based on known weaknesses.
- Define ground truth using a means that is clearly demonstrated to be of higher accuracy than the sensing system under test.
Although innovative occupancy sensors, some of them combining multiple sensing technologies, have come on the market with claims of improved performance compared to their predecessors, high deployment has yet to be achieved in practice. The lack of a well-described, technology-agnostic test method that yields reproducible results across different implementations has been a barrier to the commercial success of new occupancy sensors, as users and specifiers who have been disappointed with previous products are often unwilling to take a chance with new ones.
Currently, only an estimated 6–10% of commercial buildings in the US are equipped with occupancy sensors, which collectively save about 0.1 quads of energy a year — a fraction of what could be saved. (You can find details on DOE projected energy savings in a report on the December 2019 study.) Occupancy sensors that are specified and commissioned to be better suited to their application and meet user expectations could be expected to deliver greater energy savings and thereby reduce payback periods. Test methods that accurately identify higher-performing products and aid in specifying those that are well-suited to a given application are likely to accelerate deployment and the realization of energy savings, by removing the barrier of costly large-scale trial-and-error experimentation.
For more details, you can download the full report as a PDF.
REFERENCES
1. M. Javad Akhlaghinia et al., “Occupant behaviour prediction in ambient intelligence computing environment,” J. Uncertain Syst., Vol. 2, No. 2, 85–100 (2008).
2. I. Bastian Arief Ang et al., “Human Occupancy Recognition with Multivariate Ambient Sensors,” 2016 IEEE PerCom Workshops, doi:10.1109/percomw.2016.7457116 (2016).
3. A. Beltran et al., “ThermoSense: Occupancy Thermal Based Sensing for HVAC Control,” Proc. Build. Sys. 13, doi:10.1145/2528282.2528301 (2013).
4. D. Caicedo and A. Pandharipande, “Ultrasonic array sensor for indoor presence detection,” 2012 Proc. EUSIPCO, Bucharest, 175–179 (2012).
5. Z. Chen et al., “Building Occupancy Estimation with Environmental Sensors via CDBLSTM,” IEEE Trans. Ind. Electron., Vol. 64, No. 12, 9549–9559, doi:10.1109/tie.2017. 2711530 (2017).
6. R. Cucchiara, “T_Park: Ambient Intelligence for Security in Public Parks,” IEE Seminar Intell. Build. Environ., doi:10.1049/ic:20050242 (2005).
7. S. Depatla and Y. Mostofi, “Passive Crowd Speed Estimation and Head Counting Using WiFi,” 2018 SECON, doi:10.1109/sahcn.2018.8397119 (2018).
8. U. Gopinathan et al., “Coded Apertures for Efficient Pyroelectric Motion Tracking,” Opt. Express, Vol. 11, No. 18, 2142–2152, doi:10.1364/oe.11.002142 (2003).
9. K. Hammoudi et al., “Developing A Vision-Based Adaptive Parking Space Management System,” Int. J. Sens. Wireless Comm. Control, Vol. 6, No. 3, 192–200, doi:10.2174/2210327906666160518120212 (2016).
10. Q. Hao et al., “Human Tracking With Wireless Distributed Pyroelectric Sensors,” IEEE Sens. J., Vol. 6, No. 6, 1683–1696, doi:10.1109/jsen.2006.884562 (2006).
11. T.W. Hnat et al., “Doorjamb: Unobtrusive Room-level Tracking of People in Homes using Doorway Sensors,” Proc. SenSys 12, doi:10.1145/2426656.2426687 (2012).
12. L. Jia and R.J. Radke, “Using Time-of-Flight Measurements for Privacy-Preserving Tracking in a Smart Room,” IEEE Trans. Ind. Informatics, Vol. 10, No. 1, 689–696, doi:10.1109/tii.2013.2251892 (2014).
13. A. Kamthe et al., “SCOPES: Smart Cameras Object Position Estimation System,” Lecture Notes in Comp. Sci. Wireless Sens. Networks, 279–295, doi:10.1007/978-3-642-00224-3_18 (2009).
14. T. Labeodan et al., “Occupancy Measurement in Commercial Office Buildings for Demand-Driven Control Applications—A Survey and Detection System Evaluation,” Energy Build., Vol. 93, 303–314., doi:10.1016/j.enbuild.2015.02.028 (2015).
15. K. Lorincz and M. Welsh, “MoteTrack: A Robust, Decentralized Approach to RF-Based Location Tracking,” Location- and Context- Awareness Lecture Notes in Comp. Sci., 63–82, doi:10.1007/11426646_7 (2005).
16. T. Lowes, “High Efficacy, Multi-Functional Solid-State Lighting Platform,” US Department of Energy Office of Science and Technology Technical Report, doi:10.2172/1434906 (2018).
17. C. Luppe and A. Shabani, “Towards Reliable Intelligent Occupancy Detection for Smart Building Applications,” 2017 IEEE CCECE, doi:10.1109/ccece.2017.7946831 (2017).
18. S. Pan et. al., “BOES: Building Occupancy Estimation system using Sparse Ambient Vibration Monitoring,” SPIE Smart Struct. Mater. Nondestruct. Eval. Health. Monit. Diagn., 90611O–90611O (2014).
19. C. Papatsimpa and J.p.m.g. Linnartz, “Propagating Sensor Uncertainty to Better Infer Office Occupancy in Smart Building Control,” Energy Build., Vol. 179, 73–82, doi:10.1016/j.enbuild.2018.08.039 (2018).
20. S. Petersen et al., “Establishing an Image-Based Ground Truth for Validation of Sensor Data-Based Room Occupancy Detection,” Energy Build., Vol. 130, 787–793, doi:10.1016/j.enbuild.2016.09.009 (2016).
21. A. Prati et al., “An Integrated Multi-Modal Sensor Network for Video Surveillance,” Proc. VSSN 05, doi:10.1145/1099396.1099415 (2005).
22. O. Shih and A. Rowe, “Occupancy Estimation Using Ultrasonic Chirps,” Proc. ICCPS 15, doi:10.1145/2735960.2735969 (2015).
23. S. Tao et al., “Person Authentication and Activities Analysis in an Office Environment Using a Sensor Network,” Comm. Comp. Info. Sci. Constructing Ambient Intell., 119–127, doi:10.1007/978-3-642-31479-7_19 (2012).
24. S.P. Tarzia et al., “Sonar-Based Measurement of User Presence and Attention,” Proc. Ubicomp 09, doi:10.1145/1620545.1620559 (2009).
25. K. Weekly et al., “Low-Cost Coarse Airborne Particulate Matter Sensing for Indoor Occupancy Detection,” 2013 IEEE CASE, doi:10.1109/coase.2013.6653970 (2013).
26. A. Whitworth et al., “Unambiguous Determination of Oscillation Frequency for Multiple Objects Using Quadrature Doppler Radar,” 2018 APMC, doi:10.23919/apmc.2018.8617122 (2018).
27. Yang et al., “Counting People in Crowds with a Real-Time Network of Simple Image Sensors,” Proc. 9th IEEE Int. Conf. Comput. Vis., doi:10.1109/iccv.2003.1238325 (2003).
28 Y. Yang et al., “CeilingSee: Device-Free Occupancy Inference through Lighting Infrastructure Based LED Sensing,” 2017 IEEE PerCom, doi:10.1109/percom.2017.7917871 (2017).
29. E. Yavari et al., “Doppler Radar Sensor for Occupancy Monitoring,” 2013 IEEE WiSNet, doi:10.1109/wisnet.2013.6488651 (2013).
30. E. Yavari et al., “Occupancy Detection Using Radar Noise Floor,” 2016 IEEE/ACES ICWITS and ACES, doi:10.1109/ropaces.2016.7465363 (2016).
31. T. Yokoishi et al., “Room Occupancy Determination with Particle Filtering of Networked Pyroelectric Infrared (PIR) Sensor Data,” 2012 IEEE Sens., doi:10.1109/icsens.2012.6411114 (2012).
32. P. Zappi et al., “Enhancing the Spatial Resolution of Presence Detection in a PIR Based Wireless Surveillance Network,” 2007 IEEE Conf. Adv. Video Signal Based Surveill., doi:10.1109/avss.2007.4425326 (2007).
33. X. Zhang et al., “Water Filling: Unsupervised People Counting via Vertical Kinect Sensor,” 2012 IEEE 9th Int. Conf. Adv. Video Signal Based Surveill., doi:10.1109/avss.2012.82 (2012).
Get to know our experts
BENJAMIN FEAGIN, JR. and MICHAEL POPLAWSKI work at Pacific Northwest National Laboratory, where their research efforts are primarily supported by the US Department of Energy Lighting R&D Program and focus on characterizing various aspects of emerging lighting technology.
For up-to-the-minute LED and SSL updates, why not follow us on Twitter? You’ll find curated content and commentary, as well as information on industry events, webcasts, and surveys on our LinkedIn Company Page and our Facebook page.
