CPSBench will feature a mix of invited talks and paper presentations to inspire stimulating discussions.

Invited Talks



Reproducible Research: Failures, Successes, Challenges and (Re)Setting the Bar

Ian M. Mitchell, University of British Columbia, Vancouver, BC, Canada


Although reproducibility has been a fundamental tenet of science and engineering for hundreds of years, it has quietly but substantially deteriorated in the last few decades with the rise of first computational and now data driven research. This degradation has not been malicious (for the most part) but rather the gradual outcome of pressures and incentives rooted in academic tradition. Fortunately, the problem has been recognized and many efforts are underway to reverse the trend. In this talk I will outline some of those efforts and their successes. I will also discuss ongoing challenges, why we need to set reasonable expectations, and why we should continually revisit and raise them.

Presenter Bio

Ian M. Mitchell completed his doctoral work in engineering at Stanford University in 2002, spent a year as a postdoctoral researcher at the University of California at Berkeley, and is now Professor and Associate Head (undergraduate) in the Department of Computer Science at the University of British Columbia in Vancouver, Canada. He is the author of the Toolbox of Level Set Methods, the first publicly available high accuracy implementation of solvers for dynamic implicit surfaces and the time dependent Hamilton-Jacobi equation that works in arbitrary dimension. His research interests include development of algorithms and software for nonlinear differential equations, formal verification, control and planning in cyber-physical and robotic systems, assistive technology and reproducible research.


Invited talk

IoTMark™-BLE: How a Consortium Solved the IoT Benchmarking Problem

Peter Torelli, President of Embedded Microprocessor Benchmark Consortium (EEMBC)


This presentation explains the process by which the Embedded Microprocessor Benchmark Consortium (EEMBC) addressed the need for an IoT edge-node energy benchmark. During its multi-year development process, a diverse team of engineers had to comprehend IoT market demands to define a manageable solution space, and then select the wired and wireless protocols and define a balanced workload to represent that space in a single benchmark. This process was by no means trivial, and required a completely new testing methodology; the result is IoTMark-BLE, the first in a series of edge-node energy benchmarks offered by EEMBC.

EEMBC, an industry alliance, develops benchmarks to help system designers select the optimal processors and understand the performance and energy characteristics of their systems.

Presenter Bio

Peter has a long history in the semiconductor industry, with 20 years at Intel as a software architect, hardware architect, and integration lead. His most significant contributions include driving power convergence for the Xeon Phi product line, and developing power-aware CAD tools and methodologies that have been used at Intel for over a decade. In early 2014, Peter left Intel and entered the Portland-area startup scene, working in diverse areas such as IoT wearables and fashion technology. After presenting at the IoT Developer Conference in 2015, Peter took the job as Director of Software Engineering at EEMBC, and became President/CTO in 2017.

Accepted Papers

Evaluating Bluetooth Low Energy for IoT
Jonathan Fürst [1], Kaifei Chen [2], Hyung-Sin Kim [2], Philippe Bonnet [1]
[1] IT University of Copenhagen, Denmark
[2] UC Berkeley, CA, USA

Benchmarking Networked Control Systems
Sebastian Gallenmüller [1], Stephan Günther [1], Maurice Leclaire [1], Samuele Zoppi [1], Fabio Molinari [2], Richard Schöffauer [3], Wolfgang Kellerer [1], Georg Carle [1]
[1] Technical University of Munich, Germany
[2] Technische Universität Berlin, Germany
[3] Freie Universtität Berlin, Germany

Evaluating Low-Power Wireless Cyber-Physical Systems
Dominik Baumann [1], Fabian Mager [2], Harsoveet Singh [1], Marco Zimmerling [2], Sebastian Trimpe [1]
[1] MPI for Intelligent Systems, Germany
[2] TU Dresden, Germany

Toward Standard Non-line-of-sight Benchmarking of Ultra-wideband Radio-based Localization
Milad Heydariaan, Hessam Mohammadmoradi, Omprakash Gnawali
University of Houston, TX, USA

Towards Benchmark Optimization by Automated Equivalence Detection
Daniel Schemmel, Mirko Stoffers, René Glebke, Klaus Wehrle
RWTH Aachen University, Germany

[Invited paper] Moving Beyond Competitions: Extending D-Cube to Seamlessly Benchmark Low-Power Wireless Systems
Markus Schuss, Carlo Alberto Boano, Kay Römer
Graz University of Technology, Austria

[Invited paper] Towards a Benchmark for Low-power Wireless Networking
Carlo Alberto Boano [1], Simon Duquennoy [2], Anna Förster [3], Omprakash Gnawali [4], Romain Jacob [5], Hyung-Sin Kim [6], Olaf Landsiedel [7], Ramona Marfievici [8], Luca Mottola [2,9], Gian Pietro Picco [10], Xavier Vilajosana [11], Thomas Watteyne [12], Marco Zimmerling [13]
[1] Graz University of Technology, Austria
[2] RISE SICS, Sweden
[3] University of Bremen, Germany
[4] University of Houston, TX, USA
[5] ETH Zurich, Switzerland
[6] UC Berkeley, CA, USA
[7] Chalmers University of Technology, Sweden
[8] Cork Institute of Technology, Ireland
[9] Politecnico di Milano, Italy
[10] University of Trento, Italy
[11] Universitat Oberta de Catalunya, Catalonia, Spain
[12] INRIA, France and Analog Devices, USA
[13] TU Dresden, Germany

[Invited paper] SODA: 6TiSCH Open Data Action
Malisa Vucinic [1], Milica Pejanovic-Durisic [1], Thomas Watteyne [2]
[1] University of Montenegro
[2] INRIA, France and Analog Devices, USA