VisualBox. A Generic Data Integration and Visualization Tool

Abstract

The number of cellular Internet of Things (IoT) connections is expected to grow at a rate of 30% each year and is reaching into the billions by 2019. The world of IoT can be fragmented since data sources span a wide variety of protocols, API's, authentication methods and file formats. Data collection and processing can be complex and producing visualizations for value extraction can be a tedious task. Developers can often find themselves "reinventing the wheel" while building or using visualization libraries that present data in a specific way. VisualBox is a generic data integration and visualization tool, built as a Software as a Service (SaaS) running a front-end web application and a back-end cloud architecture using Amazon Web Services (AWS). VisualBox is built by first defining an abstract vision where problems are divided into smaller parts that are then progressively developed into a coherent system. An ecosystem of crowdsourced modules is used to allow developers to write software that handle data fetching and processing (called "integrations") and data visualizations (called "widgets"). These modules can be published to a registry where other users of the system can discover them for use of their own. Modules can be added to dashboards that produce data visualizations. Integration modules output generic data models that can be connected to widget modules. By making this separation, different widgets can be used to visualize different data models and allows for rapid value extraction, even for users without any technical or programming experience. Different approaches are explored while solving the problem of executing arbitrary user generated code and how to isolate the host system from code with malicious intent; on the client with the use of web-workers and on the back-end with the use of Docker containers. The container startup time is evaluated while using Amazon Elastic Container Service (Amazon ECS) with the Fargate launch type.