BigDataFr recommends: Feature Engineering in IoT Age – How to deal with IoT data and create features for machine learning?

Introduction

[…] If you ask any experienced analytics or data science professional, what differentiates a good model from a bad model – chances are that you will hear a uniform answer. Whether you call it “characteristics generation” or “variable generation” (as it was known traditionally) or “feature engineering” – the importance of this step is unanimously agreed in the data science / analytics world.

This step involves creating a large and diverse set of derived variables from the base data. The richer the set of variables that are generated, the better will be your models. Most of our time and coding efforts are usually spent in the area of feature engineering. Therefore, understanding feature engineering for specific data sources is a key success factors for us. Unfortunately, most analytics courses and text books do not cover this aspect in great detail. This article is a humble effort in that direction.

Table of Contents

1. The IoT Revolution
2. Nature of IoT or sensor data
3. Aggregation of data for feature engineering
   • Usage at atomic level
   • Usage at aggregated level
   • Selecting Optimal time window for aggregating sensor data
   • Types of Agreegation
   • Missing value treatment
4. Feature generation
   • Basic Features
   • Features based on relationships
   • Features based on higher order statistics
   • Features based on Outlier detection
5. Further readings

Read more
By Sandhya Kuruganti and Hindol Basu
Source: analyticsvidhya.com