https://code.kx.com/v2/wp/signal-processing/
This whitepaper will explore how statistical signal-processing operations (those which assume that signals are stochastic), can be implemented natively within q to remove noise, extract useful information, and quickly identify anomalies. This integration allows for q/kdb+ to be used as a single platform for the capture, processing, analysis and storage of large volumes of sensor data.
******
That paragraph struck me as a bit dead on.
I was only a technician, not an engineer, so the computations were over my head but the q integer data sampling to "skim" for anomalies detection algorithm is a head turner. This part of the conclusion is probably more than meets the eye.
The resulting data had a clear reduction in noise and revealed the longer-term trends present in the signal. It has also been shown that a simple and robust anomaly-detection routine can be implemented in q, allowing anomalies to be detected in large datasets very rapidly. These smoothing and anomaly detection routines could easily be integrated as real-time operations with other data capture and processing routines.Jack me up, Scotty!