Scalable change and anomaly detection in cross-correlated data
Metadata
Show metadataAppears in the following Collection
- Matematisk institutt [3781]
Abstract
Both in science and industry, the sizes of data sets are growing. It is not uncommon to encounter sets containing millions or even billions of measurements. Without appropriate tools for turning such enormous amounts of data into insight, however, the data’s value is severely limited. Apart from consisting of many measurements, a common feature of big data sets is that some properties of the data change over time. Determining whether and when changes have taken place is important in many scientific problems. For example: Is the climate changing? Has the covid-19 reproduction number changed? Is the quality of manufactured cars stable? Moreover, monitoring changes in network traffic data can be used to detect cyber attacks. Therefore, in this thesis, I have studied statistical methods for detecting changes and estimating when they have occurred. My collaborators and I have constructed efficient computer programs both for retrospective analysis of large data sets as well as for real-time analysis of streaming data. We have also demonstrated that detecting changes in a stream of data from temperature sensors could have prevented a costly and dangerous overheating event in a ship motor.List of papers
Paper I: Tveten, M. (2019). Which principal components are most sensitive in the change detection problem? Stat, 8(e252). DOI: 10.1002/sta4.252. The article is included in the thesis. Also available at: https://doi.org/10.1002/sta4.252 |
Paper II: Tveten, M. and Glad, I. K. (2019). Online detection of sparse changes in high-dimensional data streams using tailored projections. Manuscript. Manuscript. To be published. The paper is not available in DUO awaiting publishing. Available on arXiv: 1908.02029 [stat.ME] |
Paper III: Hellton, K. H., Tveten, M., Stakkeland, M., Engebretsen, S., Haug, O. and Aldrin, M. (2020). Real-time prediction of propulsion motor overheating using machine learning. Submitted for publication. To be published. The paper is not available in DUO awaiting publishing. |
Paper IV: Tveten, M., Eckley, I. A. and Fearnhead, P. (2020). Scalable changepoint and anomaly detection in cross-correlated data with an application to condition monitoring. Invited to submit a revision to Annals of Applied Statistics. Invited to submit a revision to Annals of Applied Statistics. To be published. The paper is not available in DUO awaiting publishing. Available on arXiv: 2010.06937 [stat.ME] |