Context of the problem:

I have signal data which was recorded in a software system and which shows the runtime of multiple processes over time. In total there are more than 900 processes each having roughly 400 reading points. The data is stored in an m x n matrix with roughly 900 rows (one process per row) and 400 columns (reading points/runtime values).

The signal has been divided into five segments with two segments containing values between 50 and 60, and the three remaining segments containing values between 0 and 30. Intuitively, dividing the signal into five segments seems to be the ideal segmentation and guarantees that each segment has relative consistent inner behavior (in relation to the other segments, the data, etc.).

Coming to the question:

What are appropriate methods for dividing a data set with multiple processes into an optimal number of segments? For single and simple signals/processes the segmentation seems quite intuitive and easy, for multiple processes which have a lot of noise and different behaviors not.

My first approach for solving above problem was using variance. The idea is to start with an arbitrary number of segments with the segmentation lines being equally spaced to each other.

Now, if needed, each segmentation line can be moved with the goal of minimizing the variance of two neighboring segments that are affected by moving respective line. Doing this for all segments iteratively will minimize the variance of the segments. Ideally, this will lead to a segmentation with each segment having consistent inner behavior. However, the question remains if variance can be used for finding segments with consistent inner behavior.

The problem here is that using variance for segmenting a process requires a presegmentation. Using an algorithm which determines the right number of segments would be difficult, as having one data point per segment would give the best result for minimizing the variance of each segment (this brings us back to the maximization and minimization problems mentioned above). Also, the question remains if using variance for segmentation guarantees that each segment shows relatively consistent inner behavior.