Modeling file-system input traces via a two-level arrival process

A method for analyzing, modeling and simulating a two-level arrival-counting process is presented. This method is particularly appropriate when the number of independent processes is large. The initial motivation for this method was the need to analyze and represent computer file system trace data that involves activity on some 8,000 files. The method is also applicable to network trace data characterizing communication patterns between pairs of computers. Cluster analysis with a novel stopping rule is used to decompose the arrival process into groups. The resulting clusters can be characterized using the time between clusters, the time between arrivals within clusters, and the size of each cluster. Each of these three components is then analyzed as a univariate problem. The effectiveness of this method is measured by comparing the output of a simulation driven by the original trace data to the output of the same simulation driven by the input model.