In this paper we study the behavior of a two level cache hierarchy for multiprocessors, where the processors share a common second level cache. Cache performance is studied as a function of set associativity of primary and second level caches, using trace driven simulations. The simulation uses numerical benchmarks compiled using a parallelizing and vectorizing compiler. Simulation study shows that a replacement scheme based on least recently used blocks gives better performance than a replacement scheme based on multilevel inclusion. Invalidations to ensure multilevel inclusion in the two level cache hierarchy do not significantly affect cache performance and can be further reduced by increasing set associativity of second level cache. Furthermore, the study shows that coherency traffic and cache interference are sensitive to set associativity of the cache. In spite of reduction in coherency traffic, improvement in performance with set associativity is largely due to increase in hit ratio of primary and second level caches.