Modern time-of-flight (TOF) cameras have improved to the point that with their current level of performance they are already being highly sought after in industrial, scientific, and commercial settings. Despite recent advances, TOF cameras still suffer from relatively low depth resolution, poor outdoor performance, low lateral resolution and a high cost of development. To date the most ubiquitous TOF systems have been on chip solutions such as photon mixing devices (PMD). Currently these devices only operate in the visible band and face significant challenges in improving illumination power, and modulation frequency. Here we propose a new approach to TOF imaging using an open architecture (OA) design where the demodulation and signal collection elements of the system have been operationally and spatially isolated. The design relies heavily on a novel stepped quantum well (SQW) large area modulator. The SQW is placed in the beam path between the collection optics and the imager (camera sensor). An open architecture approach allows for a modular TOF system where the imager can be freely chosen to match any application specific needs. Decoupling of the imager and demodulation stage of the system allows for a significantly higher modulation frequency and lateral resolution than what can be found in standard PMD devices. Additionally, the development cost of such a TOF system is significantly reduced. By analyzing the energy expenditure per bit, we show that our approach is fundamentally very efficient. We compute the energy per bit of our current short wavelength IR OA-TOF system to be 28 nJ up to 4 meters with a depth uncertainty below 1 percent of the imaging distance. We also show that a currently in development near IR version of the OA-TOF system can yield energy per bit values below 2 nJ, which is 10 times lower than the Kinect 2.