Optical coherence tomography (OCT) elastography (OCTE) has the potential to be an important diagnostic tool for pathologies including coronary artery disease, osteoarthritis, malignancies, and even dental caries. Many groups have performed OCTE, including our own, using a wide range of approaches. However, we will demonstrate current OCTE approaches are not scalable to real-time, in vivo imaging. As will be discussed, among the most important reasons is current designs focus on the system and not the target. Specifically, tissue dynamic responses are not accounted, with examples being the tissue strain response time, preload variability, and conditioning variability. Tissue dynamic responses, and to a lesser degree static tissue properties, prevent accurate video rate modulus assessments for current embodiments. Accounting for them is the focus of this paper. A top-down approach will be presented to overcome these challenges to real time in vivo tissue characterization. Discussed first is an example clinical scenario where OTCE would be of substantial relevance, the prevention of acute myocardial infarction or heart attacks. Then the principles behind OCTE are examined. Next, constrains on in vivo application of current OCTE are evaluated, focusing on dynamic tissue responses. An example is the tissue strain response, where it takes about 20 msec after a stress is applied to reach plateau. This response delay is not an issue at slow acquisition rates, as most current OCTE approaches are preformed, but it is for video rate OCTE. Since at video rate each frame is only 30 msec, for essentially all current approaches this means the strain for a given stress is changing constantly during the B-scan. Therefore the modulus can’t be accurately assessed. This serious issue is an even greater problem for pulsed techniques as it means the strain/modulus for a given stress (at a location) is unpredictably changing over a B-scan. The paper concludes by introducing a novel video rate approach to overcome these challenges.