The phase diagram of cuprate high-temperature superconductors features an enigmatic pseudogap region that is characterized by a partial suppression of low-energy electronic excitations. In order to understand its microscopic nature, it is imperative to identify the full symmetries both prior to and within the pseudogap region. In this talk, I will describe our experimental results of symmetry properties on YBa2Cu3Oy across a wide temperature and doping range using a recently developed nonlinear optical rotational anisotropy technique. I will show evidence that spatial inversion and two-fold rotational symmetries are broken at the pseudogap temperature while mirror planes perpendicular to the Cu-O plane are absent at all surveyed temperatures for all doping levels including underdoped, optimal doped and overdoped ones. I will then discuss how this inversion-symmetry-broken order relates to charge order and superconductivity in YBa2Cu3Oy, and how our results compare to polarized neutron diffraction, Nernst effect and THz polarimetry data on YBa2Cu3Oy. Finally, I will show a remarkable similar order revealed in Sr2IrO4 system, a strong spin-orbit coupled analog of La2CuO4, indicating a robust connection between this inversion-symmetry-broken order and the pseudogap phenomenon even beyond cuprates.