How supermassive black holes form and grow is still one of the long-standing questions in astronomy. Supermassive black holes of up to 10^9 solar masses already existed when the Universe was less than ~1 Gyr old. To reach this mass in such a short time, they should have started as seed intermediate-mass black holes (IMBHs) of 100-10^6 solar masses and grow very fast via accretion and mergers. Such IMBHs are the missing link between stellar and supermassive black holes and they should be present in the nucleus of low-mass galaxies and in the halos of large galaxies, e.g. in the form of ultraluminous X-ray sources (ULXs). In this talk I will present observational evidence that a population of IMBHs in dwarf galaxies similar to those seed black holes populating the early Universe exists. With the use of large, deep, multiwavelength surveys like COSMOS, these sources are detected up to z~4. I will also show that, in the local Universe, IMBHs can be additionally detected as extreme ULXs with powerful radio emission in galaxies having undergone a minor merger event. The later supermassive black hole growth can be explained by the accretion of gas coming from galactic scales. At kpc scales, this gas can be transported to the nuclear regions through galaxy mergers, which can trigger active galactic nuclei (AGN) activity and can be detected in the form of binary AGN. At scales of a few 100 pc, dust lanes can be the channels of inflow of material from the outer parts of the galaxy to the nuclear region, as evidenced by the finding that the spiral structure of the nuclear cold molecular gas traced by ALMA observations correlates with that of the dust. The nuclear dust lanes are in addition able to obscure the nucleus of low-luminosity AGN and to explain the collimation of the ionised gas, which questions the role of the dusty torus proposed by the Unified Model of AGN.