The early chemical evolution of the Galaxy and the Universe is vital to our understanding of a host of astrophysical phenomena. Since the oldest, most metal-poor stars are relics from the high-redshift universe, they probe the chemical and dynamical conditions of a time when large galaxies first began to assemble. Recent works have shown that the ultra-faint dwarf galaxies (with L<10^5Lsun) contain a relatively large fraction of extremely metal-poor stars and are devoid of solar-metallicity stars. This reflects a rather short or truncated star formation history similar to what one would expect to occur in a first/early galaxy. The chemical signatures of these dwarfs furthermore support the concept that small systems analogous to the surviving ultra-faint galaxies were the building blocks of the Milky Way's low-metallicity halo. This opens a new window for studying galaxy formation with the means of stellar chemical analyses. I will also include some information on our newly discovered most iron-poor star.