Cycle for model-free reinforcement learning. The agent, consisting of a neural network, proposes an action which then acts on the environment. The environment’s response to the action produces a reward, and once a terminal state is reached, the reward is used to train the agent.
An example application of end-to-end design in order to create a shaken lattice gyroscope for rotational sensing. Here, intuition from the traditional Mach-Zehnder interferometer sequence is thrown away in favor of direct quantum design, allowing one to become five times more sensitive than a two-path interferometer.
The complexities and intricacies of quantum mechanics can lead to many complicated interactions, even in highly controlled ultracold experiments. Therefore, protocols developed to accomplish a certain quantum design task typically are limited to the simplest systems such that human intuition from experts can be utilized in developing solutions. In contrast, the emerging computer science field of machine learning allows a computer to build up insight about a complex problem and propose innovative solutions that are beyond the intuition of the world’s foremost experts, e.g., A.I. chess programs beating grandmaster players using revolutionary moves. Our research group applies machine learning to complicated quantum design problems such as shaken lattice interferometry, dynamically decoupling sequences in solid state materials, and quantum circuit design. We use model-free reinforcement learning, specifically deep Q-learning, that undergoes the following learning cycle. An actor composing of a neural network proposes an action that is applied to an environment. The proposed action causes a specific response of the environment which produces a reward. When the environment reaches a terminal state, the reward is used to train the neural network for a specific design task, and the cycle iterates to reach higher rewards. Since we use model-free learning, the agent does not know whether it is proposing actions to play Tic-Tac-Toe or to teach a robot to walk along a path, and so the same machine learning code can be used from a variety of problems with only slight modifications. Moreover, our group collaborates with the Anderson lab and Sun lab at JILA with the goal of developing closed-loop learning experiments in which the agent trains off of an actual experiment and proposes certain laser sequences in real time. We have explored end-to-end design in which all intuition from, for example, a Mach Zehnder interferometry sequence is disregarded in favor of direct quantum state design from machine learning. Our group is also interested in other machine learning techniques that may be utilized in stochastic simulations, as this relates to Monte-Carlo wavefunction simulations of open quantum systems. Machine learning protocols for shaken lattice interferometry is our group’s contribution to NASA’s Quantum Pathways Institute to develop space-bound quantum-based technologies.