Spatial navigation has long been considered as a window into cognitive function. What mental models do animals have of their surroundings and how can they be used? In the lab, we recently discovered a heading direction network in the hindbrain of the larval zebrafish. This acts as a “compass”, pointing in the same direction in absolute (allocentric) space regardless of how the fish orients itself. I will review the activity of the network, and how it is affected by both the animal’s movement and external sensory cues. The small size of this vertebrate allows us to incorporate connectomic studies that probe the structure of the network and link it to is function, and seek what we may call a mechanistic understanding of a network that underlies cognitive behavior. Finally, I will highlight architectural similarities with the heading direction networks that have been identified in insects, and possible homologies with mammals, and argue that comparative approaches have a lot to teach us about neurobiology.