Hebbian plasticity relies on the correlation of pre- and post-synaptic activity for the potentiation of synaptic efficacies, and is thought to be essential for learning and memory formation. However, recent experiments suggest post-synaptic firing might not be required for plasticity to take place. Additionally, classical Hebbian models restrict plasticity to a learning phase and assume fixed synaptic efficacies during memory retrieval - an assumption not supported by experimental observation. We propose a minimal, biologically plausible model that decouples plasticity from post-synaptic activity, and optimize its parameters to solve a simple, continual associative memory task. Particularly, during a trial, pairs of one-hot stimuli are presented for acquisition, and the model is expected to retrieve both when one of them is shown. The model outperforms equivalent models with Hebbian plasticity and Recurrent Neural Networks, even at acquiring and retrieving pairs of stimuli not shown during training, an example of compositional generalization. Taken together, our results suggest that decoupling plasticity and post-synaptic activity could be essential for fast, flexible and biologically plausible learning.