Let N_{g,d} be the locus of curves of genus g admitting a degree d cover of an elliptic curve. For fixed g, it is conjectured that the classes of N_{g,d} on M_g are the Fourier coefficients of a cycle-valued quasi-modular form in d. A key difficulty is that these classes are often non-tautological, so lie outside the reach of many known techniques. Via the Torelli map, the conjecture can be moved to one on certain Noether-Lefschetz loci on A_g, where there is accesss to different tools. I will explain some evidence for these conjectures, gathered from results of many people, some of which are joint with François Greer and Naomi Sweeting.