The effective dynamics of a colloidal particle immersed in a complex medium at equilibrium is usually described in terms of a linear overdamped Langevin equation, possibly with memory. However, numerical simulations and experiments have shown that this linear model fails, suggesting that the intrinsic dynamics of the probe is actually non-linear. Focussing on the case in which the medium is described by a fluctuating and correlated Gaussian field linearly coupled to the tracer, we derive such a dynamics and discuss its various consequences, including those on the stochastic thermodynamics of a driven particle. When the field is generated by the particle itself, with negligible fluctuations, the resulting self-chemotactic dynamics turns out to display anomalous diffusion and run-and-tumble motion in low spatial dimension, which we characterise analytically.