$TextStyle = {FontFamily -> "CMUSerif-Roman", FontSlant -> "Plain", FontSize -> 20}
The axis numbering of any plots should now appear in CM. For math mode-like variables, it is required to wrap the variables in a StyleForm. For example, the plot at the top of this article was constructed using the following command:
Plot[x^2, {x, 0, 1},PlotLabel -> StyleForm[x^y, FontFamily -> "CMUSerif-Italic", FontSlant -> "Plain", FontSize -> 20]]
There are a couple issues worth mentioning. First off is that Mathematica will only use the fonts for alphanumerics: mathematical symbols will still be in Mathematica's own font. This can be worked around by manually editing the .eps file and replacing all the instances of "/Mathematica1" with "/CMUSerif-Roman". This problem also exists for greek letters, however the solution is more difficult. The problem is Mathematica has a special greek font, where, for example, "w" is mapped to omega. As a result, the .eps file will have a "w" where the omega should be. Unfortunately, I have not figured out how to specify unicode indices in the .eps format, so the only solution is to use the original CM fonts and look up the mapping. Finally, the CMU fonts can be used in other applications as well. I have used the fonts in Keynote to simulate math mode. This allowed me to quickly do math mode for simple, in text, equations, and LaTeX Equation Editor for more complicated equations.