I found some wonderful quotes at Mathematical Quotations Server. Here is a selection:

The different branches of Arithmetic -- Ambition, Distraction, Uglification, and Derision.
- Lewis Carroll

A man is like a fraction whose numerator is what he is and whose denominator is what he thinks of himself. The larger the denominator the smaller the fraction.
- Lev Nikolaevich Tolstoy

I wish to God these calculations had been executed by steam.
- Charles Babbage

In science one tries to tell people, in such a way as to be understood by everyone, something that no one ever knew before. But in poetry, it's the exact opposite.
- Paul Dirac

We used to think that if we knew one, we knew two, because one and one are two. We are finding that we must learn a great deal more about 'and'.
Eddington, Sir Arthur (1882-1944)

A Mathematician is a machine for turning coffee into theorems.
- Paul Erdös