Stack OverflowMaths in Programming - The Developer's guide
[+35] [10] Marko
[2010-10-03 21:46:28]
[ math programming-languages career-development computer-science ]

Possible Duplicate:
Useful math for programmers [1]

I'm a 22 year old ASP.NET/C#/Front-End developer with roughly 5.5 years experience. Unlike some (well probably most) of you, I didn't go to University to do a CS degree (*sigh*). The reasons behind that are, well... complicated, so I ended up teaching myself by reading books, articles, reading and posting on forums, experimenting, and so forth.

I always like to think I have a quite logical, problem-solving mind and it has definitely helped me over the years. I also like to think it's quite genetic since my dad was a very smart mathematician. Unfortunately I never really picked up much from him, but that's a different story.

After reading this question [2], I have to say that I agree with Nils' answer [3] and that a developer should first know the fundamentals of programming (statistics, computing, databases, operating systems, computer architecture, computer networks and discrete mathematics) to name a few.

I have already considered going to university, but I could probably not afford to do it full-time, and doing it part-time would mean working 8-5, then studying 5-9. Its also likely that it would affect my work performance.

But anyway, enough about me – let's get to the question. :)

Based on the above, what resources would you suggest that are specifically related to Mathematics, especially the kind that is often used in programming? (statistics, algorithms, etc.)

I'm very interested in learning, and have mostly been inspired by you, the Stack Overflow community.

All answers will be considered.


Wow, some excellent answers have come in and I'd like to thank everyone who has helped. I'm in the process of reading through all of them and seeing what the best place to start is. I am particularly interested in Boolean Algebra, Big-O notation, Calculus and basic Geometry to start with. One can only learn so much at once, so after getting myself more familiar with the topics above, I will likely move onto more complicated topics, especially since I will know exactly what I want/need to learn.

So once again, thanks to everyone for their great help, I will leave the question open in case others come with more information and I'll let the community vote up the best answers.

(2) I found many similar questions searching for "math programming", you may be able to find more helpful answers in other questions. - Greg Hewgill
(2) Yet another Big List of Foo (tm). The answer to every Big List of Foo is "It depends on the specifics of what you hope to do" because there just isn't one discipline called "programming". - dmckee
@Marko Ivanovski. If you're interested in basic geometry, you might want to try: Geometry by Lang and Gene Murrows. I haven't read it but it is by Lang (so high probability of good) and it's meant as an alternative to the atrocious coverage of geometry that is typical in highschool. - aaronasterling
Hi @Aaron, thank you for that and your excellent answer below. I went and ordered the Linear Algebra book last night even though it looks slightly more complicated than what I currently know. I will no doubt be visiting the folks over at for help :) - Marko
@Marko: MathOverflow is for graduate-level research questions. You may not get much sympathy there. - Robert Harvey
I was afraid of that @Robert, I once visited the website to find aliens having a conversation. It was weird. - Marko
(1) @Marko: If you want to convince people to reopen your question, use the comments. Stop adding reopen requests in the question. That's not what the question is for. - gnovice
Hi @gnovice I misunderstood your edit summary, I thought you meant use the question title to advertise re-opening. I'll remove it from the title now and leave it under the Possible Duplicate. Thanks for your help. - Marko
(1) @Marko, is the place to ask for help with the material that has been recommended. It's for undergraduate level math. - aaronasterling
You're a web developer; if you know how to add, subtract, multiply, and divide you will be fine. - Ed S.
(4) @Ed, I think you might find a lot of people will disagree - Marko
[+27] [2010-10-03 22:12:33] aaronasterling

I'm self educated in mathematics so I can talk about what helped me. I failed high school algebra twice (apparently, my strict no-homework policy didn't vibe to well with a school district that counts homework for at least 40% of the class) before I dropped out.

I should point out that while my list is not specifically geared for programming in any sense, it is all good to know and relates to it. It gives you the "I can understand anything" feeling. Want to know what the History Monoid is (relating to concurrent systems)? It's trivial if you know what a monoid and free monoid are. If not, you're screwed without a lot of hard work (after which, you'll know what a monoid and free monoid are ;)

I recommend that you learn linear algebra first. In saying that, I mean real mathematics, not the stuff that's focused on solving linear equations for practical applications but proof oriented mathematics. The only book I know of that approaches it from a beginners perspective and with the right attitude is Linear Algebra: An Introduction to Abstract Mathematics. [1] It's a very fun book and reads more like a modern algebra text then the sort of garbage introductions that focus on purely numerical concerns that have flooded the market place. They all have such fugly notation too. Don't ever trust a math book with glossy pages and pictures of people skiing. If it's yellow though, it's good.

after that, I would recommend calculus. My recommended introduction is Essential Calculus with Applications [2]. It's a little dover book (one of the good ones, they're hit or miss) and does a very good job of getting you up to speed on the concepts involved, e.g. it doesn't wave it's hands over what a limit is.

After that, check Vector Calculus, Linear Algebra and Differential Forms [3]. It is a very good book that is solid on the theory and deals with actual computations. It is the sole exception to the 'glossy paged math books suck' rule that I have ever seen. No pictures of people skiing though and the authors (husband and wife) actually explain what the uber-trippy picture on the cover is and how it's calculated.

Then to continue the calculus (getting optional at this point) I recommend Undergraduate Analysis [4] by Lang along with a schaums outlines (calculus and multivariate calculus) for plenty of concrete examples and techniques that Lang will assume known but which my earlier recommendations might skimp on.

After that you probably want to finish up the algebra. I learned my undergraduate level stuff from Introduction to Algebra [5]. This is the only book I've seen that actually explains that a vector space is just an embedding of a field in the endomorphism ring of an abelian group. So now you don't need to remember all of those axioms any more, they fall right out of that definition.

Then you'll have plenty of experience writing proofs and be comfortable with groups and point-set topology. You could either dig into some graph theory taking things in the discrete mathematics direction (I'm still looking for a good book on it) or you could (if you "fell into the math hole") continue with Introduction to Topological Manifolds [6]

If you're trying to learn math on your own, it's hard. I'd recommend that you find the Go club (game not language) near where you are because that's where a lot of mathematicians hang out and go is a great game to learn. Most of them are happy to help with problems if they see that you're working on it. When I was hitch hiking around out west one time, I was stuck on a problem and found the go club. When I got there, there were only two people and they were in a casual game so it was polite to talk. I asked them if either of them was a mathematician and they looked at me and exclaimed "We both are!". One of them was able to help me.

Also highly recommended after you finish the calculus stuff is Mathematics for 3D Game Programming and Computer Graphics [7]. This is the book with the example that helped me finally grok co-variant vs. contra-variant tensors. It didn't use that terminology but I happened to grok them enough to spot the example and it all fell into place. The point is that it's a very good book that leans much more toward the intuition than the formalism.


(3) Impressive bed time reading - Alkaline
(1) @Alkaline. not bedtime reading! they are very hard books but well worth the effort. - aaronasterling
(1) I know. "bed time reading" is an euphemism here. - Alkaline
I wish I found this answer earlier. Thank you sir. - SimpleButPerfect
-1, "a vector space is just an embedding of a field in the endomorphism ring of an abelian group". I don't see how this sort of opaque language is in any way helpful to developers looking to improve their maths abilities. The answer also heads off on a tangent about learning the foundations of mathematics, but the OP specifically asks about mathematics used in programming. I only felt the need to downvote/comment becuase I am also a programmer looking to improve my maths ability and this answer epitomises why intermediate/advanced maths can seem such an obsure field. - Ashley Henderson
[+10] [2010-10-03 21:59:03] ajnatural

Discrete Mathematics [1]

Boolean Algebra [2]

Modular Arithmetic [3]

Complexity Theory [4]

Of course there are many others but these would be the ones that apply to most situations and programming languages.

Lastly if you wish to get some practice at more mathematically oriented programming I recommend projecteuler [5]


(2) +1 for mentioning Project Euler - jon3laze
You will find a lot of math oriented problems at SPOJ (+ the community of problems solvers). - Łukasz Kuszner
[+3] [2010-10-03 22:16:42] Ludovico Fischer

The notes on this introductory MIT course [1] are extremely complete, it is almost like a textbook.

For a bit of history, there is this book called ' The Essential Turing [2]'. It contains a number of papers from one of the founders of computer science (and a great mathematician). The papers are actually surprisingly accessible.


[+3] [2010-10-03 22:22:31] Crashworks

The kind of math I use every day is mostly basic linear algebra and geometry — vectors, trigonometry, polynomial interpolation. I have to write an FIR polyphase interpolator this week. But this is all very domain-specific. In hiring we test all applicants on basic matrix math, but basically you can only flunk that if you forget what a dot product is.

The math curriculum where I went to school was:

  1. Calculus (not very useful in practice; I think I have to integrate or differentiate something about once every six months)
  2. Number theory [1] (very useful for dealing with binary moduli and numerical algorithms like RSA)
  3. Combinatorics, a follow-up to number theory. Useful in that it's the basis for a lot of stats, and a lot of algorithm proofs fall out of it.
  4. Computability theory [2], the stuff Alan Turing invented. The origins of big-O and P/NP comlpeteness, so useful because it leads to the biggie:
  5. Algorithm analysis [3], the bread and butter of a computer scientist. I use the stuff from this class so often I'd be f***ed if I hadn't taken it.

Where we had good textbooks I linked to them above.


Algorithmic complexity is pretty important, but on the problems most of us face (i.e. not google-sized datasets) cache interactions (memory locality, false sharing) and lock contention dominate the difference between an O(x^3/2) and O(x ln x) algorithm. - Ben Voigt
That's true. I consider memory locality and concurrency to be part of algorithm design too. R-trees are an example of one useful thing I picked up from that class where the advantage in locality outweighs the big O. - Crashworks
[+3] [2010-10-03 23:41:38] Alkaline

There's mathematical knowledge and then there's mathematics as a mean to develop abstract thinking. Topics like boolean algebra, mod operator, rounding errors are all useful knowledge but they can easily be learnt from the web. Abstract thinking and problem solving skills on the other hand requires lots of mind training. Very few software engineers ever use most of the physics or mathematics subjects they learnt at uni but the rigorous solving skills acquired is what counts.

[+3] [2010-10-04 00:43:14] Kirk Broadhurst

There's a lot of mathematics that's specific to certain areas within software, but the more generally useful areas that I've come across are:

  • logic and predicate logic: provides good understanding of basic operators, truth tables, and logical axioms. More thorough than straight boolean algebra because it also includes the concepts of 'any/all' or 'exists/not exists'.
  • set, rings and fields: includes abstract structures, the concepts of isomorphism & polymorphism, inheritance. This also forms the basis for most cryptography work, in conjunction with large prime numbers, and some good set logic (relates to SQL)
  • sums of infinite series / calculus, which leads to big O notation and algorithmic complexity.

I believe any sort of abstract mathematics is going to be useful, because it teaches you to think about abstract concepts - which is what the fun and challenging part of software engineering is all about.

I have a mathematics major instead of a CS major. I missed out on some useful knowledge but I believe I gained a lot more than I lost.

[+3] [2010-10-04 03:45:56] SK9

From a pro-mathematician: At least learn some combinatorics and linear algebra. I find the former helps me to express problems efficiently and organise my thoughts. It can also help to find an effective way to solve the problem.

To be honest anything that disciplines you to think and argue will do just as well. Even the ability to count is an underrated but essential skill.

[+2] [2010-10-03 22:07:42] ChrisW

Based on the above, what resources would you suggest that are specifically related to Mathematics, especially the kind that is often used in programming? (statistics, algorithms, etc.).

I haven't used much maths in programming beyond big-O notation, boolean logic, and arithmetic: not because I didn't know any (my university degreee was in maths) but because the programming that people have asked of me didn't need it.

Only very occasionally have I used other maths-related things: huffman encoding, queueing theory, and a bit of signal analysis (for a signal-processing application).

It's not the programming that requires maths: it might be the problem domain (e.g., I don't know, maybe you might need to know some Financial mathematics if you were programming 'quant'-type software; or if you're doing signal processing, or statistics, or combinatorics).

So, to not answer your question, I think it's the other things that are more important: for example operating systems, databases, and networks.

Math does come up in some of those other topics though -- database query optimization has a lot of math and queuing theory is pretty important in networking. - Ben Voigt
(1) @Ben Voigt - "database query optimization has a lot of math": only when you're implementing a database, or even when you're just using one? "Queuing theory is pretty important in networking" is true, but designing network components and protocols and topologies in't the same thing as programming: it's, again, a specific problem domain (not programming) that requires maths. - ChrisW
Queuing theory isn't just (or even mostly) important in protocol design. It's answers to questions like "what is the expected latency of a web service if it averages 85% CPU utilization" and "how deep does the accept queue on your socket need to be in order to average only one rejected connection per hour". For databases, forms of normalization are just as much applicable to the application architect as to the people writing the query engine. - Ben Voigt
Almost forgot lock contention. Any sort of parallel processing involving locks is all about queuing theory. - Ben Voigt
[+1] [2010-10-03 22:13:17] user465411

That's an interesting question; and I would say that applies to most of us programmers after being the industry long enough - a refresher of the basics of math and stats is really helpful.

I found the the MIT algo course very useful and keep referring to it now and then:

In general, courses on discrete math and fundamentals of statistics. I found this lecture series for stats - The UC Berkley Statistics 20 - but haven't come across anything good for discrete math though...

[0] [2010-10-14 09:06:58] Umakanta.Swain

There are many source and many software which will help you get more knowledge about mathematics. There are many Open Universities which were providing lots of distance learning course to understand and find out a certificate for yourself. Like In India - IGNOU [1] which is one of best university to start your career too.

If You will do a mathematics course you can finish it simultaneously you can do your work without any loss.

You will get study materials and the process to start your career and many more new things too.

You will get practicals/projects and case studies and many more come up with it.

So after finishing up you can start your career for doctorate too.

I think It ll be better go for a distance course which will make a practice of learning something with a certification and Degree too.

Thanks for going through it. If any comments please add without any hesitation