At CUNY we get many students doing this every year, mostly students who are planning graduate school in a field which requires more math than they took when they first got their bachelors but a few who took the coursework a long time ago or didn't do well in it.  These are some of our best students and we are happy to recommend them to graduate programs based on their coursewok even though their Bachelors is from another institution.

If you don't feel you learned enough in your undergrad courses, I'd say just retake the courses.  It has been a few years since you finished your BA and you should be able to explain to any graduate school why your original undergrad grades don't reflect well.  Make sure that you cultivate a set of professors who will make good references as well.  Know also that the Budapest Semesters in Math program will take a recent graduate and you could retake most upper level classes there, I went after my bachelors and learned some great math.