Now that I'm using NuGet to add/update my required references, is it possible to only add packages.config to source control and have the .dlls added at build time?
Normally, like described in this related question, I store third party dependencies in a folder under the solution root and check them in to source control.
We're currently not using any custom build tools or scripts: just VisualStudio's default build with the .sln file.
Yes you can! http://docs.nuget.org/docs/workflows/using-nuget-without-committing-packages