How to get multiple TeamCity build agents running on one server.
First log in to the server where you want the agents to run then open TeamCity from a browser on that box.
Go to the Agents tab.
From the top right of the page choose Install Build Agents then MS Windows Installer.
When prompted choose to run the agent installer. You may have to be an administrator since this will be installing windows services.
Choose the directory where you want the agent configuration and working directories to live. I put them under the TeamCity home directory.
Take the defaults as you work your way through the installer.
The agent directory is being configured.
Here it is important that you choose a unique name and port. The directories should be consistent with your previous choices. You may need to change the server URL and port.
When this popup appears do not click OK yet.
Open an Explorer window and navigate the the launcher/conf directory under the build agent directory you configured above.
Edit wrapper.conf and change the name of the ntservice to match your agent name as appropriate. This is important because each agent runs under a different service and they must have unique names, otherwise only one will connect at a time.
Now you can click OK on the popup.
And choose the defaults the rest of the way through the installer.
If you open Services
You should see the service you named.
The new agent should connect and be visible from the TeamCity Agents page within a few moments.
Repeat the steps to add additional agents.