I've listed some alternatives for connection management below, in order of most to least recommended.
Increase the connections allowed on the server
The total incoming connection limit on the server is determined by the lesser of the limits imposed by the operating system or maxIncomingConnections (aka maxConns in MongoDB 2.4 and earlier).
Typically Linux distributions limit file descriptors per process to 1024, of which MongoDB will use 80% for incoming connections (leaving about 819 available connections).
You can check current and available connections in the mongo shell via:
db.serverStatus().connections
For production systems it is typical to adjust the ulimit settings on Linux to allow more concurrent connections. For more best practices, I would recommend reviewing the Production Notes in the MongoDB manual.
Provide an API
If you are managing a shared server with resource limits, it is common to provide your own API rather than direct database access. This approach gives you an extra layer of abstraction so you can manage resource usage and server deployment independent of the client configuration. For example, you could move your database server or reconfigure from a standalone to a replica set, and the clients would not have to be aware of this. You can also manage custom resource limits (such as connections per client) via your API, based on the credentials the client uses to connect.
Reduce the connection pool size in the clients
MongoDB (as at 2.6) doesn't have an option to limit the connections per client. Normally client limits would be imposed via the driver (i.e. setting the connection pool size). For example, in the Java driver the MongoClient default maximum pool size is 100.
You've already suggested this isn't a desirable option as you don't want the clients to mess with the connection limits, but if you are going to impose a server side limit it would still be reasonable to have them set the pool size appropriately. Otherwise their applications will get frequent exceptions as you kill off excess connections.
Monitor client operations
If adjusting limits on the client or server isn't an option, an alternative to consider is implementing a script to count concurrent client connections (by IP) via db.currentOp() and kill excess connections via db.killOp(). You'd have to be very careful to only kill client requests. The killOp() command is a superuser command that will let you kill internal database threads as well (which can lead to unpredictable outcomes).
NOTE: This approach will be unsuccessful if your clients are connecting via a shared gateway (i.e. where the source IP does not uniquely identify a client).
Best Answer
Find out how many connections are there to the database vs how many available. This command in Mongo Shell will tell you exactly that:
db.serverStatus().connections