In talk about environmentalism, the term carrying capacity in relation to the earth often comes up.
While I don't think anyone has a firm grasp on the theoretical carrying capacity of the Earth, I can get within one or two orders of magnitude.
Fundamentally, the earth's carrying capacity (assuming the goal is to maximize the quantity of human life) is governed by three things: oxygen in the atmosphere, availability of fresh water and availability of appropriately nutritious food. Fresh water isn't particularly a problem - people are (with sufficient wealth) already extremely good at creating filtration systems capable of eliminating fresh water as a limiting factor for growth.
The oxygen issue is actually not as much of a problem as it might seem - the optimal diet for humans is photosynthesizers, as such a diet most efficiently conveys the sun's energy into human bodies. I am optimistic that said food source could be tuned sufficiently so that the sun's energy striking the earth would be the limiting factor in human carrying capacity.
To get an idea of what the earth is capable, imagine all life as we know it stripped away from the surface of the earth, which is flattened and filled in, and covered in a constant surface of 10 meter deep algae grow tanks, which are consumed for food.
While this solution ignores the earth's core as an energy source, it is probably within a few orders of magnitude of the maximum number of people who could possibly be supported on the planet.
Obviously, we are no where close to such a situation. Given humanity's fractious past, it is unlike that we will ever approach such a state.
Update: I am not suggesting that this would be a particular enjoyable experience for most of the people involved. It is merely an exercise that attempts to introduce at least a modicum of rigor to the issue.