Armor penetration rating on gear isn't the world's most intuitive system, and dimly-heard rumors of a "marks armor pen build" have been creating a lot of confusion that bubbles up pretty much daily on the WoW hunter forum.
To start with, armor penetration rating reduces target armor by a set percentage. The more armor pen rating on your gear, the higher that percentage, capped at 100% when you reach 1232 rating. It's important when considering this to factor in raid buffs like buff food and armor piercing elixirs, because it's possible for those buffs to make gemming for armor pen edge out, say, gemming for agility.
One of the places people get into trouble is when they take the above fact and consider it next to things like the warrior debuff Sunder Armor. Five stacks of sunders would mean 20% armor reduction, so it seems reasonable to think that if your raid is putting sunders or Expose Armor on the boss, then you would only need enough armor pen rating to equal 80% armor reduction.
As it happens, however, that's not how it works. What happens is that the game calculates the armor reduction from debuffs first, then applies your reduction from rating on your gear. That is: if a boss has 10,000 armor and five sunders up on it, then your armor penetration rating will be applied to 10,000 - 20%, or 8,000 armor. In a way, this reduces the value of your armor pen rating, because you're taking the same percentage of a smaller number. In terms of actual in-game DPS, though, sunders (and other similar debuffs on the boss) actually increase the value of your personal armor pen rating, because even though you're subtracting a smaller amount, you're also subtracting it from a smaller number. To continue with the example from above, imagine you had enough rating to equal 50% armor reduction. Without sunders, you're reducing that 10,000 armor on the boss to 5,000; with sunders, you're reducing 8,000 armor to 4,000.
The final mechanic to consider when you're thinking about armor penetration rating is diminishing returns. DR interacts with armor from a DPS perspective in a couple different ways. The first is that armor suffers from DR the more of it you add to a mob or player. The first point of armor mitigates more damage than the second, the second does more than the 10th, and so on. This in turn means that armor penetration rating is the only (or one of the very few) things that benefits from increased returns. The more armor you strip off of your target, the more the next point you take off benefits your damage. Second, as boss armor goes over 8,317, it applies diminishing returns to any armor reduction used against it. This makes armor reduction debuffs even more important, because it reduces the impact of those diminishing returns. Bosses in Wrath of the Lich King have 13083 armor, putting them well into that territory.