A public game means that the forums are readable to anyone who has the link. I don't know if you can apply to a game when it's at or over its player limit.
Qralloq started a
thread where GMs could post links to their public games. That's currently the only way to find public games, as the public game search functionality's been disabled for technical reasons.
The advantages, as I see, are providing a way for interested people to get familiar with your game, maybe for aspiring GMs to get ideas on how you run a game in a particular system, or players to see how a system works in practice before they join a game.
The disadvantages, well, anyone can see what you're posting. Some people don't like working/playing in a fishbowl. Especially if your game has the potential to contain anything that might be construed as offensive or unpleasant.
I'm sure there are more, but that's the way I see it off the top of my head.