In sports betting a "sharp" is someone who knows what they're doing, they're the smart money. If you've seen the movie "Casino," Robert De Niro's character is a sharp.
Betting with the sharps is a good way to make money in sports betting, but it's not that easy. Some sharps are so well known and so good that when they make a bet the whole market reacts, and a flood of money will come in on the same side as the sharp. Most books will watch these sharps moves as well and adjust their lines accordingly.
To alleviate this, many sharps, because their bets are so influential, will purposely bet a line one way early in the week in an effort to push the line in a direction to their liking and then at the last minute money will pour in on the other side of their initial wager.
Because of these reasons betting with the sharp's isn't always easy and for many sports bettor's, finding a sharp to bet with isn't nearly as profitable as finding a square to bet against.