I'm realizing I've just been doing it wrong all this time. The trick? Cook your potatoes before home frying it up!
Cut a potato into 4 pieces and put into a pot of water, fully covered. Bring to a boil, then let it go for about 10 minutes. You want it to be just fork tender. Remove them from the water when done, and roughly chop. Heat some oil in a pan over medium-high heat and throw your taters in. Let them crisp up on the bottom before turning, at least 5 minutes. Cook to desired doneness, salt and pepper to taste. Personally, I love some Cajun seasoning in there. Dig in!