It would explain both why Rick can't return to the States, and why Sam is running halfway around the world with him. Back in America, a white person (or group of people) tried to kill Sam, for whatever reason (a robbery, or a lynching, for example). Rick somehow got involved, and in the process of defending Sam, killed the attacker(s). Even though it was an act of defense, blame wound up on Rick and Sam, because it was the 1940s and the cops were racists. So Rick and Sam decided to escape to Europe, and later Casablanca. This theory would fit with what other characters say about Rick. More than once he's said to be someone who often winds up helping the "underdog." Louis speculates that Rick can't return to the States because he killed a man, and Rick confirms this; true, that conversation