In the late 1800s, at the Rugby School in England (the school is specifically in Warwickshire, but on the other hand I haven’t the faintest idea where Warwickshire might be), students for some reason started to form new slang words by adding “-er” to the end of a root word. They created “ekker” for exercise in 1891, “brekker” for breakfast in 1889, “bonner” for bonfire in 1898, and “footer” for football, perhaps the earliest example, in 1863.
Other schools, such as Harrow, picked up this habit, and eventually universities (Oxford and the like) did too. Now, they formed “footer” out of football, but in those days there was another form of football played at these schools: association football, which was the same basic game but with the rules of the Football Association in effect. So it needed a different slang term, and around 1886 or so they came up with “soccer” from “asSOCiation”.
The Football Association and its rules for the game didn’t come into existence until 1863, but people have been kicking a ball around on a field for millennia, and in English that practice had always been called “football” (or, by schoolboys in the 1880s, “footer”). So “soccer,” while mentioned in various publications from these schools, never entered common usage to the extent that it could replace “football.” But Association Football WAS a slightly different game, and specifically it differed from the form of the game played at the school that started the whole “-er” thing. That game was called “Rugby football,” or (as you might guess) “rugger.” And “soccer” stuck around just to distinguish the games.
By that time there was also a Football Association in North America — the US Football Association. But around the same time “American football” was getting popular, and there was beginning to be confusion around which football association was which, not to mention confusion about what the call the games. So the US Football Association changed its name to the United States Soccer Federation, and started consistently calling their game “soccer”.
By the way, American football is supposedly often called “gridiron” or “grid” in the rest of the English-speaking world — I always thought it was just “American football.” Except in Australia, where they have “soccer” and “football” (or “footy”), which is what they call Australian Rules Football — that’s yet another version, kind of a mix of Association Football and Rugby Football, or “soccer” and “rugger” (which is just called “rugby” everywhere else). But it was probably the US practice that moved “soccer” from slang to wide acceptance. In the US, at least.
Speaking of “gridiron”, that word comes from the 13th century and originally meant a cooking device. It was probably related to “griddle”, which is even older. Later on “gridiron” began to be used for almost anything that had a similar pattern, and by the time a word was needed for Yet Another Version of a game on a field with a ball, “gridiron” was ready and waiting. There’s a citation from the London Daily News in 1896 that sums it up pretty well:
“The ground here is marked out by white lines … thus giving it the appearance of a gigantic gridiron—which, indeed, is the technical name applied to an American football field.”