This may be construed as anti-Semitic, but why do they have dibs on that term. Semitic is supposed to mean "designating or of a group of Afro-Asiatic languages, including Hebrew, Arabic, etc."

I'm sure there are special words for all kinds of racism, we are just not aware of them.

While it's true that "semitic" refers to people who speak the languages of the middle east (primarily Arabic and Hebrew, but also some of the dead or dying languages), this becomes one of those "prescriptive vs. descriptive" debates when applying it to the word "antisemitism."

That term came into use in central Europe (Austria in particular) in the late 19th century, when "anti-jewishness" was a raging social and philosopical issue. At that time there were virtually no arabs or other semetic people living in Europe, but jews had been part of European society for centuries. There has always been anti-jewish prejudices in Europe, but when it was reaching a boiling point in and around Vienna in the 1870s, the people who wrote about it (both for and against) chose the term "antisemitism" for whatever reason, and it stuck, because at the time, they were the only semites around.

I don't know when people first started using the more general term "racism," but I think it was long after "antisemitism."