Yes I do try to make the distinction a lot, personally. People can be prone to saying or thinking things that are racially charged, sometimes it may be unconscious to them. They're not good, and should be corrected, but racist to me definitely carries some implication of deliberate malice.