The vast majority of English speakers use "racism" to mean "prejudice or hate based on race," which covers a lot more ground than structural racism. There isn't a great reason to try and redefine racism to exclusively mean structural racism, either, because individual prejudice based on skin color is bad, too.

When people see prejudice based on skin color, the response shouldn't be "whoa whoa whoa, maybe this is OK, depending on who has power here." The response should be that prejudice based on skin color is bad in any situation, but is especially harmful where the group exercising that prejudice has structural power to hurt the target group. Some types of prejudice being worse than others does not mean there is an excusable form of prejudice. It definitely doesn't mean that the less harmful forms aren't prejudice at all.

You think its bad yet you aren't offended by it? How does that make sense? If its bad then you really should find it offensive.