I went to school for Gastronomic Sciences and have worked with food justice and peasant/farmer movements for a couple years now.

The biggest issue with palm oil is that it's a cash crop produced through a monoculture, and that brings with it the destruction of local biodiversity and food ways, labor exploitation, and use of synthetic products for farming, with associated biodiversity loss and health risks for workers, who often used to be peasants who used to produce food that would be sold and bought by people in their communities, and who have been proletarianized by being pushed to work for the palm oil plantations that have taken over their land, most of the time "acquired" by land dispossession either by leveraging the legal system or by the use of armed militias to displace peasants under the threat of violence.

You can say the same about most coffee/cocoa/wheat/soybeans, but what makes palm oil particularly insidious is that it's like a cheat code for most processed foods, because it makes them have a much better mouthfeel, something you'd normally use butter or other seed oils for, but using palm oil is way cheaper to achieve the same result. So, a cheap production of palm oil using the land that would normally be used for local nutrition, using exploited labor, and traded from countries in the global south makes it the best option to achieve a "good", affordable processed food, because the externalities will not be included in the final price.

That being said, everyone I work with has no illusions about consumption habits single-handedly changing the global food system, that's some neoliberal cope, so if what you can afford has palm oil in it, or you want to get a treat that contains palm oil, it's not a moral indictment on you; It's a symptom of how broken food systems are, and how all actors of the food system need to collectively organize better and build parallel power structures to those that international trade and unequal exchange have built for the past hundreds of years.

There's something to be said about palm oil being an integral part of west African cuisine, and having been used for cooking there for thousands of years, but it is definitely a different kind of production and use to the one we see in the global north.

Sorry about the rambling and kind of shitty formatting, I wrote this on my phone so it is what it is.

The real-world production of palm oil is rife with the problems mentioned by others and yourself, but it is also typical of export agriculture in the global south. It isn't something fixable through individual action and you will find that avoiding the types of things that are bad about current palm oil production are present in basically everything else or the alternatives have their own problems. Navigating ethical consumption at the level of commodity supply chains is often deeply fraught and this is one such case. For example, cobalt is often mined by child slaves. You don't control whether that cobalt ends up in your electronics. Should we end child slavery? Yes! But it won't come from better individual consumer choices, especially because you lack sufficient information to make judgements. Your phone might have that cobalt while your laptop doesn't and even the manufacturer might not know. Is the palm oil in your crackers from land that orangurans inhabited? Was it stolen from indigenous people? Same deal, the information is hidden. Capitalist alienation is cascading levels of removal from the production process, of obfuscation that means you often can't make such decisions.

I'll give a contradictory example, though. With animal agriculture and veganism, drawing lines becomes much simpler. You just don't consume animal products to the best of your ability. You can't perfectly avoid them for the same reasons you don't know where the cobalt came from, but it is very simple to avoid animal products most of the time without having to contend with the alternative being worse, as animal ag is usually a superset of other ag and production systems. It's a "yes and" negative.

So to synthesize, if it is helpful for you, one thing you can do is try to avoid processed foods. This will decrease the length of the supply chain you know about and make it therefore more investigable. It's also better for your health. Though to be clear, it isn't free of ethical problems. In the US, for example, food is produced via the exploitation if a racialized economic underclass by monopolies that ensure Imperialist food policies globally, including the production of palm oil. But you can then take real action by fighting for farm workers' rights, helping them unionize, etc. They are your neighbors.