There's three different letters that shouldn't be mixed between each other: ъ, ы, ь.

ь is soft sign. It softens the consonant it's after and only affects the previous letter. It doesn't affects the pronouncation of any other letter or move stress in the word.

ъ is hard sign. It also always goes after consonant and signifies that this consonant should be separated with voice from next letter after consonant, without merging with it. Put in google translator words дело (affair) and подъезд (entrance) and check for the sounding of combination "de", it would be like "de" in the first case and like "d-e" in the second.

ы has nothing common with previous two ones. It's just a vowel which reads like "yi" or something like that. Check for ты (thou), вы (you), дым (smoke).

And great observation regarding pronouncation of "o" letter! It has nothing to do with soft sign, just Russian language allows you to read "o" which is not under stress as "a". It's not hard rule and is something like "British English accent" as historically there's Russian regions which are окающие ("o-saying", not using this type of pronouncation) and акающие ("a-saying", who speak like google voice does). So if you're muscovite, you'll say корова (cow) as karova, and if you're novgorodian you'll say korova. But that doesn't matter that much, as regardless if you say korova or karova, both muscovite and novgorodian would clearly understand what you're talking about.