That's not quite how the -r switch of rm works:
-r, -R, --recursive
remove directories and their contents recursively
rm has no file searching functionality, its -r switch does not make it descend into local directories and identify files matching the pattern you give it. Instead, the pattern (*.o) is expanded by the shell and rm will descend into and remove any directories whose name matches that pattern. If you had a directory whose name ended in .o, then the command you tried would have deleted it, but it won't find .o files in subdirectories.
What you need to do is either use find:
find . -name '*.o' -delete
or, for non-GNU find:
find . -name '*.o' -exec rm -r {} \;
Alternatively, if you are using bash you can enable globstar:
shopt -s globstar
rm -r -- **/*.o
NOTE: all three options will delete directories whose name ends in .o as well, if that's not what you want, use one of these:
find . -type f -name '*.o' -delete
find . -type f -name '*.o' -exec rm {} \;
rm -- **/*.o