You don't even need to use
rm in this case if you are afraid. Use find:find . -name "*.bak" -type f -delete
But use it with precaution. Run first:
find . -name "*.bak" -type f
to see exactly which files you will remove.
Also, make sure that
-delete is the last argument in your command. If you put it before the -name *.bak argument, it will delete everything.