That is correct. The addresses need to be aligned as per the restrictions in the architecture. Yes we could have an issue but software writers need to deal with it because IIUC you either have a huge performance penalty (x86 / lock prefix) or correctness issues (ARM, Powerpc) .
Ok thanks, I buy the performance argument and I found on the wiki page below that these issues will be treated in the GCC/libatomic in 4.9 release.
http://gcc.gnu.org/wiki/Atomic/GCCMM/UnalignedPolicy
Cheers, Yvan