I agree with you 100%. I was doing exactly what you suggested and I pulled out my Glock 31 and compared it to the XDM 9mm. Box to box, the options that you receive were out weighed by the XDM. You get a gun holster (great holster for beginners and range), two magazines, magazine loader, magazine holster, lock, extra grip backs and cleaning rod. With the Glock you get a magazine loader, two magazines, lock and cleaning rod. I suppose with the extra options you are saving money by going with the XDM in favor of the options that come with it. If you went out and purchased all the other options for the Glock (albeit I'm sure you would purchase a little bit better holster), I'm sure it would come out to be more than the XDM.
Honestly, it doesn't matter to me. The Glock feels great in my hands and so does the XDM. I love the aftermarket options that you have with the Glock, but I love the craftsmanship of the SA XDM. Both are excellent choices and you can't go wrong either way.