Experiencing "sound" is mostly a matter of subjective and personal taste. Describing sound to someone else is even harder, espcially for me being a non-native speaker.
Therefore I introduced my first comparison videos 1 year ago, which were still very clumsy and not recorded that well. But I continued to improve the recordings as well as optimize my whole recording setup. My idea was that two or more speakers could be compared pretty well to each other by just listening to them, without having to write a single word. When hearing two speakers on your own and noticing the change in sound when one switches over to the next one, this would be the most objective sound comparison I can provide. Of course everyone listens with different speakers, or different headphones, so everyone will be hearing something else, but the relative difference between two speakers will always remain the same. I also doubt that my recording microphone is as linear calibrated as it is claimed to be, thus there will always be some deviation from the pure original sound already starting with the microphone. But as long as both speakers are recorded in the same way, with the same microphone etc. they should still be comparable pretty well.
Meanwhile I think to have managed a kind of standardized setup, where I can record various speakers successively without even having them at hand at the same time. I have my recording room that you can see in the image below, which always remains the same. It is a small room we use as a garderobe which is acoustically quite dampened compared to normally furnished rooms. It is still not comparable to a real anechoic chamber, but all speakers sound considerably more "pure" in this room, that's why I chose it for my recordings and for all further measuerments.
I usually have a fixed distance to the recorded speaker and I have my fixed volume levels which I derived from the Soundlink Mini and continue using them. So I could even build up a data-base with all the recorded speakers so far, so everyone could switch "live" between different models for their liking. Unfortunately I have too little experience, how to achieve something like that, I am not a programmer, so I won't be able to manage something like that.
Recently I discovered Sonic Sense Pro Audio, doing something very similar, but in a much more sophisticated and professional way with the only difference that they are comparing professional recording equipment, studio monitors and headphones instead of cheap portable speakers.
But I found their headphone recordings particularly interesting, where you are given the chance of listening to various different headphone models and comparing them to the original source tracks, you must have a listen! I wonder which you think sounds closest to the original source!
Of course I cannot afford any of the recording equipment Sonic Sense uses, nor do I have the time to perform such complex tests, but I would like to add some more data to my reviews which might somehow prove what I am hearing and why this or that speaker sounds that way and not the other.
Some time ago I ordered a cheap Dayton iMM-6 calibrated measuring microphone for mobile phones or tablets to fool around with it a little bit. There are lots of different audio-apps for iOS which provide FFT spectrum analysis. Some are really expensive, so I settled for "Audio Tools" from Studio Six Digital as it seemed to be the only one which could also read the calibration-file of my microphone.
But now all problems began. How should I measure a speaker? Is pink noise precise enough, or should I use a sine-sweep instead? Which distance should I choose to the speaker? Which room? The simple iOS apps don't provide any gated measuring, which is more or less standard on most desktop-programs like REW etc and allows a quasi-anechoic measurement with any room-influence being eliminated. But I wanted to keep things as simple as possible, just attaching the microphone to my iPhone and having a mobile measuring tool.
As the iMM-6 microphone has a headphone pass-through I was able to hear what the microphone was actually measuring. I noticed that when playing pink noise through a speaker and moving it slightly to the left or right or turning it, the resulting sound changed considerably. I already noticed the same with my audio recordings. A perfectly centered perpendicular positioning of the microphone gave worse results than rather setting it up slightly offset with the speaker turned away a little bit, which usually gave the clearest sound with most portable speakers that suffer from a rather directional sound. It might also be interference of both drivers resulting in comb-filter effects, when the microphone is setup exactly within the center of both drivers.
Then I tried to elaborate a typcial measuring procedure. I should probably measure a speaker from various fixed angles, I also thought that a speaker with dynamic sound adjustment should be measured at different volume levels in order to show how the overall sound changes with the actual volume.
I tried to demonstrate this with the a quick measurement of the Bose Soundlink Mini when it was playing at 30%, 50%, 75% and 100% volume. You can clearly see that from 75% upwards the lowest frequencies are already reduced considerably resulting in a more compressed frequency response. At 30% and 50% you see an obvious bass-boost as well as a lower-treble boost above 2kHz and a strong treble roll-off starting at 12kHz which probably is the reason why the Soundlink Mini has the tendency to sound muffled.
To minimize room effects I even tried performing some outdoor measurements. During night it is mostly silent, so I set up my "portable measuring studio" in the garden and tried the same measurements outdoors to have a direct comparison with the indoor measurements.
Unfortunately also the outdoor measurements showed lots of interference, probably due to the nearby house and the ground. You would obviously need a much larger open space and a higher distance above the ground to really get quasi anechoic results. Both measurements look as if they could be a completely different speaker, in this case the Sony SRS-X3 was measured from the same 50cm distance with the same fixed level of 70dB:
All this made me finally order another calibrated USB-microphone, so that I am currently trying to manage some better measurements with my notebook and REW to see if the results are really more significant than what I got from my first quick and dirty approach.
A first measuring test with the miniDSP UMIK-1 microphone and REW of the Bose Soundlink Mini at various distances in my usual recording room shows how much the results change depending on distance from the speaker. At 1 meter there are already too much room reflections to be regarded as representative.
I also tried some further measurements in our living room, here is the result for 100cm distance, compared to the 100cm measurement from our garderobe and as you can see due to stronger reflections the living room measurement has considerably stronger bass:
If I manage a reliable measuring-workflow I would like to measure every speaker I currently have at home as well as every new speaker that I will review in future to be able to provide additional frequency response data, which will hopefully be able to add some objective insights into the real frequency response of a given speaker. You can already find my review of the Mini Jambox here, where I tried to add lots of frequency measurements and comparisons.
There are some other ressources which also provide speaker measuring data. I highly regard Brent Butterworth as professional speaker-tester, here you can see his measurement of the Beats Pill XL at various angles:
And here my indoor measurement with the new miniDSP UMIK-1 (smoothed with 1/48 octave):
Also lesnumeriques has already measured lots of different portable speakers, here their measurement of the Pill XL:
On the other hand I am not really sure what they are actually measuering. Here you can see their measurement of the Bose Soundlink Mini, which looks so much different to what I am getting:
It seems to be quite an effort to complete a whole measurement for one single speaker including setup, measurment from all angles, with different volume levels, and maybe even with different sound modes. So keep your fingers crossed that all this won't get out of my hands quickly and stay tuned for some speaker measurements in future!
