Our headphone evaluation begins the moment the product lands on our desk. We take note of the product’s packaging and presentation before diving into de-boxing it.
Also check out our picks for the best headphones as picked by our reviewers.
Out of the box
The de-boxing process involves not only physically removing the headphones from their packaging, but taking note of the out-of-box experience as a whole. We like to see a nicely presented product, and at this stage we’ll be getting our first impressions of build quality and sound quality as well. After taking an inventory of any included accessories, we take a brief listen to the headphones to make sure there are no obvious defects and get a feel for how the headphones sound right out of the box (bearing in mind that break-in almost always changes a headphone’s sound). Once completed, we will connect the headphones to an audio source and break them in by playing random music at a medium volume for at least 40 continuous hours.
The testing gear
With break-in out of the way, we’ll begin a critical listening session. This may take place over the course of several days and will, at some point, involve all the pieces in our “headphone test bench” which currently includes a HeadRoom Micro DAC, HeadRoom Micro Amp, NuForce Icon u-DAC2, NuForce iDO, iPhone 4S, Dell 15r laptop, and a variety of music tracks ranging from 128k MP3 and AAC files to uncompressed WAV and FLAC files. We also keep a reference set of similarly designed and priced headphones on hand for comparisons when applicable. For instance, if testing an in-ear headphone in the $100 price category, we will likely end up comparing it to something like the NOCS NS400 or Klipsch S4. Of course, our stable of reference headphones is changing constantly as we discover new favorites.
Our sound quality evaluations are based entirely on listening tests. Therefore, our conclusions are the result of subjective analysis. Some headphone reviewers take a very different approach, relying heavily on measurements taken by electronic testing equipment that, among other metrics, attempt to measure the frequency response curve of a headphone. We have plenty of respect for that approach, but it isn’t for us. Our choice not to employ objective testing methods is based on a few factors:
First, taking a person’s varied anatomy into account is a difficult proposition. Ear shape, ear size, head volume and chest volume are a few of the factors that combine to change the way a headphone will sound to a person and duplicating that with a machine is really tricky stuff (though some have gotten really good at it). To be fair, these factors also affect how a human will hear and analyze sound, but since it is humans that will be reading our reviews, we’re OK with that. We also have the advantage of passing our gear around to several listeners to corroborate or distinguish opinions.
Second — and this is especially important when it comes to in-ear headphones — getting the right fit is crucial in getting the best sound from a headphone. We are able to ensure, one headphone after another, that we are getting the same great fit. This is based on a wealth of experience and familiarity with our own anatomy.
Finally, and perhaps most importantly, we believe that specifications only tell a small part of the story. A lot of listeners aren’t interested in getting a flat frequency response (moreover, we believe that many of our readers couldn’t care less about a frequency response curve and that those who do will probably be reading more than one review anyway). Manufacturers know this and “voice” their headphones to sound a certain way. The Beats by Dr. Dre headphone brand is an excellent example of this. Those headphones are intentionally heavy on bass. We don’t need a frequency response chart to tell you that.
This is a pretty straightforward process. We listen to music tracks that we know inside and out. Some of the music we use was recorded, mixed and mastered by us, which gives us the best possible insight into how the reproduced track should sound. The first thing we listen for are blatant diversions from reality. If a headphone’s high-frequency response is thin, metallic and artificial sounding, we’ll hear it right away. Likewise, if vocals sound as if they are being reproduced through a toilet paper tube filed with insulating foam, it will be pretty obvious. Headphones that make these kind of errors will immediately fall to the bottom of the heap and will likely get a fairly low score.
If a headphone doesn’t do anything especially bad, we start listening for more subtle queues. For instance, we’ll listen to bass response to determine if it has been boosted, then turn our attention to the midbass frequencies to see if they have been gotten murky as a result.
After scrutinizing gads of minute details, we take a step back and consider whether the headphone will appeal to a broad enough spectrum of listeners. We may not particularly care for a headphone’s distinct sound but if that sound is going to delight a certain audience, then it has its place in the market and should be noted as such. Conversely, if a headphone requires gobs of power to sound good but is clearly meant for low-powered media players, that’s a big problem.
Fit finish and build
The best-sounding headphone in the world isn’t worth a dime if you can’t stand to wear it for more than 10 minutes, or if it is likely to break in the first 10 hours of use. We wear headphones for two to three hours at a time to determine if they are comfortable enough for longer term use. We stress headphones a bit to ensure they aren’t likely to break under moderate stress and we look at the overall design to consider basic ergonomics and assess whether or not they are aesthetically pleasing enough that people won’t mind wearing them in public.
If a headphone comes with a microphone, we’ll make test phone calls and evaluate the performance on both ends of the phone. If the headphone has buttons, we’ll press them a lot and to make sure they work and are relatively easy to use.
Writing the review
For us, simply organizing our evaluation notes into something that resembles a narrative is not good enough. We aim to give a little back-story on the manufacturer, describe our experience with the headphones and put their sound signature into a context that is understandable, even for those not familiar with the audiophile lexicon.
We hope that by reading our headphone reviews, you will walk away with a feel for whether that particular model is something you should bother to audition yourself or remove from your short list because, ultimately, the review that matters most is your own.