Getting Accurate Speed Test

  • 1
  • Question
  • Updated 3 years ago
Anyone following this site is aware of the problem for those people on the Evolution plan after being DAP'd. In addition to the inability to access lots of different web sites like Fox News, CBS News, imdb, NCAA.com there are also lots of complaints about speeds being impacted. Although I don't regularly check speeds I have occasionally done so in the past. My initial findings is that speeds since this Evolution problem started 2+ weeks ago are significantly lower than what I had experienced over the past two years. This is not anecdotal but based on actual speed tests.

Which leads to my question....how to get a reasonably accurate measurement of speed.

This morning I ran two sets of test using four different speed tests with results shown below. Note that the tests were run as fast as I could run them (consecutivley) so the tests were spread over about 5 or 10 minutes around 7:30 AM this morning. Results shown for Download/Upload...


                           EXEDE         testmy.net         speedtest.net        Time Warner      speedof.me
First Set              11.3/5.1          7.1/2.2                  13.1/3.9                10.9/3.9                .07/.42
Second Set          7.3/4.9           5.4/2.2                   5.4/1.2                  10.4/4.6                 na


Given above results, what should my answer be if someone asks what kind of speeds am I getting?? Anyone have suggestions on test approach?

Note that speedof.me regularly gives results that are a fraction of the other four. Of course, each of them touts that their test is more accurate than others.
Photo of Bruce Batty

Bruce Batty

  • 36 Posts
  • 14 Reply Likes

Posted 3 years ago

  • 1
Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
 
The location you test from must be capable of delivering a solid data stream to you or the test is no good.  It's that simple.  Test from a source that has an insane back-haul capability.  A cable company, even Comcast, does NOT have the resources to fully saturate the connection from the Internet to you.  Their local demand leaves little spare overhead to "nail" the connection, even at our paltry speeds.
Try testing to Fastmetrics in San Francisco.  Their capability is astonishing.  The only other location with their capacity is in PaloAlto CA at SingTel, which also allows testing at Speedtest.net.
Don't be concerned if you are at the opposite end of the country, satellite really couldn't care less where you are.
If you test from Fastmetrics or SingTel and get really fast speeds, don't say there must be a problem.  Nobody can send data faster than your system can actually take it.
 
 
(Edited)
Photo of Bruce Batty

Bruce Batty

  • 36 Posts
  • 14 Reply Likes
So if test results are erroneous, would they typically be low because of the factors to which you referred? Or, could the results be in error either was...too fast or too slow?

Also, wouldn't it behoove Exede to make sure that their test is not giving a result that is slower than actual?

Lastly, since I showed results for speedtest.net which you recommended, what should one conclude when the results vary so much and the tests were run less than five minutes apart? Is it necessary to run a series of five or ten tests and then average?

Thanks for your input, Craig!
Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
A slow test is an indicator that the test source isn't up to the job.  You suffered the "priority" syndrome at the source.

exede has their own problems and again, they don't have a mind-blowing resource to test with.  Also if they showed the fastest speed consistently, someone would say the results were "padded."

Speedtest.net allows manual selection of the test source.  If you allow them to choose for you, you'll just test from the location having the lowest return ping but not necessarily the fastest data stream.  You could do multiple test from Fastmetrics and SingTel and then compare what the result are from 2 different rock-solid test sources.
 
Photo of Bruce Batty

Bruce Batty

  • 36 Posts
  • 14 Reply Likes
So, what criteria would you suggest using when selecting the test source location on speedtest.net ?
Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
A slow test is an indicator that the test source isn't up to the job.  You suffered the "priority" syndrome at the source.
Maybe I should clarify this.  My statement above assumes you get extreme variances in speed test results from different sources.  Some people never get a good speed report because their exede speed is just always slow. 

Onward...  I consider the capabilities of the test source and what type internet service they perform for their customers.

Example:  Fastmetrics in San Francisco serves corporations and large businesses that need gigabit+ connections that get delivered with a guaranteed bandwidth.  They have terabit sources and the backhaul to deliver without fail.  For us to tap their system to do a speedtest is barely pocket change to them, we'll get an absolutely solid data stream from them.  I would expect using their system for a test would involve a minimum number of hops to arrive at the exede system due to their connection is directly off an internet backbone.

A cable company serves thousands of small customers and are supplied by gigabit connections or less in localized areas.  Their systems are subject to being loaded by customers to the point they experience congestion and must use traffic management to keep things going for everyone.  When you test with a cable company you may experience a choppy data stream that will appear as a slower connection at your end.  Testing from a cable company would results in more hops due to  the data navigating their system and the internet itself before arriving at exede and finally, you.

This is a rather basic explanation, but shows a solid source is important even for our paltry speeds to be accurately tested.

On a smaller scale, think of what your connection would be like for someone to use as a speed test source at their house.  See, the source capability is very important.

 
Photo of Bruce Batty

Bruce Batty

  • 36 Posts
  • 14 Reply Likes
Not sure you understood my question. You wrote "Speedtest.net allows manual selection of the test source." My question is how would i decide whether to use Kallamazoo or Indianapolis or Columbus or some other source?
Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
There is a little box on the map.  Use your mouse to slowly  slide it around to the area desired and the main map will follow, showing dots for test locations.  Move mouse pointer to the dot on the main map and a drop-down list will appear showing the various test sites to choose from.
Photo of Gwalk900

Gwalk900, Champion

  • 451 Posts
  • 471 Reply Likes

You have to first ask yourself .... "what is a speed test and why am I running it?"

As bragging rights? (my dog is bigger than your dog)

OR

As a diagnostic tool? (something is wrong!, How do I compare?)

To me it is a diagnostic measuring tool. As such, in order to be effective, I have to measure my performance with the same "Yardstick" as used to measure the performance of other users in the same subset.

It really doesn't matter what the length of that "yardstick" is ... 30", 36" or 42" ... as long as the same "measuring tool" is used to compare the relative performance of similar systems.

This is where it can get a little sticky. A two-way satellite Internet connection has more "moving parts" than a terrestrial Internet connection.

The first difference is High Latency ... and many (in fact most) speed tests will have a accuracy problem when dealing with a high latency connection.

The second issue is that a satellite connection is going to be through a proxied  web acceleration server ... and most speed tests are going to have skewed results because of that.

A third and very large important difference is the  ... multi frequency "burst transmission" aspect of a satellite connection.

In a ground based system, (for which most speed test sites are calibrated for) your "speed test load" is a nice little linear stream of digital "1's" and "0"'s just like nice little solders all in a row. A speed test then becomes a simple matter of time versus load.

A satellite connection is different.

Your "request" to run a test is from start:

Modem>ODU>Satellite>Gateway ... now ... your data hits the "real internet", follows terrestrial backbone paths to the Speed Server ... that data is then returned to the Gateway ... and here is where is gets interesting ....

Your return "data stream" is  ... broken down too as many as 5 separate  ... streams, all at a different frequency and transmitted to you has a very short term "burst" transmission up to the satellite and then down to your ODU and then "reassembled" by your Modem. Modems "modulate" and "demodulate".

You can NOT use a typical speed test site that does not take these differences that are unique to a satellite connection into account.

You can not have a helter-skelter mix of "different length" measuring tools.

You ISP has to "declare" one as being acceptable ... this "34 inch yardstick" will be used as an comparison to measure performance.

Now this is interesting.

Hughes, even though they have their own "in house" speed test (formerly used JAVA)  has Hughes Engineers recognizing testmy.net as being a site that takes into account the latency, burst and web acceleration aspects of a sat connection .... and the "yardstick" length is further defined as:

For "newer" platforms: Manual Test: 12 MB/ down: 2 MB/ up

Three the five tests spaced 5 minutes apart. Tests done three times in 24 hours .. early, mid-day, late.

Performance patterns will show if the problem does exist, and if so is it due to local issues, gateway issues ... or .... over sell/ over crowding.

At least compare Apples to Apples and not Apples to Plums.

      


Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
@ Gwalk900

Very good. One thing I keep forgetting is that I don't usually use the exede accelerator due to my use of a VPN most of the time.  My VPN is handled by my router so I don't need to mess with it all the time.  I do experience a slightly lower speed through the VPN but not so much that it would make using a it a nuisance making me question just how much benefit the ViaSat accelerator offers.
Photo of Bruce Batty

Bruce Batty

  • 36 Posts
  • 14 Reply Likes
Craig, I understand how to use the map. Reiterating my question, why would I want to use one location vs another whether it be Kalamazoo, Indianapolis, Columbus, etc, etc. ?
Photo of J&J

J&J

  • 1788 Posts
  • 1084 Reply Likes
The capability of the source site to furnish a solid data stream. A broken-up download from a wanna-be good source affects test results. I can't get any clearer than that.

I test with "who has an absolute solid connection to me to test with."  It's a test, so everything must be done to insure it's really MY connection that is the limiting factor, not a cheesy source.

The maps HERE show a test from a solid source in Palo Alto California is about as good as good as it gets, and that would be SingTel when using Speedtest.net.
(Edited)
Photo of GabeU

GabeU, Champion

  • 2337 Posts
  • 1425 Reply Likes
Reiterating my question, why would I want to use one location vs another whether it be Kalamazoo, Indianapolis, Columbus, etc, etc. ?
Normally, a speed test like speedtest.net is going to try to find a server that gives the lowest latency, as higher latencies can start to cause issues when testing speeds.  Being that Exede, or any other satellite based ISP, has an incredibly high latency already, choosing one over the other for that speed becomes negligible.  Then it comes down to a reliable server.  Some or more reliable than others, but most are fine.  

But, keep on thing in mind.  Some speed tests, like speedtest.net, are multi threaded, and with a multi threaded test, the higher the latency, the more erroneous the results can be in comparison to your true speed due to a higher latency opening the door for more issues with the test itself.  

It's just a recommendation, but I would stick with Exede's own or testmy.net.  
 
Photo of GabeU

GabeU, Champion

  • 2337 Posts
  • 1425 Reply Likes
Reiterating my question, why would I want to use one location vs another whether it be Kalamazoo, Indianapolis, Columbus, etc, etc. ?
Normally, a speed test like speedtest.net is going to try to find a server that gives the lowest latency, as higher latencies can start to cause issues when testing speeds.  Being that Exede, or any other satellite based ISP, has an incredibly high latency already, choosing one over the other for that speed becomes negligible.  Then it comes down to a reliable server.  Some or more reliable than others, but most are fine.  

But, keep on thing in mind.  Some speed tests, like speedtest.net, are multi threaded, and with a multi threaded test, the higher the latency, the more erroneous the results can be in comparison to your true speed due to a higher latency opening the door for more issues with the test itself.  

It's just a recommendation, but I would stick with Exede's own or testmy.net.  
 
Photo of GabeU

GabeU, Champion

  • 2337 Posts
  • 1425 Reply Likes
Wow.  Lot's of double posts lately.