Don’t know if you read the thread I linked to above (https://community.exede.com/exede/topics/getting-accurate-speed-test) but input from Craig would suggest that maybe testmy.net does not give the most accurate results.
In his reply to my post, Craig wrote “The location you test from must be capable of delivering a solid data stream to you or the test is no good. It's that simple. Test from a source that has an insane back-haul capability. A cable company, even Comcast, does NOT have the resources to fully saturate the connection from the Internet to you. Their local demand leaves little spare overhead to "nail" the connection, even at our paltry speeds”....and... “a slow test is an indicator that the test source isn’t up to the job. You suffered “priority syndrome at the source”...and...”nobody can send data faster than your system can actually take it.”
If he is correct, then if a run speed tests from several different sites under identical conditions, then the fastest reading would be the most accurate. In other words, if the only variable between the different sites is the capability of pushing the data through then, according to Craig, the one with the greatest capability would be the most accurate.
That being said, we also know that conditions aren’t static. I can run two consecutive tests back-to-back using testmy.net and the results might be significantly different. I always assumed that is due to conditions changing from one minute to the next. Referring to the data I posted in the other thread, you can see that the results were all over the board within a 5 to 10 minute time span.
So, back to the original question, how does one get an accurate speed test and/or how can one have confidence that the test results are accurate?
Your thoughts?With regard to John Blount’s comment above regarding latency, I have also used speedtest.net on occasion. I have concluded that knowing the latency is not really helpful. It seems to be fairly consistent across different test sites and doesn’t change from day-to-day-or hour-to-hour and is merely a reflection of the distance signals have to travel with a satellite connection.
There are a number of things that are unique to a satellite connection that make it difficult to get an accurate speed test result. The Big Issues are latency, Web Acceleration and Compression.
The same confusion and distrust of speed tests and results have been seen for years on Hughesnet and both Hughes and Exede have similar architecture.
Hughes had its own java based speed test and for a very long time was the only test that they would allow as a measurement tool in determining if a user did indeed have speed related issues. Many users were reluctant to use it due to the need for java and others were reluctant simply because of mistrust of Hughes.
Anytime a user believes he has a problem then measurements must be taken and that users performance is then compared to other users in the dame class of service and beam and gateway. In order to be effective they all must be measured with the same tool. The tool doesn't even have to be "correct" (whatever that is) it just needs to be same among all users and it needs to be acceptable to Engineering to whom is charged with responsibility of network operations.
In the case of Hughes the Engineers finally decided to accept the testmy.net speed test results but ... there are three different tests.
One test size is for the newer faster Jupiter system (manual 12/2) and a different size test for the less capable Spaceway-3 (5/2) and the legacy 7000s platform using a manual 5/1 test. The latter would equate to Wildblue system
In dealing with a terrestrial system the capacity and response and location of the server is going to be important but with a satellite connection there is more to it.
There is a Hughes Employee named Patrick Fisher that posted about 4 years ago about the "extra parts" found in a satellite connection and why the average speed test may not work.
While the two posts are 4 years old and are Hughes related, Patrick's comments shed a lot of light on the differences between satellite and ground based connections and why just "any old speed test" is not going to work as a diagnostic tool.
Patrick Fisher, Employee
- 195 Posts
- 56 Reply Likes
Also, a little more information on why we have our own test. (And no, it's not so we can doctor results).
The end result of all of these variables is that we have no idea what they are actually measuring. A Speed Test might think it is giving you more accurate results by using weird tricks to get multiple measurements at once and combine them, but maybe our acceleration software interferes with that trick and makes it look better or worse than it is. Or maybe the speed test you pick has congestion on the internet link between them and us. Or maybe they pick a server that's thousands of miles from our gateway, so you have to deal with a bunch of extra internet latency. When we have all of these questions (and when we have no control or even information from the speed test provider about how it works), then we don't really get any useful information from external speed tests that we can use to troubleshoot. Finally, the one variable that they do show - uncontrolled internet links between us and them - is both rarely a problem, and not something you or Hughes is able to fix.
Our speed test is a simple test, which is designed to do us a couple favors:
1) We use Java, so that we can completely bypass your browser and any extensions or plugins, like Adobe Flash, that might introduce other variables. We don't have to question whether Adobe is doing weird things or your browser is crawling because of other extensions.
2) We log the tests, both so that we can see the tests you as an individual have run, and so that we can run reports on all of the speed tests for users in a given area, on a specific gateway, etc. This helps us identify performance trends versus performance problems that might be caused by a single user's configuration or LAN. If everyone else on your gateway or in your beam is getting good speed test results, but you are not, we can zero in on the things that are unique.
3) We control, manage, and monitor the links between gateways and speed test servers, so we can eliminate external factors like internet bandwidth as a culprit.
4) We can skip certain satellite optimizations as described above.
We do actually go out of your gateway on the same internet connection that your real traffic uses, to get to our speed test servers. So we aren't cheating by under-sizing our internet pipes and showing you just the satellite link, either.
Hopefully this helps shed some light on why we encourage users to use our own speed test service. :)
Patrick Fisher, Employee
- 195 Posts
- 56 Reply Likes
One unfortunate issue with TestMy.net (since they really do have a very good, simple and accurate test methodology) is that our compression software is particularly good at compressing their data.
Much of the web traffic that we send toward your terminal gets compressed by our acceleration software. What that means is that if a web page has a 1MB JPEG image, we might compress it and only send 500KB over the satellite link before it gets decompressed on the other side by your terminal. If your connection is 2Mbps, we can effectively send you that 1MB image at a speed of 4Mbps. (As an added bonus, only the smaller size gets counted against your download allowance - we always pass the benefits of our acceleration on to users)
TestMy.net's data gets compressed by this, so they send you a (for example) 2MB file, we compress it on the fly to 500KB, and they get 4 times the actual raw speed of the link.
This makes their test accurate for seeing the effective throughput for compressible data, but not necessarily for something like a bulk file transfer. The Hughes test shows the actual raw speed of your connection, skipping that web acceleration.
Hopefully the above sheds a little light on why a speed test on a sat connection is more complicated.