corruption on upstream TCP data and retries/failure of HTTPS and SSH

  • 6
  • Problem
  • Updated 5 years ago
  • Solved
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members.

MODEM firmware is rev  UT_2.2.3.0.13
data corruption with or without a router
have tested with 3 different router
windows 7 and 8, and 3 flavors of LINUX
HTTPS and SSH will catch the corruption and retry,  may or may not succeed with the file upload.
FTP will normally succeed but the uploaded files are corrupted, even though the TCP layer thinks the packets are good.
example corrupted images can be seen at the link below. these were uploaded with FTP
http://vast.net/bill/junk/
Exede, you have a serious problem with your software!

 
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes

Posted 5 years ago

  • 6
Photo of ExedeKarmin

ExedeKarmin

  • 384 Posts
  • 40 Reply Likes
Hello Slowbill,

Sounds like you need to speak with my Technical Department.  Please send me an email to exedelistens@viasat.com with your account and contact information.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Slowbill,

Take full advantage of that offer to bust through scripted 1st level support - my 1st level support through NRTC is virtually impenetrable...

And don't forget those of us hanging in the breeze on the other threads below for some supportin documentation/detail ;)
(Edited)
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
I added the original files to http://vast.net/bill/junk/ and a txt file showing the bit errors in one of the 2 meg files. somewhat of a pattern to the corruption. bit zero reset to 0 then 8 bytes later bit zero set to 1.
here is a frightening thought, what if this is caused by bad RAM in a piece of equipment used for beam 329, and more frightening what if that equipment is 22,000 miles above the earth? 
bit errors below,
decimal address, Octal good data Octal corrupt data 
cmp -l ./IMG_1753org.JPG ./IMG_1753viasat.JPG 
 dec       Oct Oct    bits changed
 135627  45  44     bit 0 reset
 135635 326 327     bit 0 set
 183807  71  70     bit 0 reset
 183815 216 217     bit 0 set
 507160 131 130     bit 0 reset
 507168 120 121     bit 0 set
 630312 335 334     bit 0 reset
 630944 246 247     bit 0 set
 832740 127  26     bit 0 reset, bit 6 reset
 832748  34 135     bit 0 set, bit 6 set
 869760  14  15     bit 0 set
 870376 335 334     bit 0 reset
1123757 373 372     bit 0 reset
1123765 132 133     bit 0 set
1136273  41  40     bit 0 reset
1136281 236 237     bit 0 set
1140253 267 266     bit 0 reset
1140261 336 337     bit 0 set
1180073 357 356     bit 0 reset
1180081 306 307     bit 0 set
1191241 233 232     bit 0 reset
1191249 100 101     bit 0 set
1375194  60  61     bit 0 set
1375858 257 256     bit 0 reset
1698501 223  23     bit 7 reset
1698509 142 342     bit 7 set
1727557 357 356     bit 0 reset
1727565  56  57     bit 0 set
1802017 155 154     bit 0 reset
1802025 262 263     bit 0 set
1809717 175  74     bit 0 reset, bit 6 reset
1809725 250 351     bit 0 set
1811613 257 256     bit 0 reset
1812709  14  15     bit 0 reset
1996558  47  46     bit 0 reset
1996566  14  15     bit 0 set
 

  
(Edited)
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
I agree, if the beer can took a hit. The fix would be a few days in coming.

Lab has the truck, but the fuel cost are about 12 million dollars. And then of course, what size crescent wrench to bring???

Sure would like to see the schematic for the can in the sky. JEP seems to have a lot of details stashed away and has looked into the inter workings more than most.
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
you'er a funny guy Everett. 
on a different note, it seems like I have been seeing some complaints like
"I have had no problems for x number of years, now I use up my Gigs in x number of hours"
I cant help but wonder if these people are being impacted by corruption causing automatic updates to endlessly error out and retry, thereby using many times more data than normal.
rev  UT_2.2.3.0.13 beam 329
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
The subject of lost data has been a popular one. As to now the only answer I have seen is to give the customer some free added Gig's. Of course, many people do not post back if their issues where fixed or not, so no firm answer if it was fixable or not.
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Bill - ViaSat-1 functions as a "bent pipe".  It is strictly RF in and RF out.  It doesn't know anything about ones and zeroes.  The received signal is amplified, hetrodyned to a new frequency, boosted up to transmit power levels and away it goes.  This keeps things as simple as possible on hard to service equipment and also the "bent pipe" mentality does not lock it into any specific protocol, packet size, etc.  This allows channels to be used for almost any application as long as it doesn't exceed the bandwidth of the channel.  I'm just guessing, but I suspect ViaSat-1 also has spare receivers and transmitters in case something fails. 
Photo of Dorothy Allen

Dorothy Allen

  • 1 Post
  • 1 Reply Like
A quick hello to slowBill and a thank you for posting here about your experience.  A lot of people can realize when a problem exists but have difficulty describing exactly what it is nor do we have any idea of what's causing it much less how to fix it.  Please continue to talk about your internet experiences.  Thanks.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Dorothy,

If experiencing issues that sound like ones you may be having, describe them as best you can. There are enough of us who can make the transition/correlation to/from geek-speak to everyday English.

The more people that report an issue the less likely the issue can be dismissed, and it will receive a higher priority,  but the more detail the better.
(Edited)
Photo of Goatreich Sturmkaiser

Goatreich Sturmkaiser

  • 12 Posts
  • 2 Reply Likes
Same boat as me, I'm a webmaster and for the past 2 weeks I haven't been able to work on my websites at all, Everytime I FTP in to my site to upload, about 1/8th of my files end up corrupted and need reupload....  

And surprise surprise, I'm over my data cap again, and I'm left with a plethora of unfinished projects, broken pages I can't make public, I'm losing money, and everyday Exede continues to suck a fat one, I fall further behind on what I need to do. 
(Edited)
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
hello Goatreich Sturmkaiser, do you have SSH access to your server? if so, you might try uploading your files with SFTP which uses the encrypted SSH connection as a workaround. the transfers may fail, but when they do go trough there will be no corruption of the file. 
Photo of Goatreich Sturmkaiser

Goatreich Sturmkaiser

  • 12 Posts
  • 2 Reply Likes
4th night in a row trying to download a 795GB game patch on my PC.... 9 tries, 2 fails, 7 CRC failures, after the 4 attempt the download is throttled to 52kps...  Tried updating an Xbox One game, 1.7GB's, Downloaded on the first attempt, max speed 700kbs, installed, and game does not load do to file error, need to deleted the entire 28GB game, reinstall, and apply all updates again...  So essentially I cannot play this game anymore until YOU fix YOUR services.

Didn't even attempt to upload my work files, as the past few nights have given me nothing but img and html file corruptions. Can't fix all the broken pages I've already uploaded before noticing the problem. 

Stayed up like a Vampire.... AGAIN.... to accomplish nothing....

Thanks Exede
(Edited)
Photo of Goatreich Sturmkaiser

Goatreich Sturmkaiser

  • 12 Posts
  • 2 Reply Likes
Modem Version  UT_2.2.3.0.13
Router WNR2000v3
MTU 1490

Problems persist with or without router and no matter how many times I reset the modem.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Hi Goatreich,
I have a question, if you have contacted Exede through exedelistens@viasat.com 
did they mention a way to roll back the firmware?

Sure would seem to be the best test to determine if the problem in in the newest update.

I know Lab has been on top of this, but not sure if he has heard of a rollback possibility.
Photo of Goatreich Sturmkaiser

Goatreich Sturmkaiser

  • 12 Posts
  • 2 Reply Likes
Last night LNFZ speed seemed to only be active on some pages/files and not others. 

For example I was on Youtube, most videos would autoload at 144p and would play for about 5 seconds before freezing, buffering for 2-3 minutes, play 5 seconds, buffer again. This is exactly what happens when I try to watch Youtube during the day while over my GB's......  Then I would find one magic video here or there that autoloaded at 720 and played flawlessly without interruption.

My average page load time while over my allotted GB's is 12 seconds (UGH!), LNFZ would speed up 90% of the pages I view, but 10% are still being restricted to slow speed, taking 12s to load.

------------------------------------------------------------------------

Here's where things get really confusing.... I download a weekly 4hr long radio show in MP3 format, on average it's a 150mb file. I try downloading from the offcial website, and it starts downloading at my restricted speed of 30kbs.... so I look for the file elsewhere, downloading from various file hosts, and even torrents... all of them restricted to 30kbs. All the sources I tried are hosting the same file as the official site with the exact same file name. Then I find one torrent where the filename was altered from the original, suddenly it's downloading at 2MBS.

So now I have my radio show to listen to today while working, but I'm noticing the ongoing corruption issues while listening. There are odd echoes here and there, and on occasions there are places where the audio jumps to a different part of the show for a second or two. 

For example, you're listening to a comedy sketch at the opening of the show, and then suddenly 2 seconds of a phone conversation (that happens in 2 hours later) cuts in for no reason, then you're thrown back into sketch.

------------------------------------------------------------------------------

And as far as I can tell from the maps I saw here, I should be on Beam 321
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Hi Goatreich,
"2 seconds of a phone conversation" Now that is strange.

With the reports of the latest update being related to this issue, I wounder if there is a filtering problem with Exede Voice and Exede Internet in the big can in the sky or the servers on the ground???

You mix zero's and one's 0010110 with Hello Mommy! And you have the next hit show for HBO.

More shielding on the coax Scotty! and full speed ahead.
Photo of Goatreich Sturmkaiser

Goatreich Sturmkaiser

  • 12 Posts
  • 2 Reply Likes
It's not 2 seconds of a random phone conversation... it's a phone conversation that's a part of the show.  Let's say the the phone conversation takes place 2 hours deep into the show, but 10 minutes into listening to the show from the start, 2 second long blips of that conversation cut in on what's currently happening "music, comedy bit, dj talking etcetera.

it happened about 25-30 times in 4 hours, where 2 seconds of something from a different part of the shows timeline, just randomly inserts itselfs out of place.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Goatreich,

You've indicated above you're on beam 321 and also have the suspect modem software. From the Wildblue World forum, 321 has been identified as one of the problem beams. Your gateway is Lovelock NV, and Accelenet server is likely Salt Lake but you can verify the latter using tracert as noted below (same as mine except for beam)

slowBill indicates he received a message stating the problem has been escalated to engineering. Hopefully relief is on the way sooner rather than later.  
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Same issues being reported at these links (as well as others and all involving the latest software update):

https://community.exede.com/exede/topics/is-exede-injecting-random-data-into-compressed-or-encrypted...

https://community.exede.com/exede/topics/im-paying-for-a-service-im-not-receiving-and-im-no-longer-hesitant-to-involve-the-law

http://www.wildblueworld.com/forum/showthread.php?9164-Corruption-when-using-Router

I may (or may not) have worked around it with some MTU tweaking as described in the 1st thread, and suspect not for reasons mentioned there since I didn't tweak everything yet it seems to have abated on everything.

In addition to modem software version, including beam number, gateway and Acelenet server also may help to further isolate from my perspective (but Viasat probably knows the latter two simply from the beam number) .
(Edited)
Photo of Steve Frederick-VS1/Beam314

Steve Frederick-VS1/Beam314, Champion

  • 3203 Posts
  • 2063 Reply Likes
Hi Lab,

Just to let you know, I have not had any occurrence of these problems, however, I did not receive the suspect modem software update. I am still on Version UT_2.2.0.2.0.  Hopefully now that the holiday is over, Exede engineers will get on this in a hurry and either fix the software, or at least go back to the UT_2.2.0.2.0 version in the areas that were updated to the seemingly bug ridden version if it is even possible.

Thanks for working on this for all those who are having these problem.

Steve
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Lab, I am on Beam 329. how do I find the Acelenet server?
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
slowBill,

Sorry I missed this one, but surprise, surprise, surprise...

I'm on 329 also and it should be -  Lovelock NV Gateway, Salt Lake City Accelenet Server - as is JEP.

There's a way to tell but I have to dig into the way back stacks for it.

More commonality for the issue - my guess is that the modem update was rolled out first and once rolled out to everyone on a beam - some server changes were then rolled out to take advantage of that modem update and that's where the problem lies (or perhaps both).

Still persisting here regardless of MTU tweaking (although it does appear to lessen the frequency of occurrence). I'm guessing that while Everett has the modem update his server(s) haven't been updated. I had the update for over a week before the errors started.

P.S. Just no hit it again with the Exede Internet Image and header on this page showing SSL Connection Error.
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
slowBill (or anyone else for that matter),

Acccelenet server can be determined by doing a tracert www.google.com at the command prompt. The first result containing a recognizable city is usually it (typically the 5th or 6th line) - in my case and likely yours Salt Lake.

See the following for clarification:
 
http://www.wildblueworld.com/forum/showthread.php?9166-Service-has-been-bad-for-about-a-week-now&...
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Still experiencing issues and suspect the caper went down like this:

  • New modem software rolled out to selected beams around 12/16 consisting of the Accelenet client.
  • After being rolled out to those beams and around 12/25 changes to Accelenet server(s) servicing one or more of those beams were rolled out.
  • Problems started appearing shortly after that.
The problem with client/server technology in synchronizing change is often which comes first the client or the server, and what do you do when one or both are bad?

In this case seems pretty simple, roll back the server changes and we get a Christmas mulligan... but I could be wrong.

 
(Edited)
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Hi Lab,
My tracert returned this:


What is the ping to; edge2.losAngles9.Leve13 ??? Several timed outs also?
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Not sure but beyond your Accelenet server of Phoenix, which I;m betting hasn;t been updated yet explaining why you aren't having the problems. Don't worry about the timeouts.        
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
thanks guys for commenting. I have sent my contact info as requested. unfortunately I am only at my property with the viasat on the weekends, so I will only be able to do very limited testing remotely when viasat's technical department contacts me.

I am an electrical engineer and embedded software programmer and have had very little problem with my Exede  service for the last ~2 years and have been largely pleased with it.

Lab Rescuer, I had set my router MTU from the default 1500 to 1492 which did seem to reduce the frequency  of the corruption.

I will post back with any progress or info I might obtain.
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
slowBill - When I was having corruption issues, I had to set my MTU to 1490 based on the  Ping Yahoo.com -f -l 1472 test method.  That totally fixed my 100% repeatable corruption download errors.  Now, FWIW, I played with setting my MTU back to AUTO on my router and it doesn't make any difference.  Me thinks there are other factors going on in the Exede network.  Hopefully they will get the wrinkles ironed out.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
I just noticed that the hyper link we have been using to find the beam map's is dead.
https://www.wildbluetools.com/content/dealer/email/Beam_map-high-mid-low.html

Do they think we are too stupid to handle that much information? Is there a new beam map?
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
It went dead a few days ago, even had one of my posts removed for mentioning it.
Photo of Steve Frederick-VS1/Beam314

Steve Frederick-VS1/Beam314, Champion

  • 3203 Posts
  • 2063 Reply Likes
It appears that it was only for dealer's use, and not meant for public viewing. I think it was a great help for all of us, but from a ViaSat standpoint, it was giving out info to their competitors about their customer base.

It was good while it lasted.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
I wounder if it was related to the rant we (I was the loudest) started about getting access to the installers backdoor to diagnose the modems.

And Lap, why would they delete your post? Asking questions is what the forum is about, right?

And Steve, I bet Hughes or anyone one else who wanted the customer base info could pay to get much greater details than those maps showed.

Kind of discouraging. I don't mind trying to help people, but if my responses are going to get throttled down, or overly monitored, am I limited to the normal response?

"Are you setting at your computer right now? Is it on? Have you rebooted your device?"
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Probably consider it proprietary.

As far as the other questions, no, no, and no.

FWIW, everything lasts on the internet at least partially - Internet Archive Wayback Machine.

I'm holding my copy hostage until I get a software update... or rollback to everyday low prices.
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Everett - They deleted my post where I had included the map image as a graphic.  They sent me an email telling me they were doing it because it was deemed information for dealers only.  Steve is probably right, it gives too much info to the competition, though I think the cat is out of the bag for that one.  But, I guess the map will be updated as time goes on.  Kinda like cell phone companies use to be about their cell towers.  Didn't like to give out locations of the towers.
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
is this the map you are talking about?
http://www.northlc.com/_images/beam_priority_map.png
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Bingo!, Bill.
A little different, but the added info for the Gateways if helpful.
Thanks for the link.
Everett
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Bill - No, the one I was talking about was the one that showed the coverage for Freedom Plans, 2X & 3X data plans as well as the areas that were closed/restricted for new customers.  It would be my assumption that the beam map for your link is OK.  But, standby and see what happens. :-)
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
a bit more information, and some speculation.

doing a file compare of the original with the FTP uploaded file, I see single bit errors spread occasionally throughout the file. no displacement, just a single bit flipped here and there.

an easy way for someone to duplicate the problems affect on HTTPS  is to use Gmail's web interface, create a new e-mail, and attach a 1 or 2 meg file. as it uploads to the server it will be a bit jerky and may error out. repeated retries may get it to succeed.

the fact that the TCP checksum is good for the corrupted packets leads me to think the corruption is being introduced during a compression or aggregation process that is being applied by Exede's equipment.

this problem is obscure enough that's it's reasonable for them to have missed it in testing, but but my money is on this being just the tip or the iceberg as more people encounter it.     

     
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
slowBill,

I've pretty much concluded the same with regard to corruption issues, particularly when you, JEP and I are on the same beam, gateway server and Accelenet server (see belated replay to one of yours above) - suspect it's an issue between the Accelenet client (embedded in the modem) and Accelenet server that does all the compression/packaging magic. Also agree on the obscurity part.

Still getting the corruption errors... less frequent but still persisting (albeit manageable).

P.S. And more are experiencing it or it least taking notice. See:

http://www.wildblueworld.com/forum/showthread.php?9193-Very-unhappy-with-UT_2-2-3-0-13

P.S. If the only official response from Viasat is going to be email exedelistens@viasat.com, might as well just replace this forum and the other with a simple form that only does just that. Like I said much earlier, the longer it drags on the more likely I go ballistic, when a simple "yes we are aware of problems with this software update and are working to correct them with an ETA of ???" would suffice. The frustration stems not from the problem itself (stuff happens) but rather failure to acknowledge a problem exists.

Nevertheless, I'm pretty confident it will be corrected... as soon as the pointy-haried boss steps out of the way ;) 
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Off hand,  the fact this conversation as well as some of the associated one's have now been tagged as Acknowledged might be the best we're going to see with regard to acknowledgement.

But I did just receive an email from someone purporting to be the pointy-haired boss:

"We have permanently altered the idea of research and development. We invariably cultivate reality-based iteration. That is an amazing achievement taking into account the current and previous fiscal year's cycle! It comes off as mixed-up, but it's accurate! We will revalue our aptitude to generate without diminishing our capacity to repurpose. We believe we know that it is better to synthesize virally than to embrace magnetically. Imagine a combination of PNG and C++. It seems unclear, but it's completely true! The capability to mesh globally leads to the capability to optimize strategically. We will multiply our aptitude to innovate without reducing our ability to upgrade. The power to strategize compellingly leads to the aptitude to harness dynamically. The paradigms factor is visionary."

I'm feeling much better now...

          
(Edited)
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Hey Lab, we simply mark a thread as acknowledged when a mod has responded to the original post. I was not aware of this issue until I looked here last night before I left the office. I am waiting to hear back from our Engineering team in regards to what is being done to resolve these issues etc. I did not want to post officially without backing.
(Edited)
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Hello Kimberly, thank you for your efforts.
Bill 
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
the beam is on the contract we signed at installation.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Bill, that is some good info. I sure do not remember getting anything like that during my install. Do you have it in hand, or from an online source?

Kimberly, it would help all concerned if we could read/ see the information coming back from the Engineering team. Like Lab has mentioned many times, if the fix was found, please let us know so we do not waste time digging further into the problem.

And the email Lab got back, that is some very deep gibberish. When I paste that into Google, it returns: We are working on it.☺
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
I think PHB used this: (or was it me?)

http://www.andrewdavidson.com/gibberish/

and I'm going to start using it for all replies now - well at least some if you know what I mean ;)
(Edited)
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Everett, I have a photo of the filled out paperwork we signed. that is a snip of the site information portion. the rest is mostly contact and billing info.

I just received word from ExedeKarmin that the issue has been escalated to engineering. 

who/what is PHB? someone in Colorado probably? 
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Bill, I did mine electronically, the detail are floating in a cloud somewhere I guess. maybe my little friend Kimberly can find it???♥♥♥

Lab, that would be a good one. You could even run for office:
If I am elected "insert paragraph of gibberish here" there will be a chicken in every pot pie!

(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
PHB: Pointy-Haired Boss - Everet doesn't take over Viasat until hitting the mother lode in early 2016 and then we call him HCB - Hairy Chinned Boss ;)

P.S. This is one place we NRTC susbscribers have and edge - all of that information is displayed on our account pages (including gateway, installer, etc.).
(Edited)
Photo of John_in_MO

John_in_MO

  • 3 Posts
  • 2 Reply Likes
Same problem here. Same firmware UT_2.2.3.0.13. Looks like we're on beam 316.

First attempt at submitting this failed due to the problem.

Symptoms for us include not being able to load web pages, sometime only parts of pages, corrupt images and PDFs, etc.
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Here's hoping that Microsoft and other developers are smart enough not to update their software on our computers with a corrupted download.
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
and here's hoping Exede can still push new MODEM updates without bricking them because of corruption. ;-) yeah I know I'm just a worrywort.
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
So small favor from everyone and anyone having these issues...not to sound moderator cliche...but..please send me your info to exedelistens@viasat.com. I am collecting customer info to send to our Engineers. Thanks you guys.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Thank You Kimberly.

And in turn, please post back any info you do find on your end. There are several forum members who are attuned to the issues and might provide additional help, and or testing on the consumers end.

I know in a separate post you said, "I will not post with out backing from the tech engineers" and that is fine. But do post some of the more technical details you are provided, rather than "they are working on it".

It's becoming a hard sell when we try to provide help to disgruntled members.

Thank you again for your continuous help and perseverance under adverse conditions.
Everett
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
You got it Everett. I will do my best to hold down the fort!
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Kimberly, these are the user names I seen on this forum that seem to have experienced the corruption problem. not sure if this is of any value though.
And BTW you rock.
slowBill
David Culver
John
Dorothy Allen
Goatreich Sturmkaiser
Lab Rescuer
John_in_MO
bugzeeolboy
JEP
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Thanks slowBill this is proving very helpful! Already took down some info from your list.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
I'd send an email to that address but as an NRTC subscriber from past experience you're not supposed to help me.
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Ok thank you Lab, I'll take the needed precautions. (:
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
im having this exact same problem, i have upload/download speed okay, however, my upload packets are corrupted, and my dedicated servers at the receiving end,  disconnect me, ..  my ping tests..  http://www.pingtest.net  has been in the 650ms range.!!!!!  this is why my packets are corrupt..  too much lag time,  and the packets are getting out of sequence,  this all started for me, on Dec 21st, since than, ive done everything in the book to remedy the issue from my end, ive exhausted every possible reason from my hardware, router, switches,firewall, antivirus, even had Geeks Squad out here, because it couldnt be my IPS!!!!  ive even brought in a laptop, hooked it up to viasats  modem directly, and get the same packet speed, ((((  ive called in and have a tech support person coming on friday , i hope he can solve this, or im going to have to shop for another provider ... 
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Smokn, that ping time is normal for satellite internet (speed of light and distance up and back twice)   please send Kimberly and e-mail with you contact info, per her request above, so she can add you to the list. 
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
thanks SlowBill, i sent her an email, i also noticed my IP# has changed 3 times, in the last 5 days..  tracing each IP, ive gone from California, to unknow USA, to.. now.. Washington State,
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Thanks for sending in your info guys. I will be sending this list off tonight by 5:00pm CST and will update here accordingly. SmoknWeed, the IP can change locations quite often as we have a Static IP.
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Also, if any other users want to send in their info after 5:00pm CST, I can add your info to the list tomorrow when I arrive back at the office at 9:00am. CST. Ok I am going on lunch now so I'll be MIA for about 30 minutes but like Arnold said, "I'll be back.".
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
Hello Smokn, thanks for posting and sending your contact into to them. although my IP has largely remained the same, it did change at least once a few days ago. this may or may not be related to the MODEM software update recently. do you know what MODEM software revision you have? do you know what beam you are on? 
this map mey help you find your beam if you do not know it.
http://www.northlc.com/_images/beam_priority_map.png
Photo of Alex

Alex, Viasat Corporate Communications

  • 675 Posts
  • 305 Reply Likes
If you're having modem issues, please email me your MAC address, account or service phone number. It'll help the engineers who are testing this issue in the lab. Thanks!
socialmedia@viasat.com
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
Hi Bill, my IP has been the same, for at least ...... 8-10 months , than all of a sudden, in the last week as i said before, it started changing, 2 more times.. no, i dont know, what beam im on, i live in Sandy, Oregon, looks like a couple of them overlap in that area, according to the map you furnished here..  its either B367/B368, ...  but my IP tracert to Washington at the moment..  hmmmmm
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Smokn,

The key variable with the issue slowBill, myself and others are having is the ViaSat modem's software version which can be determined at:

http://192.168.100.1/?page=modemStatus

A software version of UT_2.2.3.0.13 is the suspect version. 

IP addresses are subject to change at any time, and static IP address are only available to business accounts (at last check). More often than not mine shows as being in Colorado, Your ping time is better than most and can't get much lower due to physics on satellite.   
  
 
(Edited)
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
Thanks Lab Rescuer, wow.. thats really cool to see that., thanks, and   yes,  Software Version UT_2.2.3.0.13 is what is showing on mine as well,
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
i do also notice my modem, goes thru a "rebooting" about 10 - 15 times a day, the only light that stays on, is the power light, the rest go out, than it starts its cycling again, of sending packets out , packets in... 
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
SmoknWeed question for Lab and Bill.

You guys are a lot tighter into the details of this issue, and I have read and watched some (70% + ) of the people with the problem's (excepts me = CO) are located along the coast, East and West.

Another thing that is associated with these areas is the "Freedom Plan". When Lab mentioned the tracert  deal/ test, I hit Phoenix, AZ. I have the update, but as Lab said, I might not have the "Server" update. Why? Because my area/ beams do not have the Freedom Plan??? I don't know? Could it be associated?

You guys and others in VA. Smokn hit both coast with the tracert returns. I base a lot of the beam/ maps details on the one JEP had up, and now is gone or relocated???

Their where just more color's and overlap of ongoing plan's and specials, follow my drift? More action = different DB/ server fields. But if the standard issue modem update does not allow for standard and special code/ server/ modem. The bubble burst. Standard server + update = OK. Special server + update = Bad.

Maybe I have been smoking too much, but I just thought I would offer an idea?

Have ya ever made a Bong? Well, 1st ya take your Pringles can, and then you aaa....
Everett
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Hi Smokn,
The re-boot issue. that's not right at all. On top of your other issues.

When you enterd this link to your modem, what did your settings look like?
http://192.168.100.1/?page=modemStatus

Do you have any red checks on your TRIA page?

Are any of the sliding setting graphs on the modem page bouncing left and right? Do things look steady? Are you having any bad weather, snow/ rain?

Just an idea,
Everett
Photo of John Horsman

John Horsman

  • 4 Posts
  • 2 Reply Likes
I added my name to the Email. I put my network behind a proxy (Squid) to see what type of errors were coming across. Errors I were getting were the following: Per Mozilla (Definitions are referenced from https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/SSL_functions/sslerr.html )

SSL_ERROR_BAD_MAC_READ -12273 "SSL received a record with an incorrect Message Authentication Code." 


This usually indicates that the client and server have failed to come to agreement on the set of keys used to encrypt the application data and to check message integrity. If this occurs frequently on a server, an active attack (such as the "million question" attack) may be underway against the server.


SSL_ERROR_BAD_BLOCK_PADDING -12264 "SSL received a record with bad block padding."


SSL was using a Block cipher, and the last block in an SSL record had incorrect padding information in it. This usually indicates that the client and server have failed to come to agreement on the set of keys used to encrypt the application data and to check message integrity. If this occurs frequently on a server, an active attack (such as the "million question" attack) may be underway against the server.

Been going for a couple of weeks now. With the prior firmware and the new one. Ended up getting Dapped in the process. But decided to cut the plan instead from the 25 gig a month to the Evolution plan so I at least have Web Access since it made LNFZ useless and I had to be there to monitor and have it reattempt.
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
Everett, ...no, when i look at the modem status, it all seems okay, nothing in red, only the Rx SNR, and Rx power seem to move, but only slightly, 1/8" back and forth, but not violently, .. and all the status are green check marks, on the TRIA status page
(Edited)
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Smokn, Did you click the "TRIA" tab of the left side. It has separate readings. That is where the red check's might occur.
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
For Bill and Lab,
My TRIA page; "TRIA Firmware Version18.20". Yours???

Just 2 more cents,
Everett
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
Everett. yes i did,   Bullfrog VG , Cable Resistance ,Watchdog Timer ,Microprocessor BIT ,Tx PLL ,Rx PLL , Temperature and i have version 12.02
(Edited)
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Running TRIA Firmware Version 13.05 here.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
TRIA at 13.05 here, but don't recall it ever changing - was under the impression it was ties more to TRIA date of manufacture (or possible recondition) - also recall from somewhere that newer systems have a newer TRIA.

P.S. Seeing much more reliability in terms of speed during LNFZ this AM - while some fluctuation that is to be expected, not as wildly fluctuating as they have been. At the same time no corruption errors since sometime yesterday - perhaps "we're done working on it"... perhaps not.

P.P.S. Everett, I'm going with option B: Server update and modem update = issues. Particularly if "we're done working on it" since modem version hasn't changed, but I'm just knocking wood.

P.P.P.S. Everett the Pringles can is apparently gaining rapid acceptance and wide-spread adoption. I just called NRTC support and their standard script now starts off with: "Have you removed the Pringles can from your TRIA?" - should have patented it. Remember to do so with Pringles Can 2.0 where you line it with gold for even greater reliability - Monster Cables is interested in a buyout.
(Edited)
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Lab - Yeah, there is a newer TRIA called the eTRIA.  It is lighter and mounts on a single tubular boom with the coax routed inside the boom.
My TRIA is reporting a temperature of -10° this morning.  The coldest I have ever seen it.
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Mines just sitting at 10° right - usually it's high enough to warm my coffee.    
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
good morning guys, my TRIA is rev 13.05 and 20 deg this morning.
Lab, maybe fixed? do you use gmail? does attaching a 1 or 2 meg image upload every time now?
I will not be at my place with the exede until this weekend, but I just asked my son to try thing out. I will let you know if things seem to be working
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
Very Good Morning SlowBill, after reading your above text, about  "maybe fixed" and i hope im not jumping the gun, i ran out, and attempted a few uploads.. to my dedicated business servers, ... and... low and behold...!!!   BINGO!!!  it worked without a hitch.. i uploaded, a couple of  very large files, without a single packet out of place.. or at least my servers think so.. because, i didnt get any error referencing "Packet corrupt" , i will monitor the board here.. and see if  engineering or a rep steps up,  with confirmation that the "bug" was located and fixed... :-)  and PS: yes, i canceled my requested service call )
(Edited)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
slowBill,

Appeared fixed this early AM and didn't have any issues when normally I expected them. Reports on the other forum indicate possibly fixed also. Hopefully some formal acknowledgement forthcoming...

Tough t tell from where I'm at right now since I'm squatting on an open FIOS connection.
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
yes, things are looking up at my end too. My son reports no errors so far this morning as well. I really hope we hear from excede that they found a solid problem and fixed it.  
Photo of Alex

Alex, Viasat Corporate Communications

  • 675 Posts
  • 305 Reply Likes
This problem in the SLC core node was ID'd and fixed last night. You should be seeing normal performance. It not, let us know!
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Thanks, Alex.
Photo of slowBill

slowBill

  • 70 Posts
  • 23 Reply Likes
man don't you just hate it when your corenode is acting up ;-)
thanks for the update Alex,
do you have any additional information? was it hardware, software, related or not to the ongoing UT_2.2.3.0.13 update or it's associated back-end changes?
(Edited)
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
i did notice, when i look at my modem status,  that my modem was "remote rebooted" approx 10 hours and 20 minutes ago..  iyou may find the same thing, on your modem status :-)
Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Thanks to everyone for sending in their info!
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Thank you Kimberly and Alex.

That is the kind of correspondence and internal forum responses we all need to prove Exede is responding to issues form the customer base.

Keep it up as needed, this fairly long post has been nothing but positive. The best one I have read. Many different members came together with great responses and personal effort.

Bravo to all !
Everett

PS: Question; Was it the Pringles can that fixed it? That was all my idea. Mine !
Photo of SmoknWeed

SmoknWeed

  • 10 Posts
  • 4 Reply Likes
to respond to your previous mention of the Pringles can...   been there.. done that !!
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
Great effort!  Glad the problem was resolved.  Now things can go back to normal, with complaints about data usage. ;-)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Exactly, let me start it off... I was promised 10GB per month, I'm not getting them and feel cheated. What are you doing with my bytes, Kimberly? 
 
Photo of Steve Frederick-VS1/Beam314

Steve Frederick-VS1/Beam314, Champion

  • 3203 Posts
  • 2063 Reply Likes
Maybe she is giving your unused bytes away in the GGG.  No, Kimberly is too honest to do that. Should you get out that Pringles can and store your unused bytes in it for future use?
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Unfortunately, with the issues here out of the way, I've re-purposed the Pringles can as a Kaleidoscope  - either that or I'm having a 60's flashback.

Nevermind, I'm going down to the Virgina ABC store to file a formal complaint with Mr. Cuervo, see if I can get some more Pringles while there, and see if he'll come home with me.    
Photo of A. Everett Neuman

A. Everett Neuman

  • 430 Posts
  • 303 Reply Likes
Hey Guy's, I just got done watching Amazon movies for 8 hours straight, and it didn't cost me a single Gig.

I was down to about 5 Gig's this morning, and then the meter acted up due to the new update fix. It showed 14.8 gigs. 

So I went to the movies. Added about a half pint of Jose Chuveiro into some blue cheese dip and mixed. It took 3 P cans of chips to finish.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Everything fine here since I was mostly offsite yesterday. Thanks again to Kimberly, Alex and others who helped isolate the problem.

You are free to move about the internet cabin... the seat belt light is off, but beware any unexpected turbulence.
(Edited)
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
We are running fast and error free this morning also.  Good to be back to normal!  Great job on the trouble shooting, Exede peoples!

Photo of Exede Kimberly

Exede Kimberly

  • 879 Posts
  • 202 Reply Likes
Thanks to you guys as well! You keep my job interesting to say the least!
Photo of John_in_MO

John_in_MO

  • 3 Posts
  • 2 Reply Likes
Hi Kimberly and everyone else!

Happy to hear that at least some of you are having some success.

Unfortunately, I can't say that things have improved on this end. Our connection is still dropping a lot. I haven't determined yet that there is any improvement in the data corruption.

Anyone else still having issues?

Thanks to everyone working on and reporting on the problems. etc.

Do any of you have a good recommendation for a utility to use to log connection up time?

Thanks again,
Photo of JEP

JEP

  • 987 Posts
  • 718 Reply Likes
John - It is best to start a new thread if you are bringing up a new problem. But, can you provide a little more info on your connection dropping?   What are the indications that you have lost connection?  How long is your connection down?  What are the lights doing on your modem when you have lost connection?  What is the value of your Rx SNR when your connection is good?  http://192.168.100.1/?page=modemStatus
Photo of Alex

Alex, Viasat Corporate Communications

  • 675 Posts
  • 305 Reply Likes
@slowbill: it was 'data corruptions of RL packets from UTs" (user terminals/modems)
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
John_in_Mo,

I'll echo JEP above in starting a new post particularly since this one as well as other related ones are now tagged as Solved.

Above in one of your posts you indicate your on Beam 316, and I don't recall that particular beam being mentioned either here or elsewhere on the other forums (and I was tracking this one pretty closely).

Regardless, you may want to do a tracert google.com (from the Dos command prompt assuming Windows).

The first recognizable city (generally the 5th or 6th hop) is an indicator of your Accelenet server - everyone reporting here and elsewhere was going through Salt Lake City.

P.S. While these errors were sometimes referred to aa connection errors in the several threads here and elsewhere, they technically weren't connection errors but rather data corruption errors exhibiting a number of specific error messages. Specifically, the connections were fine, the data traveling along those connections wasn't in too good shape.  
(Edited)
Photo of Steve Frederick-VS1/Beam314

Steve Frederick-VS1/Beam314, Champion

  • 3203 Posts
  • 2063 Reply Likes
John, the connection time is shown on the modem status page in the upper right. That will tell you how long the modem has been connected to the big shiny thing in the sky. If you have numbers in the minutes or hours, then the modem has had to reestablish communication. If it is showing days or even lots of hours, then it hasn't disconnected, and your problem is likely with your router or computer.

As the guys above have said, start a new topic, and send an email to the moderators at exedelistens@viasat.com explaining what you are going through. They will help you so much quicker than if you call phone support, or even use the Exede website to ask for help.

Good luck. 
Photo of John_in_MO

John_in_MO

  • 3 Posts
  • 2 Reply Likes
Thanks, guys. I didn't realize that you declared that this problem was solved for some of you. I'll open a new thread with the same info to re-open it for those of us still experiencing the corruption problems. Glad some of you are up and running. Hope the rest of us will be soon too. Cheers!
Photo of Steve Frederick-VS1/Beam314

Steve Frederick-VS1/Beam314, Champion

  • 3203 Posts
  • 2063 Reply Likes
We do not have the ability to declare problems are solved, that is done be the moderators who are Exede employees. We are just customers who try to help out with issues that people are having. If you are still experiencing problems, do start a new thread and the moderators will see it when they come in tomorrow.
Photo of Old Labs (VS1-329-L12FZ)

Old Labs (VS1-329-L12FZ)

  • 4281 Posts
  • 4367 Reply Likes
Good luck John-in_MO,

It took a sustained, coordinated effort (or a "ruckus" as someone else said) both here and in the Wildblue World forum to narrow the problem to the SLC server and UT_2.2.3.0.13) and get it fixed - it's logical to assume it may have been extended to other servers unless it was solely a hardware malfunction.

Note that when doing a tracert, Salt Lake City isn't being displayed anymore but rather it appears airport codes are being used (SLC in my case but that may be temporary).  As Steve states, we only declared it as a problem - not solved.
(Edited)

This conversation is no longer open for comments or replies.