Do you have adblock enabled?
If you can read this, either the style sheet didn't load or you have an older browser that doesn't support style sheets. Try clearing your browser cache and refreshing the page.

(Level 3)   Verizon gets biatch slapped by Level3 over Netflix throttling   (blog.level3.com) divider line 78
    More: Amusing  
•       •       •

5385 clicks; posted to Geek » on 18 Jul 2014 at 1:21 PM (27 weeks ago)   |  Favorite    |   share:  Share on Twitter share via Email Share on Facebook   more»



78 Comments   (+0 »)
   
View Voting Results: Smartest and Funniest

First | « | 1 | 2 | » | Last | Show all
 
2014-07-18 11:15:38 AM  
And yet, nothing will change.
 
2014-07-18 11:20:34 AM  

RedPhoenix122: And yet, nothing will change.


This.
 
ZAZ [TotalFark]
2014-07-18 11:52:35 AM  
But his attempt at deception has backfired

And that's where I decided this post was about as credible as a corporate spokesman.
 
2014-07-18 11:59:13 AM  
except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared
 
2014-07-18 01:20:19 PM  
"And I would have gotten away with it, too, if it weren't for you meddling bits!"
 
2014-07-18 01:22:11 PM  
Slashdot had this to say about it:

"Verizon has blamed Netflix for the streaming slowdowns their customers have been seeing. It seems the Verizon blog post defending this accusation has backfired in a spectacular way: The chief has clearly admitted that Verizon has capacity to spare, and is deliberately constraining throughput from network providers. Level3, a major ISP that interconnects with Verizon's networks, responded by showing a diagram that visualizes the underpowered interconnect problem and explaining why Verizon's own post indicates how it restricts data flow. Level3 also offered to pay for the necessary upgrades to Verizon hardware: "... these cards are very cheap, a few thousand dollars for each 10 Gbps card which could support 5,000 streams or more. If that's the case, we'll buy one for them. Maybe they can't afford the small piece of cable between our two ports. If that's the case, we'll provide it. Heck, we'll even install it." I'm curious to see Verizon's response to this straightforward accusation of throttling paying users (which tech-savvy readers were quick to confirm)."
 
2014-07-18 01:30:48 PM  

zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared


Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.
 
2014-07-18 01:37:02 PM  

zedster: Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared


But Level 3 has offered to pay for the hardware upgrades. To continue your analogy, Level 3 shows up with a firehose for them and a firehose for Verizion. Verizion says no, pay me first. Why?
 
2014-07-18 01:38:32 PM  

rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.


Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.
 
2014-07-18 01:38:52 PM  
This post does a better job than most of representing the problem.  People tend to think Verizon has everything set up and is just turning down throughput for fun, but this at least correctly shows that new hardware needs to be added at the interconnect points.  They're fighting over who pays for that hardware and its ongoing maintenance costs.

I have some problems with Mr Taylor's handwaving that it's "just a few thousand dollars" and offer to pay for that.  If he was truly interested in picking up the whole tab, there wouldn't be a problem.  Two big problems with his claim:
- he's only considering initial hardware purchase cost.  Ops and maintenance are quickly going to surpass that cost.
- he's talking about "a few thousand dollars" to get necessary hardware installed in Los Angeles specifically.  ok, so add a few thousand each for every other point of congestion and pretty soon we're talking about "a few million dollars".
 
2014-07-18 01:48:57 PM  

MightyPez: rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.

Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.


Yeah, big corporations take a while to move between signing a contract and implementation.  Especially with something as big as the Netflix deal.  It's not as simple as flipping a switch.

It was kind of assy of Netflix to start pointing fingers at Verizon during the interregnum.  They like to play that card while deals are being negotiated, because bad publicity for an ISP makes them more amenable to a deal that's favorable for Netflix, but it's strange to see them pull a move like that after the deal's been signed.  Maybe there's more deal in the works.

From Level3's perspective, I'm sure they'd love it if Verizon made a bunch of infrastructure improvements to handle the Netflix traffic that L3 won't even be carrying anymore in a few months.  Puts them in a good position to sell all that unused bandwidth to somebody else.

/ The thing to remember about the VPN argument is that your VPN traffic takes an entirely different route through the network.  Being suddenly faster on a VPN isn't a result of Verizon not being able to do deep packet inspection; it's just that the VPN provider is going through a less congested route.  i.e. suppose my normal route looks like Me <- Verizon <- Level3 <- Netflix.  Everybody agrees that Verizon <- Level3 is the bottleneck here.  VPN traffic might look something like Me <- Verizon <- My VPN <- Comcast <- Netflix.  Cuts out the relevant middle man.
 
2014-07-18 01:50:53 PM  

MightyPez: Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.


That's what I thought.  So it's just a matter of Verizon taking its time to set up the new system.  I'm sure there are stipulations in that contract regarding when the system has to be operational so if Netflix wanted it done faster, they should have asked that it be done faster.

The point is still that there's nothing new here.  I'm sure Verizon's speeds will look a lot like Comcast's once the upgrades are done.
 
2014-07-18 01:53:48 PM  

serial_crusher: MightyPez: rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.

Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

Yeah, big corporations take a while to move between signing a contract and implementation.  Especially with something as big as the Netflix deal.  It's not as simple as flipping a switch.

It was kind of assy of Netflix to start pointing fingers at Verizon during the interregnum.  They like to play that card while deals are being negotiated, because bad publicity for an ISP makes them more amenable to a deal that's favorable for Netflix, but it's strange to see them pull a move like th ...


Well, except for Comcast. They seemed to show improvement immediately after the agreement was signed. So Verizon is either incompetent or scummy? Both?
 
2014-07-18 01:54:05 PM  

MightyPez: It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.


That's what happens when you drive around a traffic jam
 
2014-07-18 02:00:33 PM  

MightyPez: So Verizon is either incompetent or scummy? Both?


I'd go with both.
 
2014-07-18 02:00:36 PM  

bhcompy: MightyPez: It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

That's what happens when you drive around a traffic jam


It's absurd that adding extra hops to location way outside your network is going to improve performance. Verizon is dragging their feet and allowing their infrastructure to languish (Just ask their DSL customers) effectively not giving their customers what they are paying for.

I don't particularly like being in Comcast territory, but it seems a damn sight better than being in Verizon territory with these stories.
 
2014-07-18 02:01:18 PM  

Chuck Wagon: zedster: Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

But Level 3 has offered to pay for the hardware upgrades. To continue your analogy, Level 3 shows up with a firehose for them and a firehose for Verizion. Verizion says no, pay me first. Why?


Because the cost of the hardware is insignificant - capital recovery of the VZ network capacity is expensive.

Here's an analogy.  What L3 is trying to say is that the first truck off the production line should cost $500M, because that was how much the factory cost to build.  The second truck should only cost $5000, because that was the cost of the labor and raw materials.  L3 is claiming they are providing the labor and raw materials, so the price they should get is $0.  VZ is correctly saying that price has to cover the LRIC in providing the capacity, not the incremental cost in providing the capital to terminate the connection.  Ie. L3 should be sending VZ a check every month (ie. not a settlement free peer) or Netflix and their traffic should hook up to VZ and pay each month (the deal they cut).  Why VZ is taking so long to build the ports for Netflix's server platform is what is puzzling to all of us, even those in the industry (I mean we're slow too, but it wouldn't have been more than 45 days after cutting the deal that the new circuits would be up - even if we have to deploy new ports on the core - see below for why core network capacity needs to be built).

And the "you have spare capacity" argument is a red herring.  You never run a network above 45% utilization - if you do you can't restore the damn thing when there are outages.  Only time you can run it above 45% is when you have sufficient meshiness between all your nodes - and no national carrier has enough to be able to support that.  There are essentially 4 east-west fiber routes in the US, and most carriers are on either 2 or 3.  Even the one carrier that is on all 4 does not have enough north-south routes to be able to traffic engineer restoration paths around potential city pair outages to run the network hotter than 55%.  And buying/building more capacity on the existing long haul routes does not solve the problem - more fiber routes (read super expensive) are needed, because your restoration paths need to be built to cover cable cuts.  So L3 is lying about there being enough capacity.  VZ would have to augment their network capacity out of that interconnection point if they added an additional 40Gbps of interconnections.  ~80Gbps of network capacity would have to be built towards the rest of their core network so it could remain a restorable in the face of equipment or circuit outages.  L3 is conveniently hand waving there.
 
2014-07-18 02:04:14 PM  

Chuck Wagon: zedster: Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

But Level 3 has offered to pay for the hardware upgrades. To continue your analogy, Level 3 shows up with a firehose for them and a firehose for Verizion. Verizion says no, pay me first. Why?


Free fire hose is great, but would you want to pay that water bill?
 
2014-07-18 02:05:31 PM  

serial_crusher: The thing to remember about the VPN argument is that your VPN traffic takes an entirely different route through the network. Being suddenly faster on a VPN isn't a result of Verizon not being able to do deep packet inspection


I don't think the primary complaint is that Verizon is traffic shaping, I think the primary complaint is that Verizon is intentionally keeping high-traffic interconnects under-provisioned on their end and then pretending it's not their problem. They're trying to pick a fight about who's responsible for paying for enhancements that would only benefit their own customers.
 
2014-07-18 02:05:33 PM  

serial_crusher: - he's talking about "a few thousand dollars" to get necessary hardware installed in Los Angeles specifically.  ok, so add a few thousand each for every other point of congestion and pretty soon we're talking about "a few million dollars".


He also said that his network only connects to verizon in 10 cities. A few thousand x 10 = a few million?
 
2014-07-18 02:09:39 PM  

Otfsentet: He also said that his network only connects to verizon in 10 cities. A few thousand x 10 = a few million?


Obviously I don't what they're using, but a 4 x 10gbe expansion module for a Catalyst 6500 runs about $3800. It's unlikely that the cards for the routers would be much more than that since the routers themselves are likely to be the hardcore hardware doing all the fancy work.

The hardware costs of fixing this problem would be inconsequential. I also have to seriously doubt that it would require any serious addition to personnel or maintenance. Bonding the additional inbound pipes wouldn't require any sort of major changes to the existing configuration on either end.
 
2014-07-18 02:12:04 PM  

MightyPez: bhcompy: MightyPez: It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

That's what happens when you drive around a traffic jam

It's absurd that adding extra hops to location way outside your network is going to improve performance. Verizon is dragging their feet and allowing their infrastructure to languish (Just ask their DSL customers) effectively not giving their customers what they are paying for.

I don't particularly like being in Comcast territory, but it seems a damn sight better than being in Verizon territory with these stories.


No it isn't absurd - just counter-intuitive to anyone but a network engineer.  By using a VPN service, you can bypass the congestion point.  It's a rough way of doing user level traffic engineering.  If the VPN supplier was reached on the same interconnection links, you would have seen the same performance over the VPN.

We'll leave VZ's DSL platform capacity capping for a different discussion (I experience that personally - I have a VZ DSL line as a backup to my cable connection, and I can attest busy hour (8-10pm EST) performance is terrible.
 
2014-07-18 02:13:15 PM  

Otfsentet: serial_crusher: - he's talking about "a few thousand dollars" to get necessary hardware installed in Los Angeles specifically.  ok, so add a few thousand each for every other point of congestion and pretty soon we're talking about "a few million dollars".

He also said that his network only connects to verizon in 10 cities. A few thousand x 10 = a few million?


Only seven in the US where the issues are as the interconnections in the three in Europe are at low capacity.

serial_crusher:
- he's only considering initial hardware purchase cost.  Ops and maintenance are quickly going to surpass that cost.

"Ops and maintenance" on an additional four connections at each location would be almost completely insignificant.
 
kab
2014-07-18 02:16:23 PM  
Guys guys guys.... take it easy on Verizon.

Crying poverty while benefitting from 5 consecutive quarters of double digit revenue growth is certainly a financial situation anyone can empathize with.
 
2014-07-18 02:26:04 PM  

bhcompy: MightyPez: It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

That's what happens when you drive around a traffic jam


Excellent analogy.  Surface streets are usually slower, but when there's a multi-car pile up on the freeway that's blocking most of the lanes, the surface streets can be faster.

Under ideal conditions, VPN can't be as fast as a regular connection because of the overhead.  If VPN is faster, then there's some farkery going on.
 
2014-07-18 02:27:32 PM  

MightyPez: Well, except for Comcast. They seemed to show improvement immediately after the agreement was signed. So Verizon is either incompetent or scummy? Both?


According to this article, Netflix and Comcast were working together for several months before the deal was signed. Netflix and Verizion were not working together at all before the agreement was signed.

http://arstechnica.com/information-technology/2014/06/why-verizon-wo nt -solve-its-netflix-problem-as-soon-as-comcast/
 
2014-07-18 02:28:28 PM  

serial_crusher: Chuck Wagon: zedster: Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

But Level 3 has offered to pay for the hardware upgrades. To continue your analogy, Level 3 shows up with a firehose for them and a firehose for Verizion. Verizion says no, pay me first. Why?

Free fire hose is great, but would you want to pay that water bill?


If I tell my customers that they can have unlimited water for a flat rate, but refuse to upgrade from the garden hose to a firehose, then I would be committing fraud.  Verizon should be prosecuted for this crap. Unfortunately, the FCC is run by industry insiders who are doing everything in their power to protect these assholes.
 
2014-07-18 02:29:06 PM  

Chuck Wagon: MightyPez: Well, except for Comcast. They seemed to show improvement immediately after the agreement was signed. So Verizon is either incompetent or scummy? Both?

According to this article, Netflix and Comcast were working together for several months before the deal was signed. Netflix and Verizion were not working together at all before the agreement was signed.

http://arstechnica.com/information-technology/2014/06/why-verizon-wo nt -solve-its-netflix-problem-as-soon-as-comcast/


In fact, Verizon isn't just refusing to upgrade from the garden hose.  They are purposely crimping the hose to reduce the flow.
 
2014-07-18 02:29:16 PM  

MightyPez: serial_crusher: MightyPez: rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.

Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

Yeah, big corporations take a while to move between signing a contract and implementation.  Especially with something as big as the Netflix deal.  It's not as simple as flipping a switch.

It was kind of assy of Netflix to start pointing fingers at Verizon during the interregnum.  They like to play that card while deals are being negotiated, because bad publicity for an ISP makes them more amenable to a deal that's favorable for Netflix, but it's strange to see them pul ...


Comcast was working with Netflix for months to build the connections before the deal was signed. All they had to do was turn them on. Verizon didn't start building the connections until the deal was signed. That's why it seems like it is taking so long.
 
2014-07-18 02:29:41 PM  
Damn it.  I quoted the wrong one.
 
2014-07-18 02:33:00 PM  

MightyPez: serial_crusher: MightyPez: rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.

Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

Yeah, big corporations take a while to move between signing a contract and implementation.  Especially with something as big as the Netflix deal.  It's not as simple as flipping a switch.

It was kind of assy of Netflix to start pointing fingers at Verizon during the interregnum.  They like to play that card while deals are being negotiated, because bad publicity for an ISP makes them more amenable to a deal that's favorable for Netflix, but it's strange to see them pul ...


Comcast had stopped peering with Netflix and forced Netflix to transit, which means the traffic was leaving Level3/Netflix--> some other carrier--> Comcast. Comcast reached an agreement and the traffic started going Level 3/Netflix--> Comcast. The wiring was there but turned off while negotiations took place over the impasse with the previous parity agreement. Peering (with parity) is free-ish, transit always costs since the packets both enter and leave the networks in the middle
 
2014-07-18 02:55:37 PM  
i had a relatively slow dsl line with speakeasy/megapath, like 5m, and was expensive as hell, but I used to get excellent quality Netflix streaming with no interruptions.  I switched to Verizon dsl cause it was half the price for 15m speed, and now I get half the quality stream, and it's constantly "reloading".  I was home last Wednesday and it was reloading so often it was unwatchable even during the middle of the day (not prime time tv).

is it possible to use the vpn trick using the Netflix app on a blue ray/dvd player?  if not guess I could stream from the laptop and bounce it to the tv.  ugh </lame> , and dsl is my only option where I live..
 
2014-07-18 02:58:37 PM  

RedPhoenix122: And yet, nothing will change.


What will change is when there's an additional $10 service fee from Verizon to get Netlfix and other streaming services in HD quality.
 
2014-07-18 02:58:50 PM  

NakedDrummer: i had a relatively slow dsl line with speakeasy/megapath, like 5m, and was expensive as hell, but I used to get excellent quality Netflix streaming with no interruptions.  I switched to Verizon dsl cause it was half the price for 15m speed, and now I get half the quality stream, and it's constantly "reloading".  I was home last Wednesday and it was reloading so often it was unwatchable even during the middle of the day (not prime time tv).

is it possible to use the vpn trick using the Netflix app on a blue ray/dvd player?  if not guess I could stream from the laptop and bounce it to the tv.  ugh </lame> , and dsl is my only option where I live..


Doubt your player would let you configure it to go through a VPN.  You might be able to do it at the router level though?
 
2014-07-18 03:06:33 PM  

kab: Guys guys guys.... take it easy on Verizon.

Crying poverty while benefitting from 5 consecutive quarters of double digit revenue growth is certainly a financial situation anyone can empathize with.

It's hilarious to watch a company cry about their financial woes after they bought out Vodafone's share of Verizon Wireless for $130 billion.
 
2014-07-18 03:11:26 PM  

ZAZ: But his attempt at deception has backfired

And that's where I decided this post was about as credible as a corporate spokesman.


You should have probably read the whole thing.
 
2014-07-18 03:16:48 PM  

Tobin_Lam: MightyPez: serial_crusher: MightyPez: rugman11: zedster: except it actually shows the reverse, that Verizon is keeping to their peering agreement and that traffic load is equal on both sides. Level 3 wants VZ to accept more traffic, beyond the peering agreement without having pay for it. Netflix used to CDN with Akamai, putting them in more data centers and spreading out their traffic load. Now they are pushing their own CDN, Openconnect, which is free as in razors handles. They will give you the appliance for free but require you to pay for bandwidth, power, and space to host it. They then went with Level 3 to host their servers, a company known for violating peering agreements Level 3 wants to be a CDN but act like a carrier, you can be both but you have to agree that CDN traffic gets treated differently than simple peering.

Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

Exactly, this is the exact same argument Netflix has been having with broadband providers for months, and it will likely end in the exact same way, with Netflix paying Verizon to connect directly to their network.

Eh? Netlix signed the same deal with Verizon that they did with Comcast. But for some reason Verizon is still dragging their feet.

It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

Yeah, big corporations take a while to move between signing a contract and implementation.  Especially with something as big as the Netflix deal.  It's not as simple as flipping a switch.

It was kind of assy of Netflix to start pointing fingers at Verizon during the interregnum.  They like to play that card while deals are being negotiated, because bad publicity for an ISP makes them more amenable to a deal that's favorable for Netflix, but it's strange to s ...


And you need to probably never need to comment in tech threads ever again.

If you use a vpn connection via verizon to another network then the connection is not throttled.  Which means if Verizon can see the traffic, they shape it... If they can't, it is business as usual.

But then again who expects an Apple fan boi to understand the interwebs.
 
2014-07-18 03:31:51 PM  

theflatline: And you need to probably never need to comment in tech threads ever again.

If you use a vpn connection via verizon to another network then the connection is not throttled. Which means if Verizon can see the traffic, they shape it... If they can't, it is business as usual.

But then again who expects an Apple fan boi to understand the interwebs.



not necessarily

Say VZ has an agreement with AT&T that they peer with bandwidth X, and with Level 3 with bandwidth Y. Assume X >Y.

If your packets goes VZ -> AT&T at no cost due to parity peering, likewise VZ -> L3 but if they go VZ-> AT&T-> L3 VZ has to pay for transit, so they get routed the most direct path even if that path is congested. Shaping implies the packets are treated differently, if they are all delayed due to congestion (ie that one hop takes longer for 100% of packets) it's not shaping.

Now lets say you VPN into your office which is using AT&T for it's carrier and you have VZ Fios at home, VZ sees the traffic is going to AT&T and it routes into that peer, AT&T has a better peer with Netflix and it gets routed that way. To VZ all that happened with the packet went to AT&T and didn't transit, as far as AT&T is concerned a new packet is formed that goes from your office over AT&T to Netflix. If Netflix is generating more traffic than Level3s parity agreements can handle why should Verizon than have to pay AT&T to move packets that Netflix should be paying for? In the end someone is passing the buck. Peering has kept the internet from splintering, trying to break the tier 1 system by confusing net neutrality with bandwidth peering is the issue. And even under net neutrality some QOS needs to be allowed to make sure services like VOIP don't break.

Now a CDN like Akamai pays to put the content in data centers closer to the end users so instead of someone in AZ getting packets from Level 3 in CA they get it from an Akamai CDN that peers with 2~4 tier ones and lowers the number of hops. Having too few data centers and not being a true CDN Level 3 is making packets have to travel farther which means the amount of network they are eating is larger. If under Akamai the avg packet from Netflix took 10ms and not under L3 it takes 20ms that means it is in the network for 2x the time and could be eating bandwidth in more places then before.

The big boys use Akamai, Amazon or other big foot print CDNs to get their content out and not have these issues, Netflix wants to not pay for CDNs and move the cost to the carriers. Comcast and VZ maybe in the wrong if shaping is proven, which would require showing that packets going to other services hosted in the same L3 centers as Netflix are reached fast, but Netflix is not also then in the right
 
2014-07-18 03:36:07 PM  

theflatline: If you use a vpn connection via verizon to another network then the connection is not throttled. Which means if Verizon can see the traffic, they shape it...


No, wrong. Did you even read what Level3 had to say?

This is not throttling.  It's a lack of bandwidth on the connection between the two networks.  The reason a VPN allows you to get around the problem is because a VPN takes your entry point and effectively moves it to another network, which has different connections to Netflix.  There is not a single entry point to Netflix, there are many entry points, and different backbones and ISPs use different entry points to the barrier of the network.  A VPN allows you to take a detour, usually by taking the long way around(both physically and hypothetically).  It just happens that the long way around has less congestion, so it's faster.

The core of the problem is that Verizon does not want to pay out of their pocket to upgrade the connection to the network that Netflix uses.  Other services that share a network with Netflix have the same problems, like Riot Games/League of Legends, who, last year, negotiated with ISPs to route around the issue, likely at an increased cost to them(the terms of the agreement were not made public other than Riot acknowledging that they did it).
 
2014-07-18 03:39:18 PM  

zedster: Comcast and VZ maybe in the wrong if shaping is proven, which would require showing that packets going to other services hosted in the same L3 centers as Netflix are reached fast, but Netflix is not also then in the right


And Level3's disclosure shows that they aren't, at least in any meaningful way.

MadHatter500: MightyPez: bhcompy: MightyPez: It certainly doesn't help matters that users on FIOS are finding better performance through a VPN than a direct line to the service.

That's what happens when you drive around a traffic jam

It's absurd that adding extra hops to location way outside your network is going to improve performance. Verizon is dragging their feet and allowing their infrastructure to languish (Just ask their DSL customers) effectively not giving their customers what they are paying for.

I don't particularly like being in Comcast territory, but it seems a damn sight better than being in Verizon territory with these stories.

No it isn't absurd - just counter-intuitive to anyone but a network engineer. By using a VPN service, you can bypass the congestion point.  It's a rough way of doing user level traffic engineering.  If the VPN supplier was reached on the same interconnection links, you would have seen the same performance over the VPN.

We'll leave VZ's DSL platform capacity capping for a different discussion (I experience that personally - I have a VZ DSL line as a backup to my cable connection, and I can attest busy hour (8-10pm EST) performance is terrible.


Indeed.
 
2014-07-18 03:40:40 PM  

MadHatter500: Chuck Wagon: zedster: Level 3 made an agreement where they pointed their garden hose at VZ's garden hose and neither would get wet since they would be the same power. Now they show up with a fire hose and wonder why VZ is looking scared

But Level 3 has offered to pay for the hardware upgrades. To continue your analogy, Level 3 shows up with a firehose for them and a firehose for Verizion. Verizion says no, pay me first. Why?

Because the cost of the hardware is insignificant - capital recovery of the VZ network capacity is expensive.

Here's an analogy.  What L3 is trying to say is that the first truck off the production line should cost $500M, because that was how much the factory cost to build.  The second truck should only cost $5000, because that was the cost of the labor and raw materials.  L3 is claiming they are providing the labor and raw materials, so the price they should get is $0.  VZ is correctly saying that price has to cover the LRIC in providing the capacity, not the incremental cost in providing the capital to terminate the connection.  Ie. L3 should be sending VZ a check every month (ie. not a settlement free peer) or Netflix and their traffic should hook up to VZ and pay each month (the deal they cut).  Why VZ is taking so long to build the ports for Netflix's server platform is what is puzzling to all of us, even those in the industry (I mean we're slow too, but it wouldn't have been more than 45 days after cutting the deal that the new circuits would be up - even if we have to deploy new ports on the core - see below for why core network capacity needs to be built).

And the "you have spare capacity" argument is a red herring.  You never run a network above 45% utilization - if you do you can't restore the damn thing when there are outages.  Only time you can run it above 45% is when you have sufficient meshiness between all your nodes - and no national carrier has enough to be able to support that.  There are essentially 4 east-west fiber rout ...


The problem with all of this is that the reason Netflix is moving so much traffic to VZ is that their customers want to use Netflix as a service.  VZ has a deal with these customers to give them this access for whatever they would like to use it for without having the infrastructure to support it.  Level3 aren't the ones flooding VZs network, it's their users that are flooding it - it should be VZs responsibility to build up their infrastructure to be able to deliver what they are selling.

Now, if Level3 were throttling Netflix traffic to VZ because it was flooding their network, it would be Level3's responsibility.
 
2014-07-18 03:46:01 PM  

bhcompy: The core of the problem is that Verizon does not want to pay out of their pocket to upgrade the connection to the network that Netflix uses.


No.  The cost to fix the problem, as pointed out, is so small that it could be handled out of petty cash.  What Verizon is attempting is extortion.  "That's a nice service you have there.  It would be a shame if something were to happen to it."
 
2014-07-18 03:47:06 PM  
The thing is 

STOP USING VERIZON


Where ever you can move your business to someone else. Punish them for being Asshats.
 
2014-07-18 03:52:58 PM  
Sounds like they can't play nicely together. And who ends up paying in the end? Us, the consumer. At some point the government needs to just step in and sort this out. I'm not necessarily for bigger government, but I certainly don't have the resources or options to combat this asshattery.
 
2014-07-18 03:55:23 PM  

CujoQuarrel: The thing is 

STOP USING VERIZON


Where ever you can move your business to someone else. Punish them for being Asshats.


Because Comcast is so much better.
 
2014-07-18 04:01:37 PM  

OgreMagi: bhcompy: The core of the problem is that Verizon does not want to pay out of their pocket to upgrade the connection to the network that Netflix uses.

No.  The cost to fix the problem, as pointed out, is so small that it could be handled out of petty cash.  What Verizon is attempting is extortion.  "That's a nice service you have there.  It would be a shame if something were to happen to it."


They don't want to pay.  Regardless of how many dollars it is.  For them, it seems to be a stand against unbalanced peering.  If it was a ploy to get people to use their services instead of others, than Amazon, Hulu, etc would be degraded, but they aren't

A solution to ensure that the kind of abuse that people always suggest would probably be to do similar to what we had with television networks, banks, etc in the 90s.  Force ISPs and content providers to be separate.  I really don't see a problem with that, but I'm not sure how feasible it is from a cost perspective.  Most pure consumer ISPs died on the wire a long time ago, and the ones that are left are niche like DSL resellers and satellite internet companies.
 
2014-07-18 04:03:56 PM  

CujoQuarrel: The thing is 

STOP USING VERIZON


Where ever you can move your business to someone else. Punish them for being Asshats.


Verizon DSL is the only option I have. And they keep upping the price and telling me to switch to FiOS when I complain.

FiOS ain't here. Comcast ain't here. TWC is kind of here, but to get their service they want me to install 4 poles at ~$2000/pole, and then cover their labor and other infrastructure costs.
 
2014-07-18 04:08:27 PM  

jxb465: CujoQuarrel: The thing is 

STOP USING VERIZON


Where ever you can move your business to someone else. Punish them for being Asshats.

Because Comcast is so much better.


Well, when it comes to accessing Netflix, Comcast is so much better.  At least for now.
Up and leaving is a good way to tell Verizon that you're dissatisfied with their inability to work out a good deal with Netflix and L3, and to implement that deal in a reasonable amount of time.
 
2014-07-18 04:21:00 PM  

serial_crusher: jxb465: CujoQuarrel: The thing is 

STOP USING VERIZON


Where ever you can move your business to someone else. Punish them for being Asshats.

Because Comcast is so much better.

Well, when it comes to accessing Netflix, Comcast is so much better.  At least for now.
Up and leaving is a good way to tell Verizon that you're dissatisfied with their inability to work out a good deal with Netflix and L3, and to implement that deal in a reasonable amount of time.


Comcast has usage caps.  FIOS doesn't.

And, as a consumer, there are so many other good services out there that if Netflix sucks, I'll just use Amazon Prime, or PopcornTime, or Hulu, or whatever other myriad of services there are.
 
2014-07-18 04:22:19 PM  
The fact that you "network engineers" seem to be forgetting is who Verizon's customers are. Not Netflix, not Level 3, not some other CDN or content provider - it's us. Verizon is failing to provide the very service that we pay them for: access to the internet. The interconnects aren't for Netflix's benefit, they are for ours. I don't care if Verizon tries to screw Netflix, but I'm going to be pissed if they try to screw me.
 
Displayed 50 of 78 comments

First | « | 1 | 2 | » | Last | Show all

View Voting Results: Smartest and Funniest


This thread is closed to new comments.

Continue Farking
Submit a Link »
On Twitter





In Other Media


  1. Links are submitted by members of the Fark community.

  2. When community members submit a link, they also write a custom headline for the story.

  3. Other Farkers comment on the links. This is the number of comments. Click here to read them.

  4. Click here to submit a link.

Report