Tag: wisp

Categories
Tower Uncategorized Wireless WISP xISP

ALG Antenna test vs Jirous dishes

The following are results from a series of tests of AGLcom’s parabolic dish antennas on an existing link that is 5.7 miles long. The link typically passes 80-90Mbs with a TX capacity of 140 Mbs and radios used are Ubiquiti AF5X operating at 5218 Mhz.  A full PDF with better Readability can be downloaded here..

The tests were taken in stages:

  1. 1)  The normal performance of the link was recorded.
  2. 2)  The 2′ dish at one end, B, was replaced with the AGLcom, C, dish and the link reestablished.The link performance was recorded.
  3. 3)  The 2′ dish at the other end, A, was replaced with the AGLcom, D, dish and the link reestablished. The link performance was recorded.
  4. 4)  The setting on the AF5xs were adjusted to optimize the link performance with data recorded.
  5. 5)  The 2′ dish, B was put back in the link and the performance was recorded.
  6. 6)  The ACLcom C was put back into place.

The tables below do not follow the test order as the third line of data was actually the last test performed.

Antennas:

A-Jirous JRC-29EX MIMO
B-Jirous JRC-29EX MIMO C-AGLcom – PS-6100-30-06-DP D-AGLcom – PS-6100-29-06-DP-UHP

Results:

Table 1 is the signal strength results of the various dishes on the link. The first line, A-B, is the original Jirous to Jirous. A is the first two columns of the link and are the A side and the last two columns are the B side on the link. What is of interest is that exchanging B to C in the second line brought the signal deviation between the channels to only 1db and 0 db as seen in Table 2. The third line was a result of replacing the horn on the A dish and optimizing the setting on the AF5X radios. This changed the signal by around 7db and improved the link capacity, Table 3. Clearly, the A dish had a problem with the original horn.

In the fourth line, D-B, the signal strength improved as well at the signal deviation on the two channels, Table 2 first two columns. This link was not optimized. The fifth line, D-C is both AGLcom dishes which improved the bandwidth, Table 3, and the signal deviations, Table 2. The final line, D-C, was the previous line optimized. The signal strengths moved closer together and the bandwidth improved.

Link Ch0 Ch1 Ch0 Ch1

  1. A-B  -73 -76
  2. A-C  -73 -74

A*-C -64 -66

  1. D-B  -63 -62
  2. D-C  -62 -62

D*-C -60 -60

-70 -74 -71 -71 -65 -66 -59 -59 -58 -58 -61 -61

Signal Strength (* optimized data) Table 1

Table 2 has four data columns, the first two being the measured results and the latter two being the measured difference from theory. The Jirous and AF5X calculators were used for the theory signals. Clearly the signal approached the theoritical limit with the optimization and with the change of dishes. The optimization improved the signal by ~9db for the link that we replaced the horn on the Jirous and by ~2db for the AGLcom link.

Link dSig dSig A-B 3 4 A-C 1 0 A*-C 2 1 D-B -1 0 D-C 0 0 D*-C 0 0

dSig dSig -16.5 -17.4 -17.0 -15.0 -8.0 -9.0 -13.3 -5.3 -7.0 -4.3 -5.0 -6.0

Signal strength variation from theory Table 2

The band width improvement was more obvious, Table 3, from 22 Mbs to 39 Mbs for the RX and 144 Mbs to 141 Mbs TX for the link with the horn replacement. The bandwidth improvement for the optimization of the AGLcom link was from 61Mbs to 66Mbs RX and from 211Mbs to 267Mbs for TX.

The bandwidth improvement from the original, optimized link to the AGLcom link is from 61Mbs RX to 67Mbs and from 210Mbs TX to 267Mbs. There is a clear improvement for the AGLcom link over the Jirous link.

Link BW-RX

  1. A-B  22.5
  2. A-C  39.0

A*-C 60.9

  1. D-B  61.4
  2. D-C  60.6

D*-C 66.6

BW-TX 144.6 141.4 210.0 211.0 215.0 267.6

Table 3

Conclusions:

The data supports a measurable improvement in both signal strength and bandwidth with the use of the AGLcom dishes. However, it is difficult to quantify the improvement. The Jirous dishes were identical whereas the AGLcom dishes were not. One of the jirous dishes was under performing initially but was repaired for the last tests. Additional testing is needed to provide accurate data analysis and performance comparison. The best performance tests would involve identical AGLcom dishes, ideally two links, one each of both types of dishes.

Categories
Cambium UBNT Wireless WISP

UBNT vs Cambium -The legal battle

The Recently, it was announced that Ubiquiti Networks Inc (UBNT) is suing Cambium over the Cambium Elevate.   This will be a long post, so sit back with your favorite beverage and read away.

Disclaimers. I have been in the ISP world since 1991. I cut my teeth on BBS systems and moved onto dial-up. I am also an independent Cambium certified consultant.  Read about the consultant program here... I also have clients who run a wide variety of UBNT products, and the last ISP we sold was 90 percent UBNT. We run some UBNT routers in MidWest-IX as well.  My father was an attorney for over 40 years. I grew up around attorneys, have regular conversations with friends who are attorneys, and was learning about the law from the time I was 10. Having said that, I am not an attorney. Nothing in here should be construed as an official legal opinion.

So let’s get some background on what has transpired with Cambium and their elevate software. Cambium came up with a way to load their software onto select UBNT wireless units and, after a reboot, had the cambium EPMP software active on them.

Why did this work?
UBNT Airmax radios use U-Boot loader. If you want to read all about it you can read the references at the bottom of this article under References. The thing to know is it is released under the GNU General Public License.

UBNT and Cambium EPMP both use “commodity” wifi chipsets.  This keeps the cost down and the software becomes the majority of the “special sauce” that makes them different.   This is in contrast to the UBNT Airfiber and Cambium 450 lines. These use custom made chipsets. This is is one reason those lines are more expensive.

By using an open source bootloader and commodity hardware Cambium was able to figure out how to load their own software onto the UBNT devices.   UBNT countered with modifying the bootloader to accept only signed software images. The only images that were recognized were ones signed by UBNT.  If you are interested in learning more about signed software go here: https://www.quora.com/What-does-signed-firmware-means

Cambium came up with instructions on how to downgrade and by-pass the ability to only load signed firmware onto the device.  The method I am aware of is downgrading the installed UBNT firmware to a certain version.

All in all the Elevate process turned the UBNT hardware into a device running Cambium’s software.

The gray areas aka this is why we have attorneys
There are several arguable points in this lawsuit.  If you want to read articles on the Lawsuit
https://www.law360.com/articles/1071813/wireless-co-ubiquiti-says-rival-sells-hacking-firmware

Debate #1 – The Hardware
The term Software Defined Radio (SDR) has been around for quite some time now.  Basically, this is a radio with very little RF elements to it.  Ham radio has been using SDRs for quite some time now.  The idea is the manufacturer uses off the shelf components to build a single radio which can do various functions depending on what software is loaded.  It also allows features in the chipset to be activated and licensed should the programmer want to support them.  It’s interesting to note Wireless is not the only place this is happening. Software Defined Networking (SDN) is a growing thing, as well as a plethora of devices. A PC could be considered a software-defined device.  More on that later.

So an argument could be made the UBNT devices are a software defined radio.  they did not use custom chips.  They most certainly have a proprietary board layout, but that is not a criterion in an SDR. So if a customer buys a piece of hardware, should they be able to load whatever software they want on it?

An argument saying yes they should can be pulled from many areas.  This Verge Article (more in the reference at the bottom) says the Government ended the debate in 2015 giving consumers the ability to Jailbreak their phones and devices without legal penalties.  Before that is was briefly illegal to “Jailbreak” your phone.   This was mainly lead by Apple. The government said it was fair use to Jailbreak, but not carrier unlock your phone without permission.

Apple also went through this briefly when they switched to Intel processor chips.  People were figuring out ways to load Apple OSX onto Dells, HP, and other “PCs”. The debate was whether this was legal or not. The following article sums up why these “hackintosh” computers were shut down. By clicking on the “Agree” of the End User License Agreement (EULA) before installing OSX you agree to a great number of things.   The short of it was the user license of OSX says you can not install this on non-apple hardware.  However, it says nothing about installing non-Apple Operating systems on the hardware.  Apple knows it is commodity hardware.  If you want to buy a 2000 mac and put windows 10 on it, go ahead.  They even help you with an option called Bootcamp.

Our last example is the Linksys WRT54G and DD-WRT and its variants.  A quick history of the DD-WRT Controversy doesn’t revolve much around the loading of the software onto Linksys hardware, it involves the use of the GPL license by DD-WRT. There were some FCC concerns, but we will talk about those later.

So the questions to be argued for this point:
Q1.Is the UBNT device a software-defined Radio?
2. Does the user have the legal ability to load whatever software they want to on hardware they own?

Debate #2 – Was the UBNT firmware “hacked” as they allege?
There are lots of unknowns here.  Attorneys try to prove intent in arguments like this.
Did Cambium somehow reverse engineer the UBNT software, thus violating copyright laws?  At what point is the line crossed? Since UBNT used a bootloader free to everyone, was the simple act of loading new software onto the units a hack? From what I know, and I am not a programmer, is Cambium used the bootloader to overwrite the UBNT software and install their own.  How is this any different than installing Linux on a Dell PC? Computers have a bootloader called a BIOS. On a Wireless radio, where does the bootloader stop and the software start? To me, these are clearly defined. Bootloader and Image file.

If you boot up the UBNT unit out of the box without agreeing to the EULA have you violated the EULA? Can you be penalized for loading software onto a device you never had the opportunity to see and agree to anything? Did the simple act of taking it out of a box and booting it up via TFTP cause you to agree to something?

In a Brothers Wisp video on this topic, Justin Miller mentions some arguments on why this can be allowed.

Debate 3 – Did Cambium violate FCC rules?
If we believe the user has the ability to load software onto units they own it is the user, as well who developed the software to go on the device, to follow all laws then it is not up to UBNT to police this.  This is the job of the FCC, provided it is agreed that once the user buys the hardware it is theirs.  For this specific case, UBNT claims Cambium is violated allowed power limits by loading their software onto the UBNT device.   Also, is the new device an FCC certified system? Most likely not unless it is resubmitted to the FCC for testing, and any labels removed and new ones added.  However, this is not up to UBNT to enforce this. This is the job of the FCC.

Is UBNT being a steward of the community to bring this to the attention of the FCC, thus saving UBNT from possible issues with the FCC? Maybe, but why not bring suit against any of these others?
Bitlomat
DD-WRT
HamNet

It’s interesting to note this page on HamNet

I am not a telecom attorney and I do not know the ins and outs.  From what little I know of being in the industry you have to have an FCC certified system with proper identification stickers.  I remember when UBNT had to send out stickers for units several years ago for DFS certification.  You were supposed to put them on all your upgraded radios to be compliant. By changing the software did Cambium no longer make it a certified system? Or, because they use the same chipset is it still legal in the eyes of the FCC?

Debate 4 – Collusion and the end user
This is the biggest bombshell out of this whole ordeal and actually makes my blood boil.  UBNT is suing Cambium of course.  They are also suing a distributor and an end-user ISP.   Cambium I can understand. UBNT is trying to protect their intellectual property and believe it was violated.  They have every right to do so.

The distributor I can understand the argument.  The distributor allegedly participated in distributing the “hacked” software. Not saying it’s right or wrong, but I can see why there would be the argument.

The most disturbing part of this an end-user ISP is named in the lawsuit.  UBNT is suing a customer who was using the UBNT product and then decided to switch to a competitors product.  In the case of elevate, the end-user ISP loaded the software onto their existing hardware.  If we go along with the idea of you own the hardware, UBNT is suing a customer who bought their hardware and loaded the elevate software on it.  This would be like Dell suing a school corporation for loading Linux onto new PCs they bought.

Many of the arguments you read are about you don’t own the software.  If you buy the hardware, and it has a GPL licensed bootloader and load your own software onto the device, what laws have you violated?

Imagine this scenario.  A user opens up a UBNT radio they bought.  They see it uses an Atheros chipset, like many other radios.  They write some code to talk to the hardware, all without ever looking at the software that came on the radio, boot up the unit via TFTP and load their own compiled image onto the hardware.  All the while they never have seen the UBNT software.  Did they violate any laws or user agreements?

This case and some others will help define who owns the hardware.  We know the company, in this case, UBNT, owns the software.  You have no legal standing to de-compile their intellectual property. That is cut and dry.  What isn’t, is if they are using the same hardware everyone else, the same bootloader, is that considered proprietary? If not, and you overwrite their software were you allowed to because you own the hardware. Is the GPL bootloader considered proprietary?  If we apply the analogy the bootloader is the same as the BIOS in the PC, no it is not proprietary.  The BIOS debate has already been solved in court. Many of the PC debates have been loading a company’s software onto other hardware, such as Apple Hackintosh Computers and not the other way around, such as this case. As we talked in point 1, in the PC world, Apple even gives you the tools to install other Operating systems.

If UBNT sticks code in that says the bootloader only recognizes signed images is that “hacking” to put your own software on? Is this any different than Jailbreaking an Iphone?

So what does this all mean?
Going forward I believe we will see EULA and licensing agreements change.  The hardware from a manufacturer will still be the property of the manufacturer, much like John Deere software.

The definition of what you own and have access to will change.

Proprietary bootloaders will take the place of Open Source bootloaders.

There will be a rise in manufacturers who make white box radios.  Will there be a long-term solution? Only time will tell.  We are seeing this trend in software-defined networking.

We will see more NDAs to end users about products.  I believe we will see fewer case studies on newer products.  End users will definitely be more tight-lipped about what they are doing.

So it will be interesting to see how this all plays out.  Will there be enough precedent in the hardware world to squash some of this? Or does UBNT have a case? Obviously, UBNT has a responsibility to their shareholders to vigorously defend their Intellectual property.  This case will help define where the commodity/open source items stop and where the intellectual property starts.

Where does this leave distributors? Do they want to continue carrying the Elevate product? Do they want to cut relationships with a manufacturer who has sued one of their own? The same goes for the end-user community.  Do WISPs want to do business with a company that could potentially sue them for using and talking about a competitor’s product? Do the end users own the hardware they buy? If so, how much freedom do they have? If you don’t own the product, imagine the accounting ramifications.

References
https://motherboard.vice.com/en_us/article/xykkkd/why-american-farmers-are-hacking-their-tractors-with-ukrainian-firmware

https://wiki.openwrt.org/toh/ubiquiti/airmaxm

https://www.wired.com/2015/04/dmca-ownership-john-deere/

https://www.wired.com/2010/07/feds-ok-iphone-jailbreaking/
Feds okay iPhone Jailbreaking

https://superuser.com/questions/424892/is-bios-considered-an-os
Is the Bios an Operating System?

https://www.chromium.org/chromium-os
Google Chromium OS

Categories
Uncategorized

The importance of phone numbers in a WISP

One of the things I see startup wisps do wrong is their use of phone numbers.  This is one of those details that is often overlooked but is critical. It’s critical not only for tracking but also for the sanity of everyone involved.  Let’s identify where many WISPs go wrong.

The typical startup wisp is a type A go-getter. This is what Entrepreneurs are by default.  Once they have a plan they jump head over heels in. Many may start with a simple phone number, but when they call a customer if they are on their way to do an install or something they end up using their phone number.  The problem is customers keep this cell phone.  If the office is closed they start texting or calling any number they have.  Some customers will be respectful of boundaries, but many will not.  If they are getting packet loss at 3 am they are calling and texting.  This problem compounds as you grow and you have multiple installers involved. You want customer issues tracked in some sort of ticket/CRM system. You also don’t want your employees ahev to answer customer texts or calls after hours if they aren’t being paid.  It’s one of the quickest ways for employees to get burnt out or say the incorrect things.

So how do you solve this? The simple buzzword answer is unified communications.  One of the easiest and cheapest is Google Voice. With Google Voice and others, you have a primary number. This is the number you give out to clients. They call this and it rings another phone or phones.  This can be an extension on the VOIP system it is a part of, another number, and/or cell phones.  Depending on the level of sophistication it can ring all the programmed numbers at once, or ring one, and move on to the next one. If no one answers it drops the caller into voice mail. With Google voice, the programmed numbers are all rang at once.

The inbound ringing is pretty standard.  The “trick” for the WISP is the outgoing calling. You want to be able to call a customer and have it come up as the main number’s caller ID, not your cell phone. Most PBX systems can be set up to do this with the extensions attached to them.  Cell phone calls are a little more complicated.  The way Google Voice solves this is through the use of forwarding numbers, You bring up the app, enter a number and it actually calls a different number.  Behind the scenes, it is using this forwarding number to “spoof” your number to the person you are calling.   Your phone is not calling the other party directly. Your phone calls this forwarding number behind the scenes and works it all out on the backend.

Other vendors have Apps which do similar functions. Asterisk has their DISA function.  Once you have these functions setup it boils down to training and processes.  Your installers need to remember to use the app or the function when calling customers.  As the company grows, a way to help this situation is for employees to not use personal cell phones.  If a company provides a cell phone the employee can customize voicemail, or even forward no answers to the help desk should a customer get the cell phone.

Hope this helps one of the glaring issues a startup faces.

Categories
Wireless WISP

Ac Wave 1 vs Wave 2

There has been much discussion on the performance of going from an N Series outdoor wireless system to AC.  Not all AC is created equal.  Right now there is AC Wave 1 and AC Wave 2.  Just about all the AC stuff currently in the pipeline for outdoor wireless is wave 1.  There is wave 2 indoor gear available, but for a WISP you are interested in the outdoor gear.

So what’s the difference?
For some reading about spatial streams, channel sizes, etc. look at this article https://info.hummingbirdnetworks.com/blog/80211ac-wave-2-vs-wave-1-difference

For the WISP folks who want the Cliff Notes version here are some key differences.

-Wave 1 uses 20,40,and 80 Mhz Channels.  Wave 2 can support 80 and 160mhz channels.  The 160mhz channel would be two 80mhz channels bonded together.

-Wave 1 can do 3 spatial streams.  Wave 2 does 4. This requires an additional antenna to take advantage of wave2.  This is a hardware upgrade from wave1 to wave 2.

-Wave 2 supports MU-MIMO. The AP can talk to 4 clients individually at once.  The client must also support this, which is a hardware upgrade from wave 1 to wave 2 on both the client and the AP.

The question to ask your vendors is what is the upgrade path if you are using existing AC gear.  If you are running AC currently you are most assuredly going to have to replace your AP radios and antennas.  Will your existing clients work with the new AC wave 2 aps? An important thing to ask.

 

Categories
BGP Cambium DIY epmp Mimosa Networking WISP

Lab Network

I am starting an ongoing series involving a semi-static set of devices.  These will involve different tutorials on things such as OSPF, cambium configuration, vlans, and other topics.  Below is the general topology I will use for this lab network.  As things progress I will be able to swap different manufacturers and device models into this scenario without changing the overall topology.  We may add a device or two here and there, but overall this basic setup will remain the same.  This will allow you to see how different things are configured in the same environment without changing the overall scheme too much.

We will start with very basic steps.  How to login to the router, how to set an IP address, then we will move to setting up a wireless bridge between the two routers.  Once we have that done we will move onto setting up OSPF to enable dynamic routing.  After that the topics are open.  I have things like BGP planned, and some other things. If there is anything you would like to see please let me know.

Categories
Tower WISP

Vendor Spotlight: Subcarrier Communications

Over the past several WISPA shows I have had the opportunity to chat and get to know CEO John Paleski from Subcarrier Communications (www.subcarrier.com). John is very in-tune with how the WISP industry functions in terms of tower needs.  Many of the big tower companies tack on so many fees with their towers it makes leasing a tower out of reach for many. Add on the processes in place can be a deterrent to getting equipment in place.

Subcarrier has addressed many of these hurdles for the WISP industry.  Reasonable rates for tower rent are always a concern, but if the business model is there for the WISP, they are not the primary concern many times.  Not only has subcarrier realized many WISPs are utilizing smaller equipment, but things like huge application fees are a negative for the smaller WISP. Subcarrier knows what is on their towers. Such a simple thing means a rapid and smooth deployment for the WISP.  After several conversations with JOHN, it is apparent he knows just about every tower in his inventory.  He can tell you if they will support what you are wanting to hang on that tower without running a $2000 engineering study right off the bat.  On the flip side, he isn’t compromising safety or integrity of the tower.  Many towers, such as old AT&T long lines towers were built to such high specifications if you just apply a little common sense and some quick figuring you know the typical WISP deployment isn’t going to add any significant amount of loading on the tower.

I believe that John thinks the same way many of us in this industry do.  An empty tower is not making anybody any money.  If it makes sense for both parties then a deal can be made.  Too many of the larger tower companies only look at deals that make sense for them.

I would encourage any of you looking for towerspace to check out the sites Subcarrier has.  Check out their interactive Google Search to see if they have some towers you could use. Tell them Justin sent you over.

Categories
Uncategorized

Cambium ePMP Review from CrossTalk Solutions

Crosstalk does an in-depth review of the ePMP 2000 setup.

Categories
Tower

Tower crew in today’s world

One of the questions we often are asked is why our rates for tower work are what they are. In today’s world, a tower crew needs the following, not only for themselves but to protect and do the best job for the client.

The first key is equipment.  Having a crew with proper ropes, proper lifting blocks, and pulleys, and proper safety gear goes a long way. A job can be done more efficiently with the proper tools.  In-Shape tools make a big difference. How many times have you gone to cut something with a dull blade? Tools get used up and have to be replaced.

Next up is safety and insurance.  I lump these into the same category because an insured crew is safe for the client.  Having the proper insurance protects the client from anything that may happen.  Tower work is dangerous work.  With insurance requirements comes updated training. Not only does this teach crews new methods of doing things, it helps them in becoming complacent in safety practices.

Availability is the next thing. Having a crew that can roll out in a timely manner to meet client’s needs takes a dedicated staff.  We see too many part-time crews not bringing in enough money so they are having to moonlight doing other things this lessens the availability because you have to find steady work to have quality people.

The last thing is the experience our crews have.  Having been a veteran of the WISP industry for over 12 years I have seen many ways of doing things, so Have the rest of the experienced folks in our crews. We have done night climbs, harsh weather work, and custom work.  Having someone who knows the WISP industry doing your tower work makes a huge difference.

Categories
Uncategorized

SaaS aka why I should pay per month for billing

The topic of paying per user for a billing or management platforms comes up every so often.  I was able to sit down and talk with several vendors at WISPAPALOOZA this year about the value of their customers paying a per-user fee.

The most prevalent thought is about innovation and new features.  SaaS allows the billing vendor to invest development and testing time in rolling out new features to support new equipment, and other software.  LTE platforms are the hot thing in billing integration. New additions to software take people power and hours of testing and tweaking. Without monthly recurring revenue to drive such things billing vendors would have to develop this and then charge to the early adopters as an add-on.  This can be a double-edged sword. The early adopters have to pay a premium in order to get a partial solution because the vendor has to really prioritize how their development resources are used. The Vendor is always chasing the next big thing, which means other additions or fixes tend to get pushed back. They have to finish add-ons they think more folks will want to buy first.

The next thing is plain old hosting. Hosting a software application, whether in the cloud or on your own hardware costs money.  Co-location, software patches on the OS, hardware lifecycles, etc.  This cuts down on the end-user maintenance side of the hardware but pushes it back to the vendor. The peace of mind of knowing the thing that collects your money is running is backed up, and is available as part of the monthly fee you pay.

SaaS also allows for quicker releases of bugs and new features.  Vendors have more resources dedicated to development and changes. This allows for new add-ons to become available quicker.  Take the traditional model where you get bug fixes, but major feature add-ons are either a full point upgrade or major version upgrade. This usually costs money and is a slower process.  Not only does the vendor have to spend resources advertising, but they have to deal with support and other issues. With billing vendors who charge a monthly fee fixes from companies such as Paypal or Authorize.net are almost always rolled out very quickly at no additional charge to the end user ISP.

Some companies such as Basecamp, which is not a billing platform, have taken a hybrid approach to SaaS. Every major revision that comes out is an upgrade. You can choose to upgrade or stay where you are and pay the same amount.  This can leave customers behind but still allows them to use what they are paying for.  They just don’t get new features or bug fixes.

So the next time you are figuring out why you should pay for a billing platform on a monthly, customer, or subscription basis take all of this into account.

For those looking for xISP billing, and mainly WISP billing, here is a partial list:

www.azotel.com
www.visp.net
www.powercode.com
www.sonar.software
www.splynx.com
www.ispbilling.com (Platypus)
www.freeside.biz
www.quickbooks.com

If you have more please add them in the comments.

Categories
BGP Data Center Networking WISP

The problem with peering from a logistics standpoint

Many ISPs run into this problem as part of their growing pains.  This scenario usually starts happening with their third or 4th peer.

Scenario.  ISP grows beyond the single connection they have.  This can be 10 meg, 100 meg, gig or whatever.  They start out looking for redundancy. The ISP brings in a second provider, usually at around the same bandwidth level.  This way the network has two pretty equal paths to go out.

A unique problem usually develops as the network grows to the point of peaking the capacity of both of these connections.  The ISP has to make a decision. Do they increase the capacity to just one provider? Most don’t have the budget to increase capacities to both providers. Now, if you increase one you are favouring one provider over another until the budget allows you to increase capacity on both. You are essentially in a state where you have to favor one provider in order to keep up capacity.  If you fail over to the smaller pipe things could be just as bad as being down.

This is where many ISPs learn the hard way that BGP is not load balancing. But what about padding, communities, local-pref, and all that jazz? We will get to that.  In the meantime, our ISP may have the opportunity to get to an Internet Exchange (IX) and offload things like streaming traffic.  Traffic returns to a little more balance because you essentially have a 3rd provider with the IX connection. But, they growing pains don’t stop there.

As ISP’s, especially WISPs, have more and more resources to deal with cutting down latency they start seeking out better-peered networks.  The next growing pain that becomes apparent is the networks with lots of high-end peers tend to charge more money.  In order for the ISP to buy bandwidth they usually have to do it in smaller quantities from these types of providers. This introduces the probably of a mismatched pipe size again with a twist. The twist is the more, and better peers a network has the more traffic is going to want to travel to that peer. So, the more expensive peer, which you are probably buying less of, now wants to handle more of your traffic.

So, the network geeks will bring up things like padding, communities, local-pref, and all the tricks BGP has.  But, at the end of the day, BGP is not load balancing.  You can *influence* traffic, but BGP does not allow you to say “I want 100 megs of traffic here, and 500 megs here.”  Keep in mind BGP deals with traffic to and from IP blocks, not the traffic itself.

So, how does the ISP solve this? Knowing about your upstream peers is the first thing.  BGP looking glasses, peer reports such as those from Hurricane Electric, and general news help keep you on top of things.  Things such as new peering points, acquisitions, and new data centers can influence an ISPs traffic.  If your equipment supports things such as netflow, sflow, and other tools you can begin to build a picture of your traffic and what ASNs it is going to. This is your first major step. Get tools to know what ASNs the traffic is going to   You can then take this data, and look at how your own peers are connected with these ASNs.  You will start to see things like provider A is poorly peered with ASN 2906.

Once you know who your peers are and have a good feel on their peering then you can influence your traffic.  If you know you don’t want to send traffic destined for ASN 2906 in or out provider A you can then start to implement AS padding and all the tricks we mentioned before.  But, you need the greater picture before you can do that.

One last note. Peering is dynamic.  You have to keep on top of the ecosystem as a whole.