For those of you not able to attend the US MUM presentation here is my presentation slides in PDF for my BGP session. 200 meg download.
Categories
Tag: routing
Categories
UBNT EDGEMAX 1.10.3 update route flushing
From UBNT:
New features:
Offloading – Add CLI commands to disable flow-table flushing in offloading engine when routing table changes: set system offload ipv4 disable-flow-flushing-upon-fib-changes set system offload ipv6 disable-flow-flushing-upon-fib-changes
Discussed here
Prior to 1.10.3 firmware flow-table in offloading engine was always flushed when route was updated in linux routing table. Flow flushing ensured that offloading engine got routing updates instantly but it wasted a lot of CPU time and decreased performance if routing table was constantly updated for (instance in Full BGP, big OSPF or flapping PPPoE interface scenarios)
In 1.10.3 firmware by default disable-flow-flushing-upon-fib-changes is not set which means that flow table in offloading engine is always flushed upon routing table changes same way as it used to be in previous firmware.
If you have Full-BGP table or large OSPF network they you are advised to set disable-flow-flushing-upon-fib-changes this will ensure less CPU-load and increase max throughput.
Important note for multi-WAN environments – if nexthop interface of default-gateway changes and disable-flow-flushing-upon-fib-changes is set then it will take up to flow-lifetime seconds before all existing offloaded flows switch to new nexthop interface (up to 12 seconds by default).
Offloading – Add CLI command to modify flow-lifetime in offloading engine (expressed in seconds):
set system offload flow-lifetime 24Prior to 1.10.3 firmware flow-lifetime parameter was hardcoded and was not synchronized between different ER platforms: 12 seconds on ER-Lite/ER-Poe, 6 seconds on ER/ER-pro/ER-4/ER-6 and 3 seconds on ER-Infinity.
In 1.10.3 firmware default value of flow-lifetime is set to 12 seconds for all ER platforms and now it can be modified. By modifying flow-lifetime parameter you control how much traffic skips from offloading engine into linux network stack.
If you increase flow-lifetime then:
a) Offloaded IP flows will expire less frequently and less packets will be forwarded to linux
b) CPU load will decrease and max throughput will increase
c) if disable-flow-flushing-upon-fib-changes parameter is set then it will take more time for offloading engine to detect changes in routing table
If you decrease flow-lifetime then:
a) Offloaded IP flows will expire more frequently and more packets will be forwarded to linux
b) CPU load will increase and max throughput will decrease
c) if disable-flow-flushing-upon-fib-changes parameter is set then it will take less time for offloading engine to detect changes in routing table
Offloading – add CLI command to show flows in offloading engine: show ubnt offload flows Offloading – add CLI command to show offloading engine statistics: show ubnt offload statistics
Enhancements and bug fixes:
LDP – fixed regression in 1.10.0 when LDP configuration failed. Discussed here LoadBalancing – fixed regression in 1.10.1 when LoadBalancing failed to recover if WAN interface lost&restored link in 3 second interval. Discussed here DHCP – fixed bug when DHCP server configuration failed to commit with networks other than /8, /16, and /24. Discussed here TrafficControl – fixed regression in 1.10.0 when “command not found” output was printed when running “show traffic-control …” commands. Discussed here
Categories
Quick Reference: OSPF Network Types
Point-to-multipoint is treated as a collection of point-to-point links and thus no DR/BDR is required.
Point-to-Point is a single link and no election is needed.
Broadcast: OSPF routers on broadcast networks will elect a DR and a BDR (since it is multiaccess) – OSPF packets are multicast.
NBMA: Routers will elect DR and BDR (since it is multiaccess), but since it is a non-broadcast, routers will have to communicate via unicast rather than multicast.
Categories
12 days of netmas
On the first day of netmas
my true love sent to me:
A spanning tree instance
On the second day of netmas
my true love sent to me:
2 ethernet ends
and a spanning tree instance
On the third day of netmas
my true love sent to me:
3 sfps
2 ethernet ends
and a spanning tree instance
On the fourth day of netmas
my true love sent to me:
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the fifth day of netmas
my true love sent to me:
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the sixth day of netmas
my true love sent to me:
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the seventh day of netmas
my true love sent to me:
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the eighth day of netmas
my true love sent to me:
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the ninth day of netmas
my true love sent to me:
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the tenth day of netmas
my true love sent to me:
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the eleventh day of netmas
my true love sent to me:
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
On the twelveth day of netmas
my true love sent to me:
12 routers on a stick
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance
One of the most asked questions which comes up in the xISP world is “How do I learn this stuff?”. Depending on who you ask this could be a lengthy answer or a simple one sentence answer. Before we answer the question, let’s dive into why the answer is complicated.
In many enterprise environments, there is usually pretty standard deployment of networking hardware. Typically this is from a certain vendor. There are many factors involved. in why this is. The first is total Cost of Ownership (TCO). It almost always costs less to support one product than to support multiples. Things like staff training are usually a big factor. If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.
Another factor involved is economies of scale. Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words. You can commit to buying product over time or all at once.
So, to answer this question in simple terms. If your network runs Mikrotik, go to a Mikrotik training course. If you run Ubiquiti go to a Ubiquiti training class.
Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario. Many of our xISP clients have gear from several vendors deployed. They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between. So where does a person start?
We recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.
1.Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S). There are a ton of ways to study for this certification. There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of studying for this certification, you will learn many things which will carry over to any vendor. Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks. During the course of studying you will learn, and then reinforce that through practice tests and such. Don’t necessarily focus on the goal of passing the test, focus on the content of the material. I used to work with a guy who went into every test with the goal of passing at 100%. This meant he had to know the material. CompTIA is a side path to the Cisco CCNA. For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.
2.Once you have the CCNA under your belt, take a course in a vendor you will be working the most with. At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.
3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook. Think of your networking knowledge as your end goal is to be able to build a house. Steps one and two have given you general knowledge, you can now use tools, do some basic configuration. But you can’t build a house without knowing what is involved in designing foundations, what materials you need to use, how to compact the soil, etc. Network design is no different. These are not things you can read in a manual on how to use the tool. They also are not tool specific. Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track. Just follow my philosophy in relationship to #1. Focus on the material.
Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is a key to your success. If you understand what goes on with an IP packet, learning tools like Wireshark will be easier. As you progress let things grow organically from this point. Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options. Branch out into some other areas ,such as security, to add to your overall understanding.
Never stop learning! Visit our online store for links to recommend books and products.
WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not. My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
–Butch Evans
–Dennis Burgess
–Rickey Frey
–Steve Discher
–Baltic Networks
Vendor Certification Pages
–Ubiquiti
–Mikrotik
–Cisco
–Juniper
–CWNA
–CompTIA
If you provide training let me know and I will add you to this list.
Several folks have asked if they need to pay attention and know what routing.Prefix lists in RouterOS do. The skinny is they are only used for RIP routing. Most modern networks do not use RIP and haven’t for some time. And now you know…
Categories
pfSense 2.3 released
From the folks over at pfSense
We are happy to announce the release of pfSense® software version 2.3!
The most significant changes in this release are a rewrite of the
webGUI utilizing Bootstrap, and the underlying system, including the
base system and kernel, being converted entirely to FreeBSD pkg. The
pkg conversion enables us to update pieces of the system individually
going forward, rather than the monolithic updates of the past. The
webGUI rewrite brings a new responsive look and feel to pfSense
requiring a minimum of resizing or scrolling on a wide range of
devices from desktop to mobile phones.
You can find all the details in the release announcement here:
https://blog.pfsense.org/?p=2008
Thanks for your support
Categories
How does BGP select which route?
BGP can be a complex and almost mystical protocol. For those of you who are trying to determine how BGP selects which route here is your guide. Before we get into it a couple of things to keep in mind. First, BGP is not a multipath routing protocol. This is different than what you may be used to with OSPF. BGP goes to great lengths to encure only one route is used. Secondly, there are some vendor specific rules which are applied. I will try to point those out as we go along.
1.The first test is if the next hop router is accessible.
2.If Synchronization is enabled, the router will ignore any iBGP routes which are not synced.
3.The third is Cisco specific. Cisco uses a weight attribute. The largest weight wins. Default weight is zero. Maximum weight is 65,535.
4.If the weights are the same, the highest local preference is chosen from LOCAL_PREF. It’s important to note that routers only receive this from iBGP.
5.Net up, the router checks to see if any of the possible routes were originated locally. The two main checks are either the network or aggregate commands. The network command wins if it is originated locally.
6.If two or more routes are still equal the router looks as AS_PATH. The router will prefer any iBGP routes. Outside of the AS BGP will prefer the shortest path.
7.BGP then moves on to the ORIGIN attribute. If the path lengths are the same, BGP selects IGP over EGP and EGP over INCOMPLETE routes.
8.BGP now looks at MED values. The lowest value is selected. Note, MED is only used if both routes are received from the same AS, or if always-compare-med has been enabled. Be careful with always-compare-meds as this can cause routing loops.
9.BGP will then prefer eBGP to iBGP routes. This is not the same as #5 above. Only external routes are looked at here.
10.Next IBGP costs are compared to the next hop routers. The closest one is selected.
11.Ages of routes are finally connected. This is kind of like choosing teams for dodgeball. The oldest route wins. The reason being is oldest routes are thought to be more stable.
12.And finally, if all else fails the router with the lowest router ID wins.
This is a quick low-down on how BGP “thinks” in order to determine routes. If anyone has some Cisco, Mikrotik, quagga, or other specific attributes please comment. I have reached out to Mikrotik and Ubiquiti specifically to see if this is in-line with their implementation of BGP.
Categories
Helpful OSPF times
OSPF can be a mystery to some. Understanding the default timeouts can be helpful in troubleshooting. Some vendors change these times so it is very important to realize this stuff if you start mixing vendors in your OSPF domains.
10 Seconds
Default OSPF hello timer on broadcast and point-to-point links
30 Seconds
Default OSPF hello timer on nonbroadcast links
40 Seconds
Default OSPF hold timer on broadcast and point-to-point links
120 Seconds
Default OSPF hold timer on nonbroadcast links
30 Minutes
OSPF LSA refresh timer
60 Minutes
OSPF LSA expiration timer
Categories
OSFP and areas
OSPF areas are one of the more common topics I am asked about as networks grow. Before we dig into this, we need to understand the reasons why OSPF areas were created in the first place. Next, we will go into how to apply areas to modern network designs.
Why did areas come into being?
Let’s rewind to a time where RAM in routers was very expensive. Processors were expensive. One of the biggest reasons OSPF areas came into play was ram limitations. Every route in your routing table takes up ram. The more routes you have the more ram that is taken up in each router to hold that table. Also, in order to calculate these routes processor power is used up.
So what do areas do for me?
OSPF areas have many advantages. However, in most WISP and ISP networks they serve two purposes.
The first purpose is they group similar devices into logical groups. These groups can have filtering policies applied to them.
The second purpose is more important. Implementing areas reduces the size of the routing table. By doing this your routers spend less time calculating routes, and less time updating the database during a topology change. By reducing the routing table you also speed up what is called convergence time. This is the time the entire network needs to agree on the current routing topology. If a major backbone link is flapping your routers could be spending a fair amount of their resources calculating routing tables.
An important thing to note with implementing areas is you must have a good IP network design. This means your sites/pops/towers should have a logical design which allows for easy route summarization and consolidation. If you are looking into areas make sure you can summarize your routes in that area easily. The following examples illustrate why this is important.
In the above example we have created “Area1”. Since we have a good IP network design we can summarize our routing table in and out of area1 into a few different ways. If we want to reserve the whole 10.5.0.0/16 for future expansion then we can. Or if we want to break this down into 10.5.0.0/20 or even smaller we can. Part of this depends on growth plans. With areas you have to keep in mind every area needs to touch the backbone (area 0.0.0.0) directly. Now, you can use Virtual Links to have one non-backbone area traverse another non-backbone area. However, even though is a standard, is a workaround at best. There are many disadvantages to virtual links.
Now, back to our example. If we create an area the 10.6.1.0/24 and 10.6.2.0/24 routers are the “in roads” to Area1. These are known as area border routers (ABR).The main function of ABRs is to summarize sub networks found throughout the OSPF network. It stores many copies of its link-state database in memory when one of the stored copies shows an area where the actual router is connected. The ABR holds a minimum of two copies of the routing tables. One from the backbone area, and one from each area it is connected to.
But, I thought areas were supposed to cut down on ram and CPU usage? Well, everything has a tradeoff. This is where the philosophy side of things come into play, and probably the reason you have read this far.
When, how, and should I implement areas?
In today’s modern world with fast ram, fast links, and fast processors OSPF areas are needed less and less. Routers today have more ram than even 5 years ago. This means they can hold larger routing tables and do more calculations.
If you are thinking about implementing areas the first thing to look at is your IP design. In order to take the best advantage of areas you should have a logical, and congruent design. What I mean by this is your towers should be able to summarized as much as possible. If you can fit 20 towers into a single route statement that is one good place an area would make sense. If those 20 towers are not able to be summarized then adding an area is not going to be much of a benefit to you.
Network size does not necessarily dictate the need for OSPF areas. If you have a neatly summarized IP network the need for areas is lessened.
What about if you are trying to join two different networks?
Say you purchased a neighboring ISP and want to join the new network with your own. If you have overlapping IP space then things might not mesh together well, even with areas. Most times you are better off running BGP with the two separate networks. This allows each network to have it’s own space, own routing policies, but still be able to share bandwidth and other resources. You simply don’t announce any overlapping space to each network until things are re-numbered.
One question I get in this scenario is my router can’t handle BGP. BGP is a fairly lightweight protocol. The issues arise when you start pulling in full or partial internet routing tables. This is the same concept as mentioned above with the OSPF routes.
You must be logged in to post a comment.