Paypal Donate





Technical Details

IX-Denver Basics

  • IX-Denver is located in suite 350 at 910 15th St. Denver, CO.
  • The switch(s) in 910 15th street are fed by diverse power feeds and two power supplies.
  • IX-Denver currently accepts and forwards ethertypes 0x0806 (ARP) 0x0800 (IPv4) and 0x86dd (IPv6).
  • Both access and trunk ports are available, members with trunk ports can negotiate private vlan membership between yourself and other members of the IX.
  • Access ports will be placed in the public peering VLAN only.

 

Important Facts

  • IX-Denver only uses 10GBASE-LR or 1GBASE-LR (1310NM) optics
  • The public peering VLAN is VLAN-ID 100
  • Switch VLAN 100 interface IPv4 - 206.53.175.254/24
  • Switch VLAN 100 interface IPv6 - 2001:504:58::254/64
  • Layer 2 MTU of all ports is set at 9192
  • Route server - A (bird 1.4.0) IPv4 - 206.53.175.3/24
  • Route server - A (bird6 1.4.0) IPv6 - 2001:504:58::3/64
  • Route server - B (bird 1.6.3) IPv4 - 206.53.175.5/24
  • Route server - B (bird6 1.6.3) IPv6 - 2001:504:58::5/64
  • Route server - C (bird 1.6.3) IPv4 - 206.53.175.7/24
  • Route server - C (bird6 1.6.3) IPv6 - 2001:504:58::7/64
  • Route server - A = VM physically on den1.sv01 (hypervisor #1)
  • Route server - B = VM physically on den1.sv02 (hypervisor #2)
  • Route server - C = VM physically on den1.sv01 (hypervisor #1)
  • Route servers ASN - 394594
  • Route servers policy ASN - 64599
  • Route servers IPv4 MTU is 1472
  • Route servers IPv6 MTU is 1452

 

Basic Protection For the IX and You

The safest way to connect to an IX is with a router, and we prefer you connect to us with a router. However we do recognize many switches can perform most all of the routing functions that most enterprises need from a router these days.  We just ask if you are connecting with a switch that you make your switch’s port facing IX-Denver as router like as possible; for example 1 MAC address, L3 terminated p2p connection, no loops, no spanning-tree, no discovery protocols, but mostly a p2p broadcast domain between the IX and your port. If you implement these basic best practices then our other basic protections listed below will likely be invisible to you and your network.

 

Current Implemented IX-Denver Fabric Protections:

  • BUM (Broadcast, Multicast, Unknown Unicast) traffic is rate limited to 5% of interface bandwidth.
  • 1 MAC address is permitted on a port configured as an access port and this MAC can be dynamically learned by IX-Denver’s switch(s).
  • 1 MAC address is permitted on a port configured as a trunk port but the MAC address of your router must be configured statically by IX-Denver.
  • Access ports on VLAN 100 will receive a link down for 3 minutes if more than 1 MAC address is received.
  • No BPDUs are accepted on any port - If IX-Denver receives a BPDU from your router this will result in a link-down for 5 minutes.
  • IX-Denver runs VSTP (RSTP) on all VLANs with the intent that the port will never receive a STP negotiation in response, this is a strategy in conjunction with our bpdu-block. If we receive a response to the bpdus or if we see our own bpdus come back to us the port will never leave a blocked state, this should prevent any hard loops or other accidents in most circumstances.
  • ICMP echos directed at switch IPs are rate-limited for both IPv4 and IPv6, testing to the switch with ICMP is accepted for brief periods if needed.
  • A small dedicated queue is available for network control traffic, this traffic must be sourced from your IX /24 or /64 IP and it must have dscp markings of cs6 or cs7 to be eligible to be forwarded in this queue.
  • The route server(s) accept ICMP, ICMP6 and TCP/179 traffic only.

 

Route Servers

As of March 2017 IX-Denver is introducing two new route servers, these servers are based on bird release 1.6.3 and will be referred to as Route Servers B and C.  With the introduction of these two new route servers there is new or improved functionality, improved default security filtering, physical hardware redundancy, and a roadmap to very secure filtering enforcement based on IRRDB, peeringdb, and ROAs.
 
The strict filtering enforcement features supported on route servers B and C will be phased in over time to ensure filtering at the route server level is accurate amongst our peers.  While we will not be enforcing IRRDB filtering in phase 1 we do require a network connected to route servers B and C must have an as-set object defined with one of the IRRs, filters will be built from route-sets as well if available but that is in addition to the mandatory as-set.
 
Route server A will remain as-is until the time it is appropriate to upgrade it or otherwise retire it.  

 

Route Server - A

Route Server A Communities

IX-Denver was assigned a 4-byte ASN (394594), due to the way 4byte ASN and standard  extended communities work we have implemented a route server ‘Policy ASN’ of 64599, this will allow you to use our policies to selectively advertise using 64599 as the stand-in for our ASN in your community configuration.

Action Communities

Action

0:<other member ASN>
Do not advertise to peer ASN
64599:<other member ASN>
Advertise to peer ASN
0:64599
Do not advertise to any peer ASN
64599:<your ASN>
Advertise to all peer ASNs
 
Note: Members with 4byte ASNs will not be able to use these policies at this time, your routes will be advertised to all peers by default (see bgp large community support on route servers B and C)

 

 

Route server A has a default prefix limit of 1000 for IPv4 and 100 for IPv6, your session will be restarted if you exceed these limits, custom limits are available.

 

Route Server A IPv4 Martian / Bogon Addresses

The following IPv4 addresses will never be accepted or re-advertised by the route server:
0.0.0.0/0+
10.0.0.0/8+
100.64.0.0/10+
169.254.0.0/16+
172.16.0.0/12+
192.168.0.0/16+
198.51.100.0/24
203.0.113.0/24+
223.0.0.0/8+
224.0.0.0/4+
240.0.0.0/4+

 

Route Server A IPv6 Martian / Bogon Addresses:

The following IPv6 addresses will never be accepted or re-advertised by the route server:
::/128
::/0{0 -15}
::/0{65,128}
0000::/8+
3ffe::/16+
2001:db8::/32+
2001::/33+
2002::/17+
fe00::/8+  

 

Route Servers - B and C

Route Servers B and C feature support - Phase 1 - March 2017

  • Physically diverse server hardware - Note RS A and RS C share a physical hypervisor
  • Support for BGP large communities as described in RFC8092
  • Partially automated config generation using arouteserver
  • Passive bgp sessions
  • Strict next-hop policy - BGP next-hop and client address must match
  • IPv4 prefix-lengths 8 - 24 accepted
  • IPv6 prefix-lengths 12 - 48 accepted
  • Max as path length of 32 ASNs
  • Max prefix configuration is built per the client network’s defined values on peeringdb.com
  • Reject routes with Tier 1 ASNs in the as path (unless the left most ASN)
    • ‘Tier 1’ ASNs = 174, 209, 286, 701, 1239, 1299, 2828, 2914, 3257, 3320, 3356, 3549, 5511, 6453, 6461, 6762, 6830, 7018, or 12956
  • IRRDB ASN validation BGP community tagging
  • IRRDB route prefix validation BGP community tagging
  • Periodic IRRDB pulls

Route Servers B and C feature support - Phase 2 - Future

  • Fully automated configuration generation
  • BGP community based blackhole filtering
  • RPKI status validation
    • BGP community tagging
    • RPKI enforcement opt-in per client
  • IRRDB ASN validation enforcement for all clients
  • IRRDB route prefix validation enforcement for all clients
  • Hourly IRRDB pulls

A special thank you to Pier Carlo Chiodi for his work on the arouteserver project which is being used to build the configurations for route servers B and C. If you’d like to look into this project and/or help the arouteserver project, please refer to arouteserver’s github.

 

Route Server B and C Communities

Route servers B and C support standard, large, and extended BGP communities.

Action Communities

Action

Std - 0:<other member ASN>
Lrg - 394594:0:<other member ASN>
Ext - target:0:<other member ASN>
Do not advertise to peer ASN
Std - 65500:<other member ASN>
Lrg - 394594:65500:<other member ASN>
Ext - target:65500:<other member ASN>
Advertise to peer ASN
Std - 0:64599
Lrg - 394594:0:64599
Ext - target:0:394594
Do not advertise to any peer ASN
Std - 65501:64599
Lrg - 394594:65501:64599
Ext - target:65501:64599
 
Prepend once to all peers
 
Std - 65502:64599
Lrg - 394594:65502:64599
Ext - target:65502:64599
Prepend twice to all peers
Std - 65503:64599
Lrg - 394594:65503:64599
Ext - target:65503:64599
Prepend thrice to all peers
Std - 65501:<other member ASN>
Lrg - 394594:65501:<other member ASN>
Ext - target:65501:<other member ASN>
 
Prepend once to peer ASN
 
Std - 65502:<other member ASN>
Lrg - 394594:65502:<other member ASN>
Ext - target:65502:<other member ASN>
Prepend twice to peer ASN
Std - 65503:<other member ASN>
Lrg - 394594:65503:<other member ASN>
Ext - target:65503:<other member ASN>
Prepend thrice to peer ASN

 

Informational Communities

Information

Std - 64599:100
Lrg - 394594:64599:100
Ext - target:64599:100
Route prefix is present in member’s IRRDB route-set
Std - 64599:101
Lrg - 394594:64599:101
Ext - target:64599:101
Route prefix is not present in member’s IRRDB route-set
Std - 64599:102
Lrg - 394594:64599:102
Ext - target:64599:102
Origin ASN is present in member’s IRRDB as-set
Std - 64599:103
Lrg - 394594:64599:103
Ext - target:64599:103
Origin ASN is not present in member’s IRRDB as-set
Std - 64599:200
Lrg - 394594:64599:200
Ext - target:64599:200
RPKI validated route prefix (future)
Std - 64599:201
Lrg - 394594:64599:201
Ext - target:64599:201
RPKI invalid route prefix (future)
Std - 64599:202
Lrg - 394594:64599:202
Ext - target:64599:202
RPKI unknown route prefix (future)

 

Route Server B and C IPv4 Martian / Bogon Addresses

The following IPv4 addresses will never be accepted or re-advertised by the route server:
0.0.0.0/0
0.0.0.0/8 - /32
10.0.0.0/8 - /32
100.64.0.0/10 - /32
127.0.0.0/8 - /32
169.254.0.0/16 - /32
172.16.0.0/12 - /32
192.0.2.0/24 - /32
192.168.0.0/16 - /32
198.18.0.0/15 - /32
198.51.100.0/24 - /32
203.0.113.0/24 - /32
224.0.0.0/3 - /32

 

Route Server B and C IPv6 Martian / Bogon Addresses:

The following IPv6 addresses will never be accepted or re-advertised by the route server:
::/0
::/8 - /128
64:ff9b::/96 - /128
100::/8 - /128
200::/7 - /128
400::/6 - /128
800::/5 - /128
1000::/4 - /128
2001::/33 - /128
2001:0:8000::/33 - /128
2001:2::/48 - /128
2001:3::/32 - /128
2001:10::/28 - /128
2001:20::/28 - /128
2001:db8::/32 - /128
2002::/16 - /128
3ffe::/16 - /128
4000::/3 - /128
5f00::/8 - /128
6000::/3 - /128
8000::/3 - /128
a000::/3 - /128
c000::/3 - /128
e000::/4 - /128
f000::/5 - /128
f800::/6 - /128
fc00::/7 - /128
fe80::/10 - /128
fec0::/10 - /128
ff00::/8 - /128