ICX7750 route-map help

[jeremiah_yeary]

Currently we are using a Cisco for our core switch, and we are trying to migrate over to a ICX7750. 

On the Cisco it allows a vlan interface to have an ACL, and a route-map applied to it, but I am unable to configure the ICX to do the same thing, it tells me they can not coexist in the interface.

as an example of the infrastructure, a crude map is:

we have 4 sites that are interconnected to 1 shared server farm with our own fiber. Each site has their own internet connection we need to keep separate due to static IPs assigned at the WANs.

site 1 ip subnet 10.10.1.0/24
site 2 subnet 10.10.2.0/24
site 3 subnet 10.10.3.0/24
site 4 subnet 10.10.4.0/24
shared server farm subnet 10.10.100.0/24

so the Cisco is configured with the associated vlans to the subnets, which have ACLs in place to prevent the sites to communicate to each other for security reasons other than a few exceptions, and they can share server resources. then any traffic that is not part of our internal network is routed though their designated WANs.

here is an example of the config:


--------------------------------------

interface Vlan1
ip access-group 101 in
ip address 10.10.1.1 255.255.255.0
ip policy route-map Internet-Vlan1

interface Vlan2
ip access-group 102 in
ip address 10.10.2.1 255.255.255.0
ip policy route-map Internet-Vlan2

interface Vlan3
ip access-group 103 in
ip address 10.10.3.1 255.255.255.0
ip policy route-map Internet-Vlan3

interface Vlan4
ip access-group 104 in
ip address 10.10.4.1 255.255.255.0
ip policy route-map Internet-Vlan4

interface Vlan100
ip access-group 110 in
ip address 10.10.100.1 255.255.255.0
ip policy route-map Internet-Vlan100

access-list 101 permit ip 10.10.1.0 0.0.0.255 10.10.100.0 0.0.0.255
access-list 101 deny ip any any

access-list 102 permit ip 10.10.2.0 0.0.0.255 10.10.100.0 0.0.0.255
access-list 102 deny any any

access-list 103 permit ip 10.10.3.0 0.0.0.255 10.10.100.0 0.0.0.255
access-list 103 permit ip host 10.10.3.50 host 10.10.4.50
access-list 103 deny any any

access-list 104 permit ip 10.10.4.0 0.0.0.255 10.10.100.0 0.0.0.255
access-list 104 permit ip host 10.10.4.50 host 10.10.3.50
access-list 104 deny any any

access-list 110 permit ip 10.10.100.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 110 deny ip any any

access-list 151 deny ip 10.10.1.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 151 permit ip any any

access-list 152 deny ip 10.10.2.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 152 permit ip any any

access-list 153 deny ip 10.10.3.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 153 permit ip any any

access-list 154 deny ip 10.10.4.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 151 permit ip any any

access-list 160 deny ip 10.10.100.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 160 permit ip any any


route-map Internet-Vlan1 permit 10
match ip address 151
set ip next-hop 10.10.1.254

route-map Internet-Vlan2 permit 10
match ip address 152
set ip next-hop 10.10.2.254

route-map Internet-Vlan3 permit 10
match ip address 153
set ip next-hop 10.10.3.254

route-map Internet-Vlan4 permit 10
match ip address 154
set ip next-hop 10.10.4.254

route-map Internet-Vlan100 permit 10
match ip address 160
set ip next-hop 10.10.100.254


--------------------------------------------------------------

so on the Cisco the route map policy takes precedence, then if it is not part of the allowed "set" then it moves on to the ACL assigned to the vlan interface

this way we can keep the internal from hitting our wan routers, and have our ACLs handle the permissions and restrictions for vlan cross communication.

This is a crude example, but if we can get some guidance on how to do this on the ICX7750 with the L3 firmware, that would be great!
 

[netwizz]

Q: So the "interface Vlan1" is similar to "Interface ve 1" on fastiron, to compare. 

A:  Yes.  And Cisco calls it a Software Virtual Interface (SVI).  On the ICX side they are Virtual Router Interfaces (VRI)... but they are the same in functionality




Q: so by default any device that targets 10.10.1.1 as their gateway without the ACL applied can talk to all other vlans that are known to the router.

A:   YES!  NO by technicality... Yes, by functionality...  This is generally referred to as inter-VLAN routing, but it is really talking to other subnets known to the layer-3 switch's routing table not the different VLANS.  Layer-2 Traffic (PDU type is "FRAMES" like Ethernet_II frames), is addressed via its source and destination pysical "MAC" addresses, which are not routeable!  They are switchable within a Local-Area-Network segment...

However, these Layer-2 frames contain "encapsulate within them" IP packets ("Layer-3") with contain source and destination IP addresses...  When a device triest to talk to a device on another subnet (a different network), the device does an ARP request for the IP of the default-gateway and sets the Layer-2 frames to be destined to the default-gateway's physical "MAC" address, but the IP packet actually has the destination IP address (belonging to the differnet network).

When the frame hits that Layer-2 boundary, the switch completely strips off the Layer-2 frame discarding and removing it.  It then compares the packet to its routing table to figure out what to do with it.  Presuming it sees this is a directly-conencted network on a different SVI or VRI, it re-wraps "encapsulates" that Layer-3 packet back into a brand-new Layer-2 Frame and drops it into the proper VLAN.  ARP ensues resolving the physical addresses and ultimately it gets correctly delivered to the proper device with the return being that device's default-gateway.


Q:
so vlan 1 = devices targeting 10.10.1.1 as gateway 
vlan 2 = devices targeting 10.10.2.1 as gateway
vlan 100 = devices targeting 10.10.100.1 as gateway

with no ACL in place they automatically route between, and any device can potentially communicate to another device on the other vlan with no restrictions in between.

A: YES, and they can also communicate within their own VLAN even if you place Layer-3 controlls such as access-lists.


Q: once i build the the access-list of 101 and apply it to vlan 1 on the ingress using the command "ip access-group 101 in", it automatically cuts vlan 2 traffic out, which is intended because they are completely different organizations that should not intrude on each other, but both organizations help fund shared server space in the farm, so they get access to vlan 100. Shared services like file servers, ip phones, active directory, end user management servers, exchange, etc. We do use a combination of ACL and Active Directory to restrict to appropriate use and access to the services.


A:  Understood, but an ACL like this is going to cause massive problems:
access-list 101 permit ip 10.10.1.0 0.0.0.255 10.10.100.0 0.0.0.255

It will allow communication out, but it will not allow return traffic back.  You can track TCP states for such things as "established" bits being set, but at the very least if you use probably want to look into setting up reflexive Access-Lists if this is your control mechanism.  What it basically does is if a device A talks to B to request a web page from port 80 it might look like this

TCP 10.10.1.123:48293 ==> 10.10.2.111:80

Basically, the OS generates an ephemeral port such as 44829 shown above.  When the web server responds, it sets the destination port back to that device's ephemeral port.  This is what allows large numbbers of concurrently connnected connections to servers and for a firewall to do NAT translation sharing an Internet IP between hundreds or thousands of computers yet be able to return the packets back to the correct devcie.

That said, I do not think Access-Lists are the appropriate answer.

For different organizations, I hihly recmmend at least utilizing VRFs, which are diffent virtual routing tables.

http://docs.ruckuswireless.com/fastiron/08.0.80/fastiron-08080-l3guide/GUID-FA7A07AF-9EBB-4DDC-81B6-...

Realistically speaking separating traffic is best done with different firewall zones...



I will read and answer more tomorrow...

[hashim_bharooc1]

Hey All,

Thanks for all the good info.

Yes you can make PBR so it will send to selected routes based on your criteria.  So you should be able to do routing differently for each router interface/Ve:

Configuring a PBR policy

To configure PBR, you define the policies using IP ACLs and route maps, then enable PBR globally or on individual interfaces. The

device programs the ACLs into the packet processor on the interfaces and routes traffic that matches the ACLs according to the

instructions in the route maps.

From the guide we have 3 simple steps:

To configure a PBR policy:

• Configure ACLs that contain the source IP addresses for the IP traffic you want to route using PBR.

• Configure a route map that matches on the ACLs and sets the route information.

• Apply the route map on untagged interface or on virtual interface.

If you are still having issues maybe you should open a Case with our  TAC team and work with them.

Thanks

Best Regards

Hashim

[netwizz]

Q: On the Cisco, by adding the "ip policy route-map Internet-Vlan1" to the interface this forwards the non internal ip address ranges off to their firewall/WAN. due to the organizations funding and policies, they have to have their own internet installed at each site showing the connection and bill in their name at their address.

A:  Makes sense

Q:  so the way the rules are in effect are not in the order as they are listed when doing a show run.

the route-map is applied first using the ACL:

access-list 151 deny ip 10.10.1.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 151 permit ip any any

this works and is in use this way, for first of all their firewall at each site handles DHCP rules and static assignments, so if a device is mis-configured on the vlan with the wrong ip and/or subnet then it just does not work. I would rather have something not work than have a vulnerability.

so as the "permit route-map" is implemented, it is the same on a cisco as it is on the fastiron. The "denied" range does not get sent to the "set ip next-hop" but continues on to the normal core routing. the "permit" range of everything else not of our ip gets sent to the firewall to deal with.

A: If a device is misconfigured on a VLAN with the wrong subnet, it will not have any routing even without a route-map.  It will not be able to even ARP for the gateway.  Your ACL above puts everything into the route-map except 10.10.1.0/24 to 10.10.0.0/16

Q: On the Cisco, the above mentioned "denied" items in the "route-map" then move on to the rule of "ip access-group 101 in" on the interface, which then helps regulate what ips or ranges or protocols they can communicate to and with. plug one end of the straw, and no fluid moves through, since it requires bi-directional traffic to function in our environment. if the access-group does not define the traffic, then it moves on to the default routing tables on the switch that sends it off to another firewall for analysis.

A:  Good to know.  Thanks.  On the ICX, you can probably apply “ip access-group 101 out” but not “in” at least the documentation is specific the “in” direction does not work.

...

Q: the rules that I have work on the Catalyst, i just need to get them to work, so i can swap it with the ICX.

A:  You may need to reverse your access lists taking into consideration all possible sources.

Realistically, your ACL most likely acts like a one-way check-valve not allowing return traffic.  For that there are reflexive Access Lists that dynamically add return entries that age-out upon completion of return traffic.  This is the most straight forward resource I found on those:

http://packetlife.net/blog/2008/nov/25/reflexive-access-lists/

BTW, I doubt reflexive access lists will work with route-maps; since, the dynamically created list that allows traffic to flow the opposite direction most likely needs to be applied inbound.

.............

I would probably revamp this and try VRFs to visualize the routing table, so in essence each organization has a separate virtual router.  Here is an unrelated example:

You might need more device memory set asside to run VRFs depending upon Fastiron build family:

system-max ip-route 15168

vrf Organization46

 rd 46:1

 address-family ipv4

 ip route 0.0.0.0/0 192.168.255.1

 exit-address-family

exit-vrf

interface ve 46

 vrf forwarding Organization46

 ip address 192.168.46.1 255.255.255.0

 ip helper-address 1 192.168.255.1

!

Just remember if you do that you would need to look at the routing table by name:

SSH@TEST#sh ip route vrf organization46

Total number of IP routes: 3

Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric

BGP  Codes - i:iBGP e:eBGP

OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2

        Destination        Gateway         Port          Cost          Type Uptime

1       0.0.0.0/0          192.168.255.1   ve 101        1/1           S    27d20h

2       192.168.46.0/24    DIRECT          ve 46         0/0           D    255d

3       192.168.255.0/30   DIRECT          ve 101        0/0           D    27d20h