topic Re: icx 7150 routing in ICX Switches

icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 20:14:45 GMT

Needing a little help here please. We have an environment with no router, but we do have a layer 3 switch. Please refer to the diagram; we have a server with IP address 192.168.1.3 connected to Ethernet port 1 of the Brocade Ruckus ICX 7150 switch. We have a fiber link on port 9 that goes out to a layer 2 switch. On port 9 we have virtual interfaces for vlan 51 at 10.174.241.99 and vlan 351 at 11.174.246.99. From the Brocade Ruckus we can ping the Camera at 11.174.246.30, so we are routing traffic as intended from the Brocade through the layer 2 switch (that has tagging) to the camera at 11.174.246.30. What we need to do is have the server connect from its IP address of 192.168.1.3 to the camera at 11.174.246.30. What configuration method would work best to achieve this goal?

Re: icx 7150 routing

netwizz — Thu, 13 Feb 2020 20:37:43 GMT

Good Afternoon:

It is hard to visualize your setup because the masks weren't mentioned, but the 7150 can serve teh same function as a router. Presumably, you create the router-interface ve interfaces??? and on int ve 51 set an IP within the subnet, and on int ve 351 set an IP within the subnet.

Then presumably, you carried port 9 to the Layer-2 switch, but I presume you used a media converter??? because there are no 7150 units with SFP or SFP+ slots to connect a fiber traceiver from port 1/1/9. On the 7150, it is common place to place a fiber tranceiver into 1/3/x the way they are numbered.

None the less, the port you are sending to the Layer-2 switch needs to carry both VLANS using 802.1q, so under each VLAN you would set that as TAGGED..

vlan 52 name something by port
tagged e 1/1/9
router-interface ve 52
!
vlan 351 name something-different by port
tagged e 1/1/9
router interface ve 251
!

int ve 52
port-name default gateway for 10.x network
ip add 10.x.x.x/yy
!

int ve 351
port-name default gateway for 11.x network
ip add 11.x.x.x/yy
!

On your layer-2 switch, I am confused that you said, "NATIVE" because that usually refers to an untagged interface that is also tagged in another VLAN. In ICX terminology, this used to be referred to as a dual-mode port, which regardless of terminology carries untagged ethernet frames into a particular vlan. That said any particular interface can be untagged into only one vlan given the constraint the switch needs to know what VLAN to sort untagged frames it receives into (as well as transmit).

***

The reason your ping is likley working is that most likely your ICX 7150 is sourcing the ping from 11.x.x.x, which is on a directly-connected network, but this is just speculation being I cannot see your environment.

Regardless, to make this work, you will need some routing to occur between your 192.168.1.0/24 (presumably I am guessing at the mask) network.

Perhaps:

vlan 192 name servers by port
untagged ethe 1/1/1
router-interface ve 192
!

int ve 192
port-name Default gateway for 192.168.1 network used by servers.
ip add 192.168.1.1/24
!

***

This assumes the server is directly connected to interface 1 on the 7150 and that you have the default-gateway defined on the server as 192.168.1.1, that the mask is 255.255.255.0, that the server sends traffic without an 802.1q tag on the frame... that your network is not more complex than diagramed.

***

Lastly, may I suggest for private use to stick with the RFC1918 IPs. 11's are Internet routable.

You want to variably subset your private network subnets from these larger subnets:

192.168.0.0/16
172.16.0.0/12
10.0.0.0/8

Hope this helps.

Re: icx 7150 routing

adam_foss — Thu, 13 Feb 2020 20:41:39 GMT

Does the server already have an existing gateway configured?
It may need a static route put in to point at the network the camera is on.

if the server doesn't have a gateway configured, Create another VE on the L3 switch for that vlan.

Re: icx 7150 routing

netwizz — Thu, 13 Feb 2020 20:52:31 GMT

Nope, this is a misnomer.

No static route should be needed giving the diagram above because there is only one (1) layer-3 device doing any routing, and this device would automatically add to its routing table the directly-connected routes for any IPs assigned to its interfaces - typically VRIs (Virtual Router Interfaces), which are your "interface ve xxx". Other vendors call these SVIs (Software Virtual Interfaces), and those are typically "interface vlan xxx." It is the same concept though.

Regardless, as a general rule of thumb once you place an IP address with its Mask on an interface, that entire subnet will show up in the routing-table as a directly-connected route meaning that layer-3 device owns that subnet.

A static-route would be to tell another layer-3 device that doesn't have that subnet or know how to get to that subnet to get to that subnet via a next-hop IP or via one of its interfaces.

In this case with this diagram, the routing table will look something like this if Op assigns 192.168.1.1/24 to ve 192, and places at least one actual interface that is UP into that VLAN, so the VE changes its state to up/up (required to get the route inserted):


SSH@OPsICX7150#sh ip route
Total number of IP routes: 1
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
        Destination        Gateway         Port          Cost          Type Uptime
1       192.168.1.0/24  DIRECT          ve 192         0/0           D    1d6h
SSH@OpsICX7150#

There would be other routing table entries for subnets directly-connected on ve 52, and ve 351, too.

****

If you are referring to the server needing a static route, while that's true, it will already have a default route, which is used by the OS to get to any network which is not in the 192.168.1.x network the server is assigned. Any other IP would cause the server to forward those packets to its default-gateway, which if the config tweaks I suggested are made would be 192.168.1.1, and the server would find that being its subnet 192.168.1.3/24 is within the same subnet as the default-gateway. That is to say the server is on the same layer-2 subnet as the default-gateway. In reality, it is just plugged into interface 1/1/1

In Windows, you can do a c:\>print route if you really want from a cmd prompt.

You would be looking for something like this
print route
Unable to initialize device PRN

C:\Users\Netwizz>route print
===========================================================================
Interface List

===========================================================================
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0       192.168.1.1      192.168.1.3   281

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 20:56:38 GMT

Very Helpful NetWizz! You are so correct, we are actually in port 1/3/1 for the fiber link. I was trying to simplify things and I made it more confusing. Thank you! We are running subnet mask of 255.255.255.0 on all subnets. You are also correct on our ve settings:

vlan 51
tagged ethernet 1/3/1
router-interface ve 51
int ve 51
ip address 10.174.241.99/24

vlan 351
tagged ethernet 1/3/1
router-interface ve 351
int ve 351
ip address 11.174.246.99/24

The vlan of the layer 2 switch is 51 for its subnet address of 10.174.241.20.

The server only has ip of 192.168.1.3 and mask of 255.255.255.0, we left the gateway empty.

We are adding this now:
vlan 192
untagged ethe 1/1/1
router-interface ve 192
!
int ve 192
ip add 192.168.1.1/24

We will set the server gateway to 192.168.1.1 and do some testing and get back to you. Thank you so much!!! You are awesome

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 21:05:08 GMT

Hi Adam. The server did not have a gateway, but with NetWizz's input we added a ve on port 1/1/1 with ip 192.168.1.1, and now have added the gateway of 192.168.1.1 to the server. We can ping the ve port of 192.168.1.1 now, but if we try to ping the tagged ve's on 1/3/1 we fail in transit.

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 21:14:23 GMT

This is what our sh ip route looks like;

01 10.174.241.0/24 DIRECT ve 51 0/0 D 1h46m

02 11.174.246.0/24 DIRECT ve 351 0/0 D 1h46m

03 192.168.1.0/24 DIRECT ve 192 0/0 D 12m52s

04 192.168.45.0/24 DIRECT e mgmt1 0/0 D 2h2m

Re: icx 7150 routing

adam_foss — Thu, 13 Feb 2020 21:18:13 GMT

just to clarify, I mentioned the static route in the case that the server was already using a different router.

I know he said he didn't have one, but sometimes things are omitted.

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 21:24:18 GMT

All good, no router here. I know it's confusing. I'm trying. If we had a router, we'd be done last week. lol

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 21:33:39 GMT

With all that is added, from the server we can ping 192.168.1.1 on the ve. From the Layer 3 switch Brocade switch however, we cannot ping the server at 192.168.1.3. Any ideas?

Switch#ping 192.168.1.3
Sending 1, 16-byte ICMP Echo to 192.168.1.3, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.

It's most strange as the server shows up in arp:

Switch#sh arp
Total number of ARP entries: 2
Entries in default routing instance:
No. IP Address MAC Address Type Age Port Status
1 10.174.241.20 000b.abec.9d67 Dynamic 0 1/3/1 Valid
2 192.168.1.3 54b2.0382.fd5e Dynamic 2 1/1/1 Valid

Re: icx 7150 routing

adam_foss — Thu, 13 Feb 2020 21:44:39 GMT

Duplicate IP somewhere?
you don't have anything else configured as 192.168.1.1 do you?
it could be responding to the pings when you think you are pinging the VE.

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 22:01:53 GMT

Pretty much a closed lab network. Just the server, ruckus, layer 2 switch, and camera. Nothing duplicate. Here's the running config:

RuckusSWitch#sh running-config
Current configuration:
!
ver 08.0.90eT213
!
stack unit 1
module 1 icx7150-c12-poe-port-management-module
module 2 icx7150-2-copper-port-2g-module
module 3 icx7150-2-sfp-plus-port-20g-module
stack-port 1/3/1
stack-port 1/3/2
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 51 by port
tagged ethe 1/3/1
router-interface ve 51
!
vlan 192 by port
untagged ethe 1/1/1
router-interface ve 192
!
vlan 351 by port
tagged ethe 1/3/1
router-interface ve 351
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
aaa authentication login privilege-mode
enable telnet password .....
enable super-user-password .....
enable aaa console
hostname RuckusSWitch
ip add-host-route-first
ip router-id 192.168.2.1
!
username admin password .....
!
sz registrar
!
interface management 1
ip address 192.168.45.44 255.255.255.0
!
interface ethernet 1/3/1
speed-duplex 1000-full
!
interface ve 1
!
interface ve 51
ip address 10.174.241.99 255.255.255.0
!
interface ve 192
ip address 192.168.1.1 255.255.255.0
!
interface ve 351
ip address 11.174.246.99 255.255.255.0
!
end

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 22:10:53 GMT

Firewall was turned on at the server blocking our pings. We can now ping from the ICX 7150 switch to the server, however the server still cannot ping the ve addresses of 10.174.241.99 or 11.174.246.99, or the end devices at 10.174.241.20 and 11.174.246.30. Any ideas? Help please.

Re: icx 7150 routing

tony_butler_9y6 — Thu, 13 Feb 2020 22:55:12 GMT

We've made a few updates since this was posted, but we still can't communicate across the subnets. I realize I left off a lot of information while trying to stick to the basics. We've figured a lot out, but we're not there yet. We made a VE on port 1/1/1 of the ruckus and assigned it vlan 192 with ip 192.168.1.1. We set the gateway of the server to 192.168.1.1. We can now ping the gateway on the switch, and from the switch we can ping the server. We still cannot ping across the subnets.

The fiber port isn't on port 9, it's actually on port 1/3/1.

We've added the static routes below on the server, but it didn't help, still can't ping the camera at 11.174.246.30.

route add 10.174.241.0 mask 255.255.255.0 192.168.1.1
route add 11.174.246.0 mask 255.255.255.0 192.168.1.1

Here is a copy of our running switch config:

RuckusSWitch#sh running-config

Current configuration:
!
ver 08.0.90eT213
!
stack unit 1
module 1 icx7150-c12-poe-port-management-module
module 2 icx7150-2-copper-port-2g-module
module 3 icx7150-2-sfp-plus-port-20g-module
stack-port 1/3/1
stack-port 1/3/2
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 51 by port
tagged ethe 1/3/1
router-interface ve 51
!
vlan 192 by port
untagged ethe 1/1/1
router-interface ve 192
!
vlan 351 by port
tagged ethe 1/3/1
router-interface ve 351
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
aaa authentication login privilege-mode
enable telnet password .....
enable super-user-password .....
enable aaa console
hostname RuckusSWitch
ip add-host-route-first
ip router-id 192.168.2.1
!
username admin password .....
!
sz registrar
!
interface management 1
ip address 192.168.45.44 255.255.255.0
!
interface ethernet 1/3/1
speed-duplex 1000-full
!
interface ve 1
!
interface ve 51
ip address 10.174.241.99 255.255.255.0
!
interface ve 192
ip address 192.168.1.1 255.255.255.0
!
interface ve 351
ip address 11.174.246.99 255.255.255.0
!

end

Any and all help/suggestions are appreciated.

Re: icx 7150 routing

tony_butler_9y6 — Fri, 14 Feb 2020 01:38:58 GMT

Our original configurations left out gateway as we had no router and were working on layer 2. After adding the layer 3 switch with ve's we never went back and added the gateways. Once we added the ve ip's as the gateways for each device in its subnet it worked like a charm.

Re: icx 7150 routing

netwizz — Fri, 14 Feb 2020 13:08:39 GMT

Glad you got it. Sorry, I hadn't responded earlier, but I had just gotten off work after spending about three hours a night sleeping much of the week. I was exhausted.

It is good that adding the Gateways worked because my next suggestions were to be to ask you if you left out some Access Lists in the posted configuration.

As for the server, you shouldn't need these:

route add 10.174.241.0 mask 255.255.255.0 192.168.1.1
route add 11.174.246.0 mask 255.255.255.0 192.168.1.1

That is because there should be a route already by default on any Windows, Mac, or Linux box for 0.0.0.0 0.0.0.0 via 192.168.1.1, which is a catch-all..

You can verify that is present with "print route"

It simply means if it is not within 192.168.1.0/24 to forward everything to 192.168.1.1

*****

I would highly recommend SSH instead of Telnet:


crypto key zeroize rsa
crypto key zeroize dsa
crypto key generate rsa mod 2048



crypto-ssl certificate generate

To validate it is enabled, you can type:


RuckusSWitch#show ip ssh

If you want, you can create an ACL to lockdown remote management. Let's say you call this access lists 99. You need only put in the permit statements because there is already an implicit deny at the end, but you can put specific hosts and subsets like this. You can use wildcard masks, too if you like.

ip access-list standard 99
permit host 10.1.2.3
permit 10.2.0.0 0.0.255.255
!
exit

Locking down web and ssh management to your ACL above:

!
ssh access-group 99
web access-group 99
!
!

I usually setup logging to syslog and a console logout:

console timeout 30
logging host 10.1.2.3

If it has Interet Access or there is an NTP server, I would certainly set the timezone and clock:

clock summer-time
clock timezone us Eastern
!
ntp
 server 10.1.2.3
!

Secure web management only should you want to use web management:

no web-management http
web-management https

Some Banner:

banner motd ^

------------------------------------------------------------------------
Some Name
Some business warning text approved by your legal department...
Asset 123456
------------------------------------------------------------------------
^

A little hardening of SSH:

ip ssh  authentication-retries 2
ip ssh  timeout 30
ip ssh  idle-time 30
ip ssh  scp disable
ip ssh  encryption disable-aes-cbc

Better logging:

logging console 
logging persistence

After this, you should use a tool such as Secure CRT or Putty selecting SSH to connect to the device.

Should you want to see network flows, you can use pretty much any NetFlow Analizer tool. Although I currently use Solar Winds Orion even though Manage Engine's NetFlow Analizer was better for me, you might find a free one. I see Solar Winds has one, but you would probably want to check the licencing.


sflow agent-ip 
sflow sample 512
sflow polling-interval 30
sflow destination 10.1.2.3
sflow enable
 
!

On the physical interfaces you want to track, you most likley need to add "sflow forwarding"