Rebuilding my PC - You guys be interested in watching it streamed?

[xartin]

Considering you have your CCIE perhaps you could assist me with something? I've been having a problem with packet drops on my lan which i'm assuming is related to my vlan setup. I mainly run my entire data subnet on vlan 1, I have physical trunk lines run through my house from the 3550 to each of the 2924 switches alongside a data line for each switch.

My 3550 does have the emi router image enabled with all data routed to the 3620. the 2924 switches are set to use the 3550 switch for a router which is appartently the correct configuration on the switches when you have one emi enabled Level 3 switch with two access level 2 switches.

What i'm sure i'm missing and this has been a project i've been putting off adding as crawling through my attic isn't a favored task for a fat kid

I need to add another line between my 2924 switches so spanning tree works properly because my trunk lines will not route data due to spanning tree disabling one of them.

Where I could use some help is with my 3620 router config. I need to setup the 3550 switch and 3620 router so it uses preferably one ethernet interface for both trunk and data which will be a routable data line.

It's unfortunate that my 3620 isn't supported by CNA. It works fine otherwise. given my level of experience with linux i've for the most part setup everything using putty.

[xartin]

After messing around for a couple hours setting up my trunk lines again and reviewing link graphs with cisco network assistant it appears as if the 2924 switch on my desk is dropping around 900,000 packets per second with either of the two ethernet cables when either is configured to access mode between the 2924 and the 3550.

There's no packet errors just a consistent amount of packet drops. I have had this switch for several years and it's not unreasonable to assume it's a bad switch but I want to be certain before I replace it. I've tried disabling routing on my 3550 distribution switch and rechecked but the same packet drop rate seems to be persisting. Is there a packet data rate limit on vlan 1? I'd love to get this figured out because I upgraded my internet connection on Friday to 50MBit down and 3MBit up and I'm strangely only able to get 20Mbit from my desk.

[Tim]

Well real quick, do a paste of the "show int" on the interface that is doing the packet drops.

[xartin]

# switch on my desk

odin#sh int fa0/1
FastEthernet0/1 is up, line protocol is up
Hardware is Fast Ethernet, address is 000d.65a0.e241 (bia 000d.65a0.e241)
Description: data line to gargolye
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Auto-duplex (Full), Auto Speed (100), 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:17, output hang never
Last clearing of "show interface" counters never
Queueing strategy: fifo
Output queue 0/40, 0 drops; input queue 0/75, 0 drops
5 minute input rate 572000 bits/sec, 48 packets/sec
5 minute output rate 18000 bits/sec, 32 packets/sec
1118085284 packets input, 2994896399 bytes
Received 6109239 broadcasts, 0 runts, 0 giants, 0 throttles
39 input errors, 39 CRC, 0 frame, 39 overrun, 39 ignored
0 watchdog, 6038116 multicast
0 input packets with dribble condition detected
648125372 packets output, 2719217673 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

Besides this switch I also traced the source of my low download speed with some assistance from my isp's tech support hotline. The guy i spoke with this morning claims my gateway interface is making my modem have unrecoverable packet errors. I'm keen to believe him because I get 40MBit down with the modem plugged directly into my notebook and only 20MBit down inside my house. The unfortunate side to this scenario is i have a motorola surfboard 6120 which is a VERY new DOCSIS 3 modem so there's not a lot of information about them online. my isp also greedily locks down the modems config pages so I have to call them every time I want assistance with the bloody thing

# annoying misbehaving router

matterhorn#sh int fa0/0
FastEthernet0/0 is up, line protocol is up
Hardware is AmdFE, address is 0010.7b3f.b041 (bia 0010.7b3f.b041)
Description: Outside interface to VLAN/DMZ$ETH-WAN$
Internet address is xx.xx.xx.xx/22
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/25/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 554000 bits/sec, 87 packets/sec
5 minute output rate 17000 bits/sec, 35 packets/sec
7568474 packets input, 476921351 bytes
Received 1405514 broadcasts, 0 runts, 0 giants, 0 throttles
5909 input errors, 0 CRC, 0 frame, 5909 overrun, 0 ignored
0 watchdog
0 input packets with dribble condition detected
3036654 packets output, 186595973 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

FastEthernet0/0 is up, line protocol is up
Internet address is xx.xx.xx.xx/22
Broadcast address is 255.255.255.255
Address determined by DHCP
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is 192
Inbound access list is 107
Proxy ARP is disabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are never sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is enabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Feature Fast switching turbo vector
IP multicast fast switching is disabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is enabled, interface in domain outside
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
BGP Policy Mapping is disabled

[Tim]

Well your interfaces don't show any errors as you described. What are you using to quantify that you are dropping 900kpps?

[xartin]

Cisco Network Assistant's link graphs were showing the packet loss but as the above interface data suggests it appears as if CNA is incorrect.

I've done some further testing in the last week since posting last with regards to the larger issues with the new motorola surfboard 6120 docsis 3 modem my isp was kind enough to drop in my lap which is become a more pressing concern since the isp's IT support hotline has no interest in assisting with figuring out the problem.

why a standard cisco router connected to a sb6120 ( now two different routers with different ios software revisions) both with 100MBit ethernet interfaces cause the router to nearly lock up from the ssh session and console port as well as this

303190 input errors, 0 CRC, 0 frame, 61074 overrun, 242116 ignored


That's a LOT of packet input errors and ignored packets for a good working used router purchased from a reputable ebay seller... I bought the 2621XM early last week to determine if my 3620 had some potential hardware flaw. I had planned on upgrading the 3620 for some time due to it's age and available IOS upgrades so it's not a complete loss to find out it's still not working correctly and hasn't entirely solved my problem at hand.

Also both routers will not manage to sustain higher than 27MBit download from my modem. MY isp plan is 50 up 3 down. I get excellent 9ms ping to the speedtest.net server in my city but as soon as I download anything else I'm doing instantly gets timeouts or latency issues. Understandably trying to stream to JTV which was my initial plans for this modem upgrade does work but playing wow while streaming I get upwards of 1000ms latency in game.

Considering what either of these model of routers are capable of by design specs there should be some logical answer. The 2621XM running advipservicesk9-mz.124-25c definitely has some better ip layer performance tuning abilities that my 3620 will never have which has been beneficial in attempting some further interface tuning but i've come up empty handed even after attempting some interface queue tuning, trying to set iomem manually to 15 which should me far more than whats required considering the 2621XM router having 128MB of dram, smart init on this router defaults to 3...

I've tried forcing 4098 hold queue in the interface connected to the router to see if the interface routing queue was insufficient, ip cef routing and ip classless are enabled, I've redone my nat translations with a route map to ipsec specifications as well as without a route map but still no winning dice roll... Setting up cisco QoS is far beyond my level of experience as well. The only curious thing that stands out is the cpu proc list often has the ARP process among the highest cpu loads.

hades#sh proc cpu sor
CPU utilization for five seconds: 7%/3%; one minute: 7%; five minutes: 7%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
3 5952 317 18776 2.04% 0.18% 0.04% 66 SSH Process
10 3464344 2408735 1438 2.04% 2.22% 2.36% 0 ARP Input
1 24 11 2181 0.00% 0.00% 0.00% 0 Chunk Manager
2 313772 9976 31452 0.00% 0.00% 0.00% 0 Load Meter
4 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN
5 408084 8589 47512 0.00% 0.37% 0.29% 0 Check heaps
6 518548 991 523257 0.00% 0.00% 0.00% 0 Pool Manager
7 0 2 0 0.00% 0.00% 0.00% 0 Timers
8 0 1 0 0.00% 0.00% 0.00% 0 Crash writer
9 3392 1711 1982 0.00% 0.00% 0.00% 0 Environmental mo
11 0 2 0 0.00% 0.00% 0.00% 0 ATM Idle Timer
12 4 2 2000 0.00% 0.00% 0.00% 0 AAA high-capacit
13 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
14 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager
15 0 2 0 0.00% 0.00% 0.00% 0 DDR Timers
16 4 2 2000 0.00% 0.00% 0.00% 0 Entity MIB API
17 7260 355 20450 0.00% 0.00% 0.00% 0 EEM ED Syslog
18 42672 14058 3035 0.00% 0.00% 0.00% 0 HC Counter Timer
19 0 2 0 0.00% 0.00% 0.00% 0 Serial Backgroun
20 0 1 0 0.00% 0.00% 0.00% 0 RO Notify Timers
21 0 2 0 0.00% 0.00% 0.00% 0 SMART

hades#sh ver
Cisco IOS Software, C2600 Software (C2600-ADVIPSERVICESK9-M), Version 12.4(25c), RELEASE SOFTWARE (fc2)
Technical Support: Cisco - Shortcut
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Thu 11-Feb-10 23:02 by prod_rel_team

ROM: System Bootstrap, Version 12.2(7r) [cmong 7r], RELEASE SOFTWARE (fc1)

hades uptime is 13 hours, 26 minutes
System returned to ROM by reload at 01:43:24 CST Wed Jan 19 2011
System restarted at 01:47:52 CST Wed Jan 19 2011
System image file is "flash:c2600-advipservicesk9-mz.124-25c.bin"


That IOS build is the second newest available for this router...

hades#sh int fa0/0
FastEthernet0/0 is up, line protocol is up
Hardware is AmdFE, address is 000d.28b3.bf60 (bia 000d.28b3.bf60)
Internet address is xx.xx.xxx.xx/22
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 7/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/449392/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 3030000 bits/sec, 341 packets/sec
5 minute output rate 59000 bits/sec, 127 packets/sec
22764588 packets input, 3429881689 bytes
Received 3357191 broadcasts, 0 runts, 0 giants, 0 throttles
303190 input errors, 0 CRC, 0 frame, 61074 overrun, 242116 ignored
0 watchdog
0 input packets with dribble condition detected
9893025 packets output, 584806415 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out


My cisco 3620 had the same or similar issues with this modem.

matterhorn#sh ver
Cisco Internetwork Operating System Software
IOS (tm) 3600 Software (C3620-IK9O3S6-M), Version 12.3(17a), RELEASE SOFTWARE (fc2)
Technical Support: Cisco - Shortcut
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Mon 12-Dec-05 21:59 by evmiller
Image text-base: 0x60008B00, data-base: 0x61A18000

ROM: System Bootstrap, Version 11.1(19)AA, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
ROM: 3600 Software (C3620-IK9O3S6-M), Version 12.3(17a), RELEASE SOFTWARE (fc2)

matterhorn uptime is 4 days, 9 hours, 48 minutes
System returned to ROM by power-on
System image file is "flash:c3620-ik9o3s6-mz.123-17a.bin"

[xartin]

At this point i'm open to suggestions tim I pretty much learned everything I know up to this point on my own with practical lab study as well as trial and error. No doubt your CCIE experience will have some ideas. My access lists i'm sure could use a bit more polish but as far as i'm aware it hasn't been a contributing issue.

My 2621XM router config. Mostly similar to my 3620 config with the exception of the IOS revision and hardware specific differences.

hades#sh run
Building configuration...

Current configuration : 6530 bytes
!
! Last configuration change at 01:50:08 CST Wed Jan 19 2011 by xxxxx
! NVRAM config last updated at 01:50:09 CST Wed Jan 19 2011 by xxxxx
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
service compress-config
!
hostname hades
!
boot-start-marker
boot-end-marker
!
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication ppp default local
aaa authorization exec default local
aaa authorization network default local
!
aaa session-id common
memory-size iomem 15
clock timezone CST -6
clock summer-time CDT recurring
no network-clock-participate slot 1
no network-clock-participate wic 0
ip cef
!
!
no ip dhcp use vrf connected
ip dhcp excluded-address 192.168.0.1 192.168.0.199
ip dhcp excluded-address 192.168.0.241 192.168.0.255
!
ip dhcp pool localpool
network 192.168.0.0 255.255.255.0
dns-server 64.59.176.13 64.59.176.15
domain-name eliteitminds.com
default-router 192.168.0.250
lease 0 12
!
ip dhcp pool STATIC-1
host 192.168.0.110 255.255.255.0
client-identifier 0100.248c.624d.e3
default-router 192.168.0.250
dns-server 64.59.176.13 64.59.176.15
domain-name eliteitminds.com
!
ip dhcp pool vdpnpool
network 192.168.200.0 255.255.255.0
dns-server 64.59.176.13 64.59.176.15
domain-name vpn.eliteitminds.com
default-router 192.168.0.250
lease 0 12
!
ip dhcp pool STATIC-2
host 192.168.0.120 255.255.255.0
client-identifier 011c.6f65.3135.8b
default-router 192.168.0.250
dns-server 64.59.176.13 64.59.176.15
domain-name eliteitminds.com
!
!
no ip domain lookup
ip domain name eliteitminds.com
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
!
password encryption aes
!
!
!
!
!
!
!
!
!
!
!
!
crypto pki trustpoint TP-self-signed-3144107307
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-3144107307
revocation-check none
rsakeypair TP-self-signed-3144107307
!
!
crypto pki certificate chain TP-self-signed-3144107307
certificate self-signed 01

- snip -

quit
memory statistics history table 12
username xxxxxxxxx privilege 15 password 7 xxxxxxxxxxxxxxxxx
!
!
ip ssh time-out 60
ip ssh authentication-retries 2
ip ssh logging events
ip ssh version 2
!
!
!
!
!
interface FastEthernet0/0
ip address dhcp
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
no cdp enable
!
interface FastEthernet0/1
ip address 192.168.0.250 255.255.255.0
ip nat inside
ip virtual-reassembly
duplex auto
speed auto
!
router eigrp 100
network 192.168.0.0
auto-summary
!
ip forward-protocol nd
!
!
no ip http server
ip http access-class 2
ip http authentication local
ip http secure-server
ip nat inside source static tcp 192.168.0.110 3389 interface FastEthernet0/0 3389
ip nat inside source static udp 192.168.0.120 41126 interface FastEthernet0/0 41126
ip nat inside source static tcp 192.168.0.120 41126 interface FastEthernet0/0 41126
ip nat inside source static tcp 192.168.0.253 22 interface FastEthernet0/0 22
ip nat inside source static tcp 192.168.0.110 46114 interface FastEthernet0/0 46114
ip nat inside source static udp 192.168.0.110 46114 interface FastEthernet0/0 46114
ip nat inside source static tcp 192.168.0.253 80 interface FastEthernet0/0 80
ip nat inside source static tcp 192.168.0.253 443 interface FastEthernet0/0 443
ip nat inside source static tcp 192.168.0.253 21 interface FastEthernet0/0 21
ip nat inside source static tcp 192.168.0.253 20 interface FastEthernet0/0 20
ip nat inside source route-map nonat interface FastEthernet0/0 overload dhcp
!
access-list 1 remark SSH Access-class list
access-list 1 permit 192.168.0.0 0.0.0.255
access-list 1 deny any log
access-list 2 remark HTTP Access-class list
access-list 2 remark SDM_ACL Category=1
access-list 2 permit 192.168.0.0 0.0.0.255
access-list 2 deny any log
access-list 20 remark SNMP ACL
access-list 20 permit 192.168.0.0 0.0.0.255
access-list 20 deny any log
access-list 118 remark NAT Access-class list
access-list 118 deny ip 192.168.0.0 0.0.0.255 192.168.0.0 0.0.0.255
access-list 118 permit ip 192.168.0.0 0.0.0.255 any
snmp-server engineID local 000000090200003080C52F80
snmp-server community snmp view cursociscodefault RO
snmp-server community xxxxxxxx@es0 RW 20
snmp-server community xxxxxxxxxx RW 20
snmp-server trap link ietf
snmp-server contact xxxxxxxxxxxx@eliteitminds.com
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
!
route-map nonat permit 10
match ip address 118
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
password 7 xxxxxxxxxxxxxxxxx
line aux 0
exec-timeout 60 0
password 7 xxxxxxxxxxxxxxx
line vty 0 4
access-class 1 in
access-class 1 out
exec-timeout 0 0
password 7 xxxxxxxxxxxxxxxx
transport input ssh
transport output ssh
line vty 5 15
access-class 1 in
access-class 1 out
exec-timeout 0 0
password 7 xxxxxxxxxxxxxxxxx
transport input ssh
transport output ssh
!
ntp clock-period 17180294
ntp server 208.80.96.96
ntp server 72.51.27.50
ntp server 209.139.208.96
ntp server 142.46.83.6
ntp server 216.18.23.161
ntp server 66.96.30.35
ntp server 72.55.148.42
!
end

[Tim]

Okay this sounds silly, but first step in troubleshooting input errors is to replace it with a new cable, as short as possible. Make sure you are using twisted shielded Cat6e.

Next step is to isolate the two interfaces with the errors. Isolate all traffic to 0. Then increase traffic to 10pps, then 100, then 1000, etc. Monitor how the input errors are occuring. If they always occur, then remove cables and check the actual interface connectors to be clean/etc.

If they always occur at high PPS, then isolate if its at an exact PPS. If you can isolate an exact PPS, then max out another interface on the route, and re-do your test while that other interface is maxed. If the PPS errors are different rates, then replace the routers memory as corrupt memory segments can lead to input errors.

Do PPS tests on both routers to figure which one is sending errors VS receiving them.

[xartin]

I figured out the problem with the router after some further research...

http://www.cisco.com/web/partners/do...erformance.pdf

The traffic I was trying to push through the router was too much for it according to the design limitations so I've ordered a cisco 3745

If that router has problems i'm definitely not going to be pleased with my isp.

[Tim]

Originally Posted by xartin

I figured out the problem with the router after some further research...

http://www.cisco.com/web/partners/do...erformance.pdf

The traffic I was trying to push through the router was too much for it according to the design limitations so I've ordered a cisco 3745

If that router has problems i'm definitely not going to be pleased with my isp.

Your best bet is to not use that PDF, and instead do the troubleshooting I mentioned. Those steps will give you the information you need to really figure out what is needed.

PPS is not a good performance indicator, just an average, since packet size makes all the difference.