I discovered my very own Cisco IOS bug.. Hooray.. After weeks of have network address translation problems on our new Cat6509, Chris and I found a new bug in Cisco’s operating system… Ick. Needless to say this meant much pain until we worked around it by setting up a BSD box to handle the NAT Wink.
In addition to the bug, we also discovered a flaw in how Cisco implements NAT. Essential NAT (or overloaded many-to-one translation) works by hiding several machines behind a router. All conversations go through the router who remembers who on the inside is talking to who on the outside. From the outside (in this instance, the rest of the Internet) it appears as if all the conversations are with the router and from the inside (our local network) it looks normal.
It’s a good trick and normally provides a reasonable level of security because the outside world can’t connect directly to the machines behind NAT. In theory. In the IOS implementation it seems that the last machine to make a connection out gets all traffic to the router’s overloaded address that doesn’t match an active translation. Basically it boils down to a random chance that an attacker can hit any machine behind the NAT. Sure it’s harder to exploit than a non-NAT network; but then Cisco really should be dropping those packets like any sane implementation of NAT.
We found this bug when we firewalled the NAT to protect against this flaw in IOS, we also turned on IP inspection (our in non-Cisco terms: stateful firewalling). This is what caused the fatal bug in IOS and left us with a router that rebooted every half hour. Ick. Apparently the rest of the Cisco users haven’t noticed that NAT is brain dead in IOS and haven’t tried to firewall it. Cisco is working on a fix but it will be a while; hence the BSD box.
Posted on: March 21st, 2003 under lifelines, work.