The Importance of Good Change Control Practices

A while back one of our Server Admins logged into a server that runs our SNMP management application. Immediately he was hit with an IP address conflict message. Some other machine had taken his IP address. He was on his toes and wrote down all the information that the message gave him (system name and MAC address). Then he sent out an email to the technology group asking who was responsible for this system and no one responded.

Needless to say it raised a red flag in my mind. We started an internal investigation to find out where the system was, what it was, what it was doing, etc... I immediately ran some scans on the system to find out what I could about it. Everything came back blank. NMap and several other scanners all reported that it couldn't tell anything about the OS because the fingerprint matched too many different things. The MAC address was reporting back as all 0's (00:00:00:00:00:00). Finally Nessus was able to tell me that it thought it was a Samba system. A quick check of the team determined that no one here was even familiar w/ Samba much less had deployed one.

Now we decided to shut down the port that it was connected to and hunt it down. Of course the cable had to be traced and it was a mess. Once we finally found the system it turned out to be an ILO port on a DB server. One of the very DB teams that we had asked about it and they denied knowing anything. That is another topic for another post.

Now we have a change control process that works pretty well. It's still young and his not fully automated yet but if the proper procedures had been followed we could have eliminated this whole fiasco. They could have had a free IP assigned for them to use and lots of time and manpower could have been saved. Not to mention the gray hairs that it added to my head. It's a good thing that I'm a blond so they don't show (no blond jokes allowed). :)

So please follow the proper procedures and policies that your company has in place. They are there for a reason and it's not all about making the auditors happy.