Project Completion

After several long, long weeks of 15 to 18 hour days and no weekends my network conversion project is finished. All in all it went pretty well. There were some problems along the way but nothing that couldn't be overcome quickly and mostly painlessly. Friday night and Saturday were long days because we closed at 2:00 pm Friday and then opened again on Saturday at 10:00 am. During that window we had to get everything done that we had not been able to do before hand. Things like replacing the drives on the PC's and finalize local network setup. The company that we were doing business with came in and pulled their drives and network equipment and left us with a mess to clean up. Much of that is still waiting to be done. After about 1:00 Pm Saturday things calmed down pretty much except for a couple of password resets.

I feel like a big monkey has been removed from my shoulders and I can finally relax. I reintroduced my self to my wife and 2 girls yesterday and we spent the whole day together. Today I'm going over my "punch list" and assigning tasks to my field tech. I may even go home a little early. :)

Today is the day

Today all of my hard work over the last several weeks comes to a head. By the end of the day our network will be completely different that it is today. There will be no remnants of our current partners network and it will all be managed in house.

All in all things have gone pretty well. We had a few unexpected things that popped up and a few things that didn't go as planned, but nothing that couldn't be worked through with minimal pain. My biggest regret is that many of the security features that I planned in had to be put on hold until after we get through this transition. Obviously we have good security in place it's just not exactly what I feel we need. I have it all planned out, have an implementation schedule that will have it all in place within a few months.

Hopefully starting next week I will be able to be back to blogging on a regular basis and about something other than this project.

Project Report 1/24/07

10 days to go. All was going well and then a couple of things didn't go as planned and we had to pull back and regroup a little. Doing all of this with limited manpower and resources is really tough. I'm learning lots but it's by trial and error.

Most everything is in place now. We're ironing out a couple of things and down to the mundane tasks for the most part. Soon we will actually start using parts of the new infrastructure in production. Hopefully that will flush out any bugs before the full roll out on 2/3.

I've got one thing that I'm struggling with. It's an ongoing issue that I was hoping would be solved when I replaced my circuits and routers, but it's still around. Here are the details. If anyone has any thoughts please contact me.

Windows 2003 Domain
13 Sites
Each site has a 2003 domain controller with DNS; WINS
All PC's are XP SP2 running Office 2003
Everything is fully patched.
DNS and WINS have been verified as working properly and replicating properly.

Both of these happen only from remote sites.
If I try to access an intranet site via a web browser I can't connect until I ping the IP address.
If I try to access Exchange via Outlook 2003 I have to ping the exchange server by name.

I've posted on several support boards for Microsoft, checked other sites, asked people who are far smarter than I am and have not found the answer yet.

I posted this on the Security Catalyst site yesterday and have gotten a couple of ideas that I'll try today, but just in case they don't lead anywhere I'm open to other ideas.

Project Update 1/17/07

Feeling much better! Even though things are still on a VERY tight schedule they are moving along right nicely. All of my WAN links are up. My AD is designed and deployed to the first 2 servers. The slow part is the hard disk images. Hopefully by the first of next week all of my remote sites will have servers and switches installed and then we will start converting PCs. We are doing between 1 and 4 per site ahead of time; depending on the number of PCs in each location. Then the evening of 2/2 we will do the rest of the machines. At that time all that will have to be done is put the drive in the PC, boot it up, join the domain and push AV.

Project Progress

The routers are configured and being installed. Active Directory is ready to go. I'm sill deploying images one at a time. Trying to get a money to help but he won't return my calls. Maybe I shouldn't refer to him as a monkey.

I'm feeling pretty good about most of this. Just need a nap so I can keep a clear head.

Had a small problem last week. My tape library died and so I had to buy another one. I got an Exbabyte - VXA 320. It's nice. Rack mounted and fast. Much better than the old HP SureStore DLT 7000 that we had been using.

Just wanted to post something so everyone would know that I'm still here. Maybe next time I'll be more alert and my writing won't seem like something from a desert island castaway.

Project Update 1/11/07

My routers have arrived and I'm pushing out images to the desktops. This is a slow process because we are replacing the hard drives only. I don't have a duplicator so I have to do the images one at a time. 87 of them have to be done. I'm still trying to get the other things that I need ordered ( waiting on management sign off) and am waiting on Sprint to finish the installation and turn up of my MPLS circuits.

I have 23 more days to get it all done.

OK, it's officially panic time. In one month my work life will have changed drastically. By Feb 2 at the VERY latest I have to have the following projects completed.

1. New WAN circuits installed, equipment configured and tested. Still waiting on Routers. If anyone has any pull at Cisco PLEASE ask them to put a super rush on my order.
   
2. New Domain built, secured, tested. This includes about 15 servers and 100 workstations.
3. Associated equipment and security to allow the new domain and the existing domain to talk.
4. New physical security systems. both alarm and video.
5. Exchange Server upgrade to handle additional users.

Much of this can be prepped ahead of time but the actual swap out of PCs has to be done on the 2nd. Once we get past the initial phase we can then focus on the "ancillary" projects that have to be completed shortly after that. Why does this justify a "panic" time? Mainly because of a shortage of manpower and the fact that my Cisco order is still up in the air on at delivery date. It's hard to connect remote offices w/o equipment. Plus due to the fact that we are a small company I have over site responsibilities in other areas that don't fall under Security or IT even. These things also require planning and coordination time if not actual "hands on" time.

All that said if you don't hear from me much hopefully you will understand. I'm hoping to blog about many of the experiences.

Vendor Selection

As I've mentioned before there are a lot of changes taking place where I work. Many of those changes involve us doing things for ourselves that have been done for us in the past. So I've spent a lot of time meeting with vendors lately. As we have gone through the process of meeting with various vendors to either provide a product or service I've been pretty impressed with most of those we have meet with. The FUD has been kept to a minimum (contrary to my post a few months ago) and the meetings have been productive for the most part.

We have to get 3 bids for most of these projects so at times we talk to several vendors and then narrow our list to the top 3 or 4 to actually invite to submit a bid. We did just this with one service that we needed and a couple of weeks ago I sent an RFP to the 3 selected vendors. Then early last week I received a bid from one of the vendors that had NOT been selected. At first I didn't think much about it because that stuff happens. But then it hit me that the bid included my internal RFP document that I had created and maintained control over. No one else in my company even had a copy of it. I quickly checked my sent items box to make sure that I had not sent it to the wrong vendor and I hadn't. Then I checked my Exchange logs and other audit logs to see if someone else in my company got a hold of it and sent it out. No evidence of it anywhere. Next I called the vendor to see where they got the document. The guy I had been dealing with there was out of the country until the end of this week and no one else knew anything.

That leaves only 2 options that I can see (if anyone else sees any others please let me know). Either the email was intercepted after it left my exchange box or one of the 3 chosen vendors shared it with this other company. The first I can live with (like it or not). The second does not sit will with me. Well I sent the vendors a letter outlining the situation and asking for them to do an internal investigation. Two of them have called back very concerned and with unequivocal denials that it happened by anyone within their company. No response from the third. Are they still investigating or is their silence convicting them?

I doubt that it came from anyone of the actual sales people or their trusted group that helps them put together a proposal, but maybe someone a little farther down the food chain who stands to make a few bucks from a "friend" if the other company actually gets the contract. Who knows. I do know that the 4th company is still not in the running. Their price was much lower, but I think that I would be getting what I paid for and that is not what I need.

If anyone has any thoughts on this or if something similar has happened to you please write me and let me know. This is a first for me and I'd love to know how others handled it.