Jim DeVries' Infrastructure and Technology Blog

Windows Vista Service Pack 1 Beta Overview

I found this document while looking for something else and thought people might be interested in it.  It (obviously) discusses the changes we'll see with the upcoming SP1 for Vista.  There are some "features" in the gold release that I'm excited to see updated.

One thing that doesn't receive much fanfare in this document is the addition of Secure Socket Tunneling Protocol.  SSTP is similar in function to the PPTP VPN protocol that many small businesses use, but it runs over port 443 (HTTPS) exclusively.  In our office, many of the consultants that work offsite cannot access the VPN, because the client's firewall disallows PPTP.  The SSTP connection will look no different to the firewall than a secure connection to Ebay.  On the server side, the most recent beta of Windows Server 2008 supports this.  It will be interesting to see in action.

Also noteworthy on page 5, two Vista updates (link 1 link 2) are mentioned.  The goal of these updates is to improve performance and reliability.  This is only the second time I've seen them referenced, so I suspect a lot of people don't even know they exist.

Software Deployment within Virtual Server

At Inetium, virtualization is a fundamental part of our infrastructure.  It allows us to create individual environments that mirror our client's production environments, keeps developers from interfering with one another's work, and uses our server hardware more efficiently than the 'one server, one environment' approach.  Virtualization has been a big success but it has brought it's own set of management "opportunities."  One of the challenges is to create and package tools so users who aren't server admins can perform routine maintenance on their virtual machines.

Because our hundred or so virtual machines aren't all members of the same domain, management tools such as SMS or Altiris, are challenging to apply.  While I was pondering this challenge, I was also kicking around the question of how we ensure that the Virtual Machine Additions (VMA) are installed on each virtual machine.  At some point, it occured to me that we could combine the methos of deploying VMA with the scripts and applications we want to make accessible to our users.

Although the Virtual Machine Additions get their own screens within Virtual Server, what's happening behind the scenes isn't very complex.  The VMA installer comes prepackaged as an ISO image.  When you check the box to install the VMA, Virtual Server attaches the ISO to the guest machine, as if you inserted a CD-ROM.  As a part of Microsoft's Autorun technology, the guest's Windows OS looks for a file called autorun.inf that tells it what to do with the CD-ROM.  In this case, the autorun file tells the server to run the application Windows\setup.exe.  The setup program, of course, leads you through the process of installing the Additions.

This technique can be easily extended to other applications.  All you need is a tool to create an ISO file from source files, rather than a source CD.  The tool I found is an application called Magic ISO Maker.  It has a Windows Explorer-like interface and allows you to drag and drop files into your new ISO.  There's a lot of other functionality I haven't even tapped into yet.  It's reasonably priced at $29.95.

 Once you have an ISO maker, you simply move the files your application needs into an empty ISO image and create the proper autorun.inf file to automatically launch the installer.  This MSDN link (http://msdn2.microsoft.com/en-us/library/aa969327.aspx ) lists all the acceptable parameters for the autorun file.  The VMA ISO has one line: "OPEN=Windows\setup.exe"  If the program you're launching is a Windows Installer-based package, you can pass along the usual command-line parameters, such as /qn for a quiet installation.

After creating the ISO file, move it into the directory %PROGRAMFILES%\Microsoft Virtual Server\Virtual Machine Additions.  Virtual Server uses the Search Paths setting to enumerate ISOs in other directories, so you can create an alternate directory if space concerns or administrative policy prohibit you from putting them on the system drive.  Make sure that NTFS permissions are set correctly so that only authorized users can create these software packages.

Once the file resides in a searchable directory, use the Virtual Server web interface to attach your ISO image to your virtual machine.  As long as a user is logged into the machine and the autorun file is properly configured, your new deployment package will start running.  (The one downfall of this method is that a user has to be logged in with the appropriate rights for the installer to run.)

There are a lot of ways we can use this method of deploying software.  One of the ideas I'm currently working on is creating a script to clean excess junk off the virtual machine images.  I also want to automate the process of defragging a disk, running the precompactor, and finally shrinking the drive.  I've also modified the VMA ISO to eliminate any required interaction on the user's part.

In my next post, I'll show I used this technique, along with Microsoft's BGInfo, to create an install package to display the Virtual Server host machine's name within the guest machine.

Business Desktop Deployment

Do you qualify for customized planning services from a Microsoft Certified Partner to assist you with your Business Desktop Deployment project?

With the power of BDD 2007, light touch or zero touch deployment you can dramatically reduce both the cost and time of desktop deployments and reduce the number of support calls at the same time.  The amazing thing we have found is that benefits are recognized throughout a range of clients from smaller environments using Small Business Server all the way up to a recent client of ours who had 17,000 desktops.

In a recent New Horizons event that Inetium presented at, we found that less than 50% of the attendees were aware of all the benefits of their Software Assurance.

For those who want to understand if you qualify for free desktop deployment planning services (DDPS) from Microsoft, you can contact your Microsoft Licensing expert, OR contact Rick Flath from New Horizons for assistance.  Rick provides strategic planning services for those who want to better understand what they have and how to make sure they are maximizing the value of what they have already paid for.

p.s. For those who attended the recent New Horizons event, the slide show is attached to this blog post.

I'm Being Trained by my Car

Over the weekend, I bought my first new car, a 2007 Toyota Camry Hybrid.  I've been thinking about a hybrid for quite a while but I'm quite tall and the really small vehicles such as the Honda Insight didn't look too appealing.  I test-drove the Camry a couple of times and had serious misgivings about the size.  However, once I found a model without a moonroof, I got the headroom I wanted and the seat behind me was usable once again.  All obstacles cleared! 

The thing I find most interesting about the car is all the cues it gives you about the efficiency of your driving.  There is a big gauge to the left of the speedometer that ranges from 0 MPG to 60MPG and even further on to the "E Zone"  The needle can swing dramatically, based on how you use the gas pedal and the terrain.  It bums me out when I pull away from a light nice and easy, but still only manage to get 10MPG until I reach my cruising speed.  I've also learned that maintaining a steady speed can be done on battery power alone, when crossing flat terrain.  (Lesson learned: don't waste a good flat by accellerating through it.)

 There is another display that shows a number of statistics about fuel consumption but they are a little longer-term than the MPG gauge.  My favorite display is the Cruising Range one -- it's nice to see I can drive another 550 miles on a tank of gas!  But the display I use most often is an animated display showing the tranfer of energy throughout the system.  When the gas engine or the electric motor are powering the wheels, an arrow pulses from the engine and battery icons to the motor.  When you decellerate, either by coasting or using the brakes, you're shown the transfer of energy back from the wheels to the battery.  This is how I begin to understand what conserves energy and what consumes it.  I wouldn't have guessed that the car can recapture energy just coasting down a small hill, but now I back off the accellerator until I reach the bottom.  At the end of each trip, the display gives you the overall MPG for the trip.  Any trip over 35MPG also garners you an "EXCELLENT!"

As I was considering all these gauges and displays, it occurred to me that most any vehicle could be fitted with these types of instruments.  Drivers would then have accurate guidance on how to increase their fuel economy.  For me, it's almost like a video game, albeit one without a lot of action.  Rather than mash the accellerator, I attempt to finess better mileage by watching the gauges and reacting accordingly.  It's something car makers should consider for future vehicles, hybrid or not.  Training people to get even 10% better fuel economy would have a huge impact on emissions and oil dependency.

Recipe for ReadyBoost

I've begun to think that the new ReadyBoost feature of Windows Vista requires eye of newt or some other magical ingredient.  Actually, it's not the operating system so much, but the hardware itself -- until ReadyBoost came along, there was just no reason to get into the nitty-gritty performance aspects of the ubiquitous USB memory stick.

For some background, ReadyBoost is a new strategy Microsoft devised to increase the performance of the paging file.  It accomplishes this by writing a copy of the paging file out to a flash device, as well as putting it to the hard drive.  When Vista wants to pull that information back into RAM, it can be retrieved from flash much faster than the hard drive because there are no moving parts to wait for.  For Vista users, the result can be a substantial increase in performance, depending on the particulars of the PC.

These two metrics, seek time and sustained throughput are numbers we don't ordinarily think about with respect to flash memory.  I don't think anyone one expects high performance from a USB stick.  Vista changes this laissez-faire attitude, out of necessity.  It requires a device of 256MB or greater, a minimum of 2.5MB/s throughput and, I assume, a seek time that is less than that of a hard disk, or about 15ms.

I went into this thinking that ReadyBoost was going to be a piece of cake.  Here's a list of all the devices I ended up trying out:

1. Sandisk Cruzer Micro - 4GB (link)
2. Centon Pro flash drive - 1GB (link)
3. Sandisk Ultra II CompactFlash card - 1GB (link)
4. Sandisk SD Card - 512MB (link)
5. Sandisk CompactFlash card - 256MB (out of production)
6. Apacer CompactFlash card - 256MB
7. Sandisk 8 in 1 USB card reader (link)
8. Inland Products 'Multi in 1' card reader (link)
9. Dazzle CompactFlash <> PCCard adapter

After trying all these devices out in every possible combination, I found a number of things that were counterintuitive to me:

1. The Cruzer Micro worked, but not until I tested it the second or third time.  It failed to meet the performance requirements the first couple of times.
2. I grew a love-hate relationship with the Centon USB stick.  After I started digging into performance questions in earnest, I downloaded the program HDTach from Simpli Software.  While the Centon had a really awesome sustained throughput, (14+MB/s) the seek time was awful, approximately 65msec.  As a result, I wasn't able to get it working either.
3. Some USB sticks have both 'fast' and 'not fast' memory.  This money-saving architecture allows the stick to give good performance, but not in a sustainable manner.  This renders the stick unusable with ReadyBoost.
4. All USB card readers are not the same:  The Sandisk unit isn't even labelled for USB 2.0 performance, but in informal testing, cards in it achieved up to twice the throughput of the Island Products device.
5. My vaunted Ultra II CF card is going back - it performs at about a quarter of the advertised rates.
6. The SD Card returned the second fastest bandwidth rating of all memory devices.
7. ReadyBoost generally won't work with devices plugged in to external card readers.  Apparently if the card reader appears to be a disk drive regardless of whether there is any media, it's not your day.  This same principle got in the way of my great idea, using the Dazzle adapter.  I really wanted a device that could be permanently placed in my laptop. 

In the end, I've settled on the Cruzer Micro stick.  I didn't want an external device but there don't appear to be any alternatives for my Dell Latitude D620.  I like my laptop, but for once, I'm jealous of my Inspiron-toting coworkers that have SD Card readers built in.

Fortunately, there won't be this much variability when the new hybrid hard drives come out!

This blog entry of Tom Archer's has a more detailed accounting of the requirements for ReadyBoost.

Net Neutrality

One of the biggest issues facing the internet right now is 'Net Neutrality,'  Unfortunately, a lot of people aren't familiar with the concept or issues.  The battle is between content providers such as Google and Skype, and broadband providers such as Comcast or Verizion.  The outcome, however, affects end-users of the internet, like you and I, as well as smaller internet content providers such as www.carsforsale.com.

Generally speaking, the net is very egalitarian in its current form.  If you start an MSN Messenger chat, your data is treated with an equal priority as your neighbors' data, which might be a web page.  This equality is the 'neutrality' part of net neutrality.

 Broadband providers have a little different perspective.  They see the Googles of the world getting rich using the provider's network to deliver their data to the end users, while they get no cut at all.  There are two different ways the providers can attempt to level the playing field:

1. They can charge the content provider outright for the right to pass traffic across the broadband network, or
2. They can prioritize the traffic of a competitor so the end user experience for the competitor's customers are better than the first provider.

Both of these methods will have a chilling effect on the internet.  They can both lock small and emerging businesses from the market as well as lock end users into specific services that their ISP has established a financial relationship with.  As an end user myself, I don't want my ISP where I ought to go for my news, VoIP, etc.

This past year, the House of Representatives passed a telecommunications bill that touched on this topic but is considered weak in terms of consumer protection.  The senate is currently considering bill HR.5252, which does have meaningful protections built in. 

www.savetheinternet.com is a great site that goes into a lot more detail on this issue.  They have a list of current senators and the position many of them take on net neutrality.  The link is here.

What can you and I do?  Contact your senator!  Here's a complete list of senators, as well as their phone numbers and a link to a contact form.  Let them know where you stand on this issue.

Resources:

• http://www.savetheinternet.com
• http://www.google.com/help/netneutrality.html
• http://www.microsoft.com/freedomtoinnovate/industry/letter.aspx
• One of the original sparks of this debate: http://www.washingtonpost.com/wp-dyn/content/article/2006/02/06/AR2006020601624.html

Hurricane Katrina - Nov 2006

I had a very unique opportunity this past weekend, but first, some background.

Inetium is a member of the Pohlad family of companies, which you would most likely recognize as the owner of the Minnesota Twins.  They own many different businesses, spread across the country.  They also have a charitable foundation.  On Friday morning, about sixty Pohlad employees flew to New Orleans to spend 2 1/2 days helping victims of Hurricane Katrinia.

I wrote a brief summary of the trip to the coworkers in my office this morning; the rest of the blog entry is the content of the email.  I've modified it slightly to protect the privacy of the people we helped out. 

As most of you know, Phyliss, Jon and I were selected to participate in the second Pohlad Family Foundation trip to New Orleans, which we just got home from last night.  I’m sure I speak for all three of us in thanking the Pohlad foundation for the opportunity to serve in this way.  The trip was simultaneously humbling, saddening, and also inspiring.  It will be something none of us will ever forget.

There were approximately sixty employees representing the different business units, coming from all corners of the US.  We were divided into four teams of fifteen, and were each assigned a home to start the rehab process.  The families chosen to receive our help have neither the financial means or the physical ability to repair their homes. 

Both homes we worked on had water damage from floodwaters that were probably chest-high while standing on the home’s first floor.  After a year, there is so much mold and rot that the house has to be stripped down to the studs and bare floor.  If the home is structurally sound, the wood can be treated and the process of rebuilding the interior can begin.

Our first home was owned by a seventy year old man named Eugene.  He was born and raised in his same house, and he wants nothing more than to live the rest of his life there.  It’s a safe assumption that Eugene has lived in absolute poverty for many years; the hurricane was just the final straw.  We spent a full day removing clothing, belongings, wood and drywall.  While Eugene’s home may never be livable in the way we would think of it, we did the work that he probably wouldn’t have finished in his lifetime if he were to do it alone.

The second home that we worked on, we spent both Sunday and Monday morning on.  We didn’t get a chance to meet the homeowner but her son worked alongside us  both days.  His name was Jerome.  This house was in much better condition structurally – we all knew when we left that the house would someday be livable again.  It was different in that from the amount of possessions, a whole family had clearly grown up there.  Jerome, too, had grown up in the house we were working on, and he shared a couple of stories with us as we were working.

Along with the organizers from the Pohlad Foundation, we worked exclusively with Americorps volunteer assigned to Catholic Charities.  These volunteers were college students or recent graduates, who had set aside a part of their lives to work in the toughest of conditions.  In addition to organizing and leading the work crew, they in many cases outworked the rest of us.  They were an inspiration.

Creating a Multi-user FTP site with IIS

In our webhosting business, we have many customers who require FTP access to their files.  Our need then, is to create a single FTP site that allows many different customers the ability to manage their web content, keep other FTP users out, and be easy for us to manage.  This is the process we use:

1. Create a local 'FTP Users' group
2. Grant that group the right (found within the local security policy) to Log On Locally. 
3. Create a folder on a data drive, which must be formatted with NTFS.  This will become the root of your new FTP site.  (Your web content should always be on a drive without any executable or OS code on it.  This prevents directory traversal flaws from affecting you.)
4. On that folder, break inheritance (Properties | Security | Advanced.)  Remove the existing permissions. 
5. Add these account permissions back to the folder: 
   1. Administrators (full control)
   2. IUSR_machine (read) and
   3. FTP Users (Read)
   4. You may need to add the IWAM account with the same permissions as the IUSR account, depending on whether you're using multiple app pools or not.
6. Set the FTP users right to apply to this folder only.
7. Install the FTP service (Control Panel | Add/Remove Programs | Application Server | Internet Information Server | FTP Service)
8. In the properties of the new Default FTP site, go to the Security Accounts tab and disable anonymous connections.
9. On the Home Directory tab, point the default FTP sites' home directory at the folder you've created.  Make sure that the Read, Write, and Log visits check boxes are all checked.
10. The Messages tab has four boxes; the Banner message is displayed prior to authentication and is the place for any sort of security message.  The Welcome box can contain any usage notes or instructions to users.

At this point, we have the server prepared for use; now we have to set up a user:

1. For each customer, create a local user account. 
2. Set the account to have a non-expiring password and clear the Must Change Password box. 
3. Make them a member of the FTP users group. 
4. Create a folder under the FTP home directory, with this folder name identical to the username that will use it.  It can inherit rights from the parent folder. 
5. Double check to make sure the FTP Users group doesn't have any rights within the folder.  If it does, inheritance isn't correctly set up for that group on the parent folder.
6. Take the user account you've created and give it Modify rights on their folder and all subfolders.

Some admins use Windows Quotas to ensure that the drive isn't filled, either intentionally or unintentionally.  To test the setup, log into the ftp site with the new username and password.  A failure to log in could be one of these things:

1. The password is set to change on next login,
2. The user isn't a member of the FTP Users group
3. The permissions on "their" folder aren't right.   

You should also confirm that you can both upload a file and then delete it.

You should remember that both the username and the data tranferred between sites is unencrypted -- IIS has no facility to do so.  Sensitive data should be transmitted in an already encrypted form.  We have a couple of customers who use PGP on their files prior to transmitting; a scheduled job on our servers unencrypts them and moves the data to its final destination.  There are third-party servers such as Ipswitch's WS_FTP Server that do have encryption at the FTP application level.

In conclusion, IIS can be used to create a flexible, easy to administer, and secure FTP server. 

Dell Announces Battery Recall

Dell has announced a recall of approximately 1.2 million laptop batteries.  The affected batteries were sold between April 1st, 2004, and July 18th, 2006, with Dell Inspiron, Latitude, XPS and Precision Mobile Workstations.

The impacted models include:

• Latitude: D410, D500, D505, D510, D520, D600, D610, D620, D800, D810
• Inspiron: 500M, 510M, 600M, 700M, 710M, 6000, 6400, 8500, 8600, 9100, 9200, 9300, 9400, E1505, E1705
• Precision: M20, M60, M70, M90
• XPS: XPS, XPS Gen2, XPS M170, XPS M1710

To determine if your battery is included in this recall, go thru the following steps:

Find your battery's model number.  Remove it from the computer after powering it down.  The battery will have a sticker with the Dell part number on it, a series of five sets of characters.  Take the second set of characters, drop the first character, and see if the remaining ones are in this table:

If your battery's part number is not in this list, you are not affected by this recall.

If your battery IS being recalled, go to this Dell website and reenter the part number.  After the site confirms that you're affected, you'll be directed to an online form where you can order a replacement.  (at no cost)

For Inetium customers, we'll be happy to answer questions and help you through this process. 

Catching Up

Wow, I've been neglecting my poor blog.  Things have been crazy busy in the Infrastructure group and for me personally.  In any case, there were a couple of MS goodies that have elevated themselves on our priority list and I thought I'd share.

The first is Microsoft's new file-based machine imaging technology, ImageX.  One of my coworkers, Matt, has been using it already with some of our XP machines.  Because it's file-based, there are a lot of cool things that can be done, like updating the source image without the deploy / reimage cycle, and single-instance storage.  It's also hardware agnostic so preloading drivers for different platforms is an easier task.

Microsoft has also released a feature pack for SMS 2003 which will enable desktops to be provisioned and deployed straight from SMS.  It also integrates with the User State Migration Tool to automate the transition of user settings and files from one computer to another.

This feature pack uses RIS for deployment, so it is a unicast technology only.  Who knows, they may be able to come up with the technology - blast out the common files via multicast, and then pass along the difference via unicast.  Generally, we'd expect those unique settings to be just a tiny portion of the overall deployment.

This stuff looks good.  It rolls up a ton of manual processes and helps customers create a more tightly-bundled desktop provisioning, deployment and management process.

Kerberos / Delegation Worksheet

Finally getting back to this topic...  I created a worksheet for one of my customers that detailed the configuration of Kerberos & delegation.  There are really two tracks that need to be followed:  1) Confirm that authentication works on the client, front-end server, and back-end servers, and 2) Confirm that Active Directory, trust relationships and DNS are all configured correctly.

This list certainly isn't inclusive of all delegation scenarios but it should be helpful.

Track 1 : Client and Server Authentication

Track 2: Delegation & AD Settings

Some of the formatting blew up between Word & CS, so apologies for the table being a bit hard to read.

Understanding E-mail Communications

Slashdot has a story / discussion regarding an article from the Christian Science Monitor on the topic of e-mail communications and why it is so easily misunderstood by the recipient. 

The bottom line is that email authors tend to assume that the reader will have the same biases and perspective.  This erroneous assumption raises the potential for every ambiguous phrase to lead the reader someplace the author wasn't going.

Their suggestion for avoiding this pitfall?  Pick up the phone and call instead.

Rough Week in the Security Department

Microsoft has taken heat for many years about the number of security fixes released on a yearly basis.  In that same timeframe, Mac users felt immune to viruses & worms.  The increased popularity of Macs has raised their attack visibility, as more hackers get "under the hood" of OS X. 

My inbox today contained a link to Apple's latest security bulletin.  To summarize:  Apple today has released 25 security fixes for OS X, both the server and workstation flavors.They also released security fixes for QuickTime on Windows, which struck me as ironic.  Update: My QuickTime control panel applet offers Updates, but it says 7.0.4 is current.  I had to go to Apple's site and download a whole new install package to get the .

Microsoft, for its part, released three security fixes, two of which are classified as "critical."

One of those three was for the Macromedia Shockwave player, which was released in conjunction with Adobe, who owns the Shockwave suite..  That pretty much rounds out the trifecta.  I wonder how Adobe managed to get their patch to be deployed by MS's own delivery systems?  Perhaps Shockwave is in such broad distribution that MS found in their best interest to get the patch out there.  Update:  It's bundled with XP - makes sense for MS to distribute if they're the ones that gave it to us in the first place.

Hopefully next month will go a little smoother.

Configuring Windows Time Service

Time synchronization is an almost invisible, but critical, task on your network.  Windows 2000 and 2003 Active Directories will always attempt to use Kerberos to authenticate users from one computer or service to another.  Kerberos relies on accurate time to prevent credential spoofing.  If the two machines are more than five minutes apart in time, the receiving computer won't accept the Kerberos ticket as authentic.  Time synchronization also allows logs from different servers and network devices to be compared and trusted as an accurate sequence of events.  (Often used in security analysis.)

Getting the Windows Time Service to work correctly was something I struggled with for a long time.  Since I haven't always followed a strict change management process (ahem) my experiments probably took me further from the solution.  One of my new coworkers asked me about difficulty he was having, and it prompted me to fix things once and for all.

These are the key things to keep in mind:

• Group policy is only used when you have an unusual configuration, such as using a non-Windows time server internally.  A Cisco router would be a candidate for this role.
• The domain controller with the PDC Emulator role is the root of your internal time infrastructure.
• An NTP client application can help you diagnose networking issues.

To determine which domain controller holds the PDC role, open Active Directory Users and Computers.  Right-click on the domain object in the left-hand pane, and select Operations Master.  Click on the PDC tab - the dialog box will tell you which server you're looking for.

Windows Group Policy has a number of Time service settings you can manage.  These policies are located at Computer Policy | Administrative Settings | System | Windows Time Service.  To confirm that Group Policy configurations aren't interfering with your efforts, use the Group Policy Management Console to find any GPOs that modify any of these settings.  If you find any, modify them this way: 

• The Global Configuration Settings GPO should be Not Configured. 
• Under Time Providers,
  • Enable Windows NTP Client should be Enabled
  • Configure Windows NTP Client should be Not Configured.
  • Enable Windows NTP Server can be disabled as long as this GPO does not apply to the PDC.  You can manage this with GPO blocking or not applying the GPO above the PDC's computer object.

Log into the server with the PDC emulator role.  Download the NISTIME application from here.  This file is the actual executable, no installer required.  Run the application, go to File | Select Server, and check the checkbox labelled "Using NTP Format" alongside the first timeserver listed.  Click OK.  Next, click Query Server and select Now.  If you get an immediate response, your firewall is allowing Network Time Protocol (NTP) traffic to pass back and forth.  If the application gives a "No response" error, look into issues with your firewall(s).

Next, you will configure the PDC emulator to use an external time source.  Start a command prompt and enter the following:  net time /setsntp:time.windows.com,0x1  The 0x1 parameter is needed if you enter the NTP server's name as a fully-qualified domain name.  You can leave the comma and the parameter off if you enter an IP address.  Note that using IP addresses is not a best practice as providers may change IPs on their servers and update the DNS entry without widely announcing the change.

Next, open a command prompt and enter w32tm /config /update.  This too should return a result quickly.  Compare the time on the computer clock with the results of the NISTIME application.  When our systems are working, the time difference is  generally under 2/100ths of a second.

As a final check of the servers, enter the w32tm /monitor from the command prompt.  This command should list:

• all the domain controllers
• identify the PDC emulator
• list the time differences between the PDC emulator and the computer running the w32tm command
• list the source of each server's time.  Your PDC emulator should list an external time source, while the other domain controllers should list the PDC emulator as their source.

Assuming the server is now retrieving time correctly, the final step is to make sure clients are updating correctly.  If you made any changes to your Group Policy, run GPUPDATE from the command prompt to refresh your client's settings.  Once that has completed, check the application event log on the computer and look for an event from SeCli stating that the security policy has been applied successfully.  You can stop and start the time service by entering net stop w32time && net start w32time at the command prompt.

If the client settings are configured correctly, you can look in the System event log and find an event from source W32time stating that the time service is now synchronizing with time source {any domain controller.}  You can also enter w32tm /resync at the command prompt.

If there are large adustments to be made to the client's clock, they will not be made abruptly.  The computer will slow its clock to 1/4th to 1/2 the normal speed until the time is synchronized.  The event in the System Log will be the immediately available sign that the updates are working.

More on Kerberos and Delegation Troubleshooting

After working on three or four projects, troubleshooting delegation issues isn't as difficult as I first thought.  I'll list the tools I use and then start working thru the troubleshooting steps. (I'll assume people have a basic understanding of the technology and terminology.)  This article will cover client PC settings and the settings on the server that the client is talking directly to.  The next article will have a process for walking thru an installation that uses service accounts and/or hostheaders.  (The truly complicating factors)

Microsoft has a pretty complete set of tools for troubleshooting Kerberos issues.  The ones I rely upon are:

• adsiedit.msc
• search.vbs (a part of the Windows support tools, on the server CD)
• logon success auditing
• setspn (limited, adsiedit is much easier to work with)

This should be all you need, assuming that the underlying Kerberos infrastructure is working correctly.

For troubleshooting web applications, the first things to check are the clients' browser settings.  When you open the page you want to delegate, if you are prompted for a username / password, either Internet Explorer or IIS isn't configured to pass integrated credentials along.  If you  then authenticate to the page, check to see which security zone your browser says it is in.  Integrated authentication only works in the Local Intranet and the Trusted Sites zones.  I recommend not using the Trusted Sites zone, as the Local Intranet zone offers a bit more security to the client's browser, and has a rule to allow a hostname url (ie without the domain name, such as http://portal) to automatically be a member.

If IE claims you are in the Internet zone, the browser settings need to be adjusted.  You can either manually add the site to the Intranet zone, or deploy that change via Group Policy. 

Once the security zone is correct, confirm that integrated authentication is enabled.  Within IE, go to Tools > Internet Options > Security.  Highlight the Local Intranet icon and click the Custom Level button.  Scroll to the bottom of the list of security rights and confirm that the "Automatic Logon only in..." radio button is selected.  Cancel out of that dialog and go to the Advanced tab of Internet Options.  Scroll down to the security section and confirm "Enable Integrated..." is checked.

At the client level, the only things remaining to check is that the computer is a member of a domain that is in the same forest as the target server.  Confirm that the  variance between the server's clock and the computer's clock is not greater than five minutes (time zones notwithstanding.)

To confirm the client is configured correctly, go to the server and open up the Local Security Policy.  Drill into Local Policies and then Audit Policy.  Enable SUCCESS auditing for account logon and logon events.  In the security event viewer, this audit setting will generate entries every time someone authenticates to the local server.  The important part here is that the event record will show the authentication method used, whether it is NTLM or Negotiate.  (AKA Kerberos)

If the logon events show only NTLM entries, we need to look at some of the server's authentication settings.  On the web app or virtual directory you're trying to authenticate to, check the website's properties.  Go to the Directory security tab and click the Edit button within the Authentication box.  For integrated authentication to work correctly, it has to be the ONLY box checked.  Any other auth methods, including anonymous, will prevent kerberos from working.

According to this KB Article, the auth methods of IIS 5.0 can be confirmed by entering the following line, launched from c:\inetpub\wwwroot.  I assume IIS6 comes configured correctly, out of the box.

cscript adsutil.vbs get w3svc/NTAuthenticationProviders

If you don't get this response,

NTAuthenticationProviders       : (STRING) "Negotiate,NTLM",

you need to reconfigure the the server so it does.  The usual disclaimers about modifying your metabase apply - have good backups!  To change the auth methods, enter the following: