Failing maintenance plan on SQL Server 2005 when databases are offline

My backup/maintenance plan recently started failing with the following (not so helpful) error message:

Code: 0xC0024104
Source: Update Statistics
Description: The Execute method on the task returned error code 0x80004002
(Unable to cast object of type 'System.DBNull' to type 'System.String'.).
The Execute method must succeed, and indicate the result using an
"out" parameter.

It turns out the error was as a result of one of my databases being offline – the plan was set to work on all databases (the default), but there’s a checkbox that tells SQL Server to skip databases that aren’t online, and it’s not checked by default when you create your maintenance plan. Checking it solved the problem and the maintenance plan ran normally again.

To find the box, open your maintenance plan, and then in each task that runs in your databases, open the properties and click the dropdown, then check the box as in this picture:

Changing SQL Server data file locations in clustered instance using service SIDs

Recently I added a second LUN to a clustered instance of SQL Server to isolate the logs files (and another for the TempDB) and I mounted them to an empty NTFS folder as I’ve done before, but when I started SQL Server, I recieved the following error:

CREATE FILE encountered operating system error 5 (Access is denied.)
while attempting to open or create the physical file
'X:\MSSQL10_50.INSTANCENAME\TempDB\tempdb.mdf

The error means that SQL Server doesn’t have NTFS rights to the location of the TempDB, but when I tried to add those rights, the permissions weren’t granted to the domain proxy account as I’d expected, but were instead granted to the service SID account, MSSQL$InstanceName. I attempted to grant the permissions to this account at the new location, but couldn’t get it to resolve to an actual account. What finally worked was:

  1. In the permissions change dialog box, change the “Location” from the domain to the local machine (even though it’s a cluster and your using a domain account to run the service)
  2. In the text box, type “NT Service\MSSQL$INSTANCENAME” and click “Check Names”

Even though that appears to be a local account, it will resolve properly on all the cluster nodes involved. This step, as opposed to using the domain proxy account the service is running as, was necessary because (during the initial SQL Server setup process) I’d selected to use the proxy account SID to host permissions rather than a domain group. The better choice permissions-wise, but the source of some confusion!

Clustering walkthrough for SQL Server 2008 on Windows 2008

I recently stumbled across a great walk-through for clustering SQL Server on newer versions of Windows. It’s really thorough – everything from setting up iSCSI (in this case, to simulate a shared disk when it’s physically attached to one node – not ideal, but lets you test the walk-through), adding the required server roles, preparing the servers, and then a walkthrough of every screen in the SQL installation process. Thanks to the writer of this awesome blog!

http://dbperf.wordpress.com/2010/07/10/walkthrough-cluster-setup-sql-win-2008/

Slow SQL Server queries over a linked server from Windows Server 2008

We recently ran into an issue where linked server queries where sometimes taking significantly longer than they should have – queries that executed in under a second on the source server took two minutes when executed from a remote SQL Server over a linked server connection. We assumed it was something to do with our Windows Failover Cluster network connections, but it was happening on non-clustered servers as well.

After some digging, we discovered that it happened when the source server was Windows Server 2008 (or 2008 R2), regardless of what OS the remote linked server was running, and the issue wasn’t tied to any particular version of SQL Server. As it turns out, it related to everybody’s least favorite network setting, TCP Auto-Tuning – disabling it on the affected source servers resolved the problem immediately, and the queries returned instantly again.

To remedy the situation, follow these steps:

  1. On the source SQL Server (the one executing the query, not the linked server), open an elevated command prompt (Right click on CMD.EXE and click “Run as Administrator”)
  2. To disable TCP Auto-tuning, run the following command:
    netsh int tcp set global autotuninglevel=disabled
  3. To disable Windows Scaling Heuristics (another feature that can speed up clients, but is no good for servers), run:
    netsh int tcp set heuristics disabled
  4. This will automatically take effect after Windows has the chance to refresh some network communications – you can either wait about an hour, or, if you’d prefer it take effect immediately, restart the SQL Instance. A reboot isn’t necessary for this setting to take effect.

I’m not sure why these settings are on by default for servers – they seem to speed up internet browsing and other client-type activities (though there’s no shortage of complaints online about them grinding Windows Vista/7 clients to a halt as well), but can cause big problems on servers.

Lightweight, single-row alternative to OUTPUT clause in T-SQL

SQL Server 2005 adds the option for an OUTPUT clause in your query to act upon table rows and return the old and new values. When I’ve done queuing in the past, I’ve used the clause to mark a row as processing and return the value, all in a single operation, so it’s lightweight and threadsafe. For example, like this:

UPDATE TOP (1) dbo.MyQueue
   SET ClaimedBy = @Server,
       ClaimTime = @ClaimTime
OUTPUT INSERTED.QueueID,
       INSERTED.SomeData1,
       INSERTED.SomeDate2,
       INSERTED.SomeData3
  INTO #OutputTable (QueueID, Column1, Column2, Column3)
 WHERE Some Criteria...

To do this, you’ll need to create a table called #OutputTable that has the right schema, which works well if you’re returning multiple rows from your query, but is a little cumbersome to work with if you’re only doing one row at a time. If you’re only returning a single row from your UPDATE query (as I am here), there’s an alternative to OUTPUT that’s easier to use – just do variable assignment inline in the UPDATE statement! The query above becomes:

UPDATE TOP (1) dbo.MyQueue
   SET ClaimedBy = @Server,
       ClaimTime = @ClaimTime
       @QueueID = QueueID,
       @OutputVar1 = SomeData1,
       @OutputVar2 = SomeData2,
       @OutputVar3 = SomeData3
 WHERE Some Criteria...

Notice the reversed variable assignment in the second query? I’ve done away with my table, and my OUTPUT clause, and now I just have the relevant values from the row I’m interested in. Much easier to work with, and as an added bonus (though I hope you’re not in this situation), it works just fine in SQL 2000.

The caveat is that it’s only good for a single row, and it only works for UPDATE – if you’re using DELETE, you’ll still need the temp table and an OUTPUT clause.