Category Archives: Database Administration

A quick glimpse at your Log Shipping Status

Almost every shops I’ve worked in, Log Shipping is an important part of their disaster recovery plan (2012 has changed the game with availability groups). When you configure log shipping natively, it also creates a couple of alerts – Log shipping Primary Server Alert and Log shipping Secondary Server Alert (hope you have a SMTP Server and have configured database mail correctly). These alerts are invoked by a job which keeps calling the following stored proc:

exec sys.sp_check_log_shipping_monitor_alert

It does a fair job of notifying team members (DBAs/System Engineers, etc.) when the log shipping falls behind. Most of these are configurable parameters, and you can also modify these thresholds by updating metadata/system tables (be careful while doing this however)!

I have been in situations where the alert system (SMTP) connectivity stops working (briefly, until restarted), or there are some network connectivity issues. Sometimes with ad-hoc index rebuilds (online rebuilds), I try to keep close eyes on how far behind the log shipping is, without relying heavily on log shipping monitors/alerts. The following combination of T-SQL and PowerShell comes in in the process. With PowerShell, I do not have to run the same query on multiple machines anymore.

What gets called

USE [DBARepo]
GO
 
 
SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
create view [dbo].[DBA_restore_history_checker]
as
(
 
SELECT
rsh.destination_database_name AS [Database],
rsh.user_name AS [Restored By],
CASE WHEN rsh.restore_type = 'D' THEN 'Database'
  WHEN rsh.restore_type = 'F' THEN 'File'
  WHEN rsh.restore_type = 'G' THEN 'Filegroup'
  WHEN rsh.restore_type = 'I' THEN 'Differential'
  WHEN rsh.restore_type = 'L' THEN 'Log'
  WHEN rsh.restore_type = 'V' THEN 'Verifyonly'
  WHEN rsh.restore_type = 'R' THEN 'Revert'
  ELSE rsh.restore_type 
 END AS [Restore Type],
rsh.restore_date AS [Restore Started],
bmf.physical_device_name AS [Restored From], 
 rf.destination_phys_name AS [Restored To]
FROM msdb.dbo.restorehistory rsh
INNER JOIN msdb.dbo.backupset bs ON rsh.backup_set_id = bs.backup_set_id
INNER JOIN msdb.dbo.restorefile rf ON rsh.restore_history_id = rf.restore_history_id
INNER JOIN msdb.dbo.backupmediafamily bmf ON bmf.media_set_id = bs.media_set_id
WHERE rsh.restore_date >= DATEADD(dd, ISNULL(-1, -30), GETDATE())
AND destination_database_name = ISNULL('DBName', destination_database_name)
 
 
)

Caller:

foreach ($ServerInstance in get-content "C:\serverlist.txt")
{
    ##$ServerInstance = "My Hardcoded Server was here. This is commented out."
    $Database = "DBARepo"
    $ConnectionTimeout = 30
     
    $Query = "select top 1 * from dbo.DBA_restore_history_checker order by [restore started] desc"

    $QueryTimeout = 120

    $conn=new-object System.Data.SqlClient.SQLConnection
    $ConnectionString = "Server={0};Database={1};User Id=username;Password=password;;Connect Timeout={2}" -f $ServerInstance,$Database,$ConnectionTimeout
    $conn.ConnectionString=$ConnectionString
    $conn.Open()
    $cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
    $cmd.CommandTimeout=$QueryTimeout
    $ds=New-Object system.Data.DataSet
    $da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
    [void]$da.fill($ds)
    $conn.Close()
    $ds.Tables
}

As you might have noticed, the above PowerShell script can be used with almost everything, and the best part is that it allows you to iterate through the list of servers you provide.

Happy PowerShelling!

How I do my backups

Several years ago, I got my hands on one of the database servers that I did not know existed within my organization (ex-employer). It was managed by a completely different department, on a different VLAN (or else SQLCMD -L would have most likely found it), and they needed some help shrinking it – the log had grown tremendously (it was a onetime thing). After shrinking the log, I talked to the department so that the DB could be backed up (no backups in the backup history, anyone?). Backed it up, restored it to a test, and ran DBCC CHECKDB. Corruption was found. Tried various methods of repair, ultimately it came down to REPAIR_ALLOW_DATA_LOSS.

Risk was – data loss. Plus no backup. The first thing to do was to do a full backup, and restore it on a different server just in case it causes more harm (be careful of this – if you restore 2000 to 2005, or 2005 to 2008, you cannot backup from there and restore that to earlier version). After talking to developers on what the purpose of the database was, and if any quick stopping of the services were allowed, the answer was usual (24/7/365) operation. I resorted to the last possible way – create an empty shell with all the tables (transfer the schema), switch the database (so that all the inserts are not lost), do a quick repair, check against the restored copy of the database and transfer the lost data, switch it back, and then transfer all the data from the shell.

It was a painful process, and needed careful planning plus management approval. The key takeaway from this lesson was to design an automated process which not only backup up the database, but does automated restores (based on your preferences – weights/values of databases, on the given date/time), automated CHECKDB, and build reports so that you know what was happening. Additionally, send an email out if there was a problem with the restore or if the CHECKDB failed. It got more interesting in the process, and I started keeping tabs on how long the backups were taking, growth sizes, and all kind of interesting things.

The experience did help re-enforce the conviction in my mind – have a good backup plan or you are going to express highway towards unemployment. The same process that I built over several years ago has been refined as time goes by, and I occasionally dwell into network transfer speeds, etc. A basic structure of what has worked for me over several years is outlined below.

test-backup

If you are reading this, and for some reason focusing on this has not been your primary concern, here are a few things that might help

  • RichCopy – you can script out from the GUI and create a batch file. Make sure you have a log file too.
  • Instant File Initialization – makes the restore go fast. No zeroing out of database file.
  • Databases cannot be downgraded. Do not expect the database you restored in newer version server can be restored to lower version
  • More to add..

A little bit of code to help you get started

RichCopy Portion – What gets called

My CopyToNetworkLocation.bat looks like this:

"Drive:\LocationOfRichCopy\RichCopy 4.0\RichCopy64.exe"  "Drive:\BackupLocation\"  "\\NetworkLocation\Backups\ServerName\" /FSD /TSD /TSU /CCD /CLW /NE /P /V /QA /QP "Drive:\LocationOfRichCopy\RichCopyScripts\ServerName.log" /UE /US /UD /UC /UPC /UFC /UCS /USC /USS /UPR

SQL Job Configuration – The caller

EXEC xp_cmdshell 'Drive:\LocationOfRichCopy\RichCopyScripts\CopyToNetworkLocation.bat'

Remember, you have to enable xp_cmdshell to run this command. You can briefly enable, and disable it. Or you can use Windows Scheduled Task to schedule this job.

Trending SQL Server Job Duration by Day

As I am getting used to the new environment, I wanted to keep tabs on SQL Server Agent job and see how the database growth is impacting SQL Server Agent jobs in terms of duration (daily). I had stumbled upon a query to analyze it by hour and that would work in data warehouse situations where ETL is happening every hour. My objective was to analyze it by days, and have it dynamic (had to monkey around with PIVOT function and dynamic columns). Finally, bingo!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
 
DECLARE
@columns VARCHAR(2000)
, @query VARCHAR(8000)
 
 
SELECT @columns = COALESCE
(
@columns + ',[' + CONVERT(VARCHAR, run_date) + ']', + '[' + CONVERT(VARCHAR, run_date) + ']'
)
from msdb.dbo.sysjobhistory
group by run_date
order by run_date
SET @query = '
;WITH JobHistory AS
(
SELECT
b.name as JobName
, a.run_date as [RunDate]
,(a.run_duration / 10000 * 60 * 60 -- Hours
+ a.run_duration % 10000 / 100 * 60 -- Minutes
+ a.run_duration % 100 -- Seconds
) / 60.0 AS [DurationMinutes]
FROM
msdb.dbo.sysjobhistory a WITH(NOLOCK)
INNER JOIN msdb.dbo.sysjobs b WITH(NOLOCK)
ON
a.[job_id] = b.[job_id]
-- AND b.[name] = @JobName
AND step_id = 0
AND run_status = 1
)
SELECT *
FROM
JobHistory
PIVOT
(
SUM(DurationMinutes)
FOR [RunDate]
in
(' +
@columns
+
')
) as pvt
 
order by JobName'
exec(@query)

The original idea about trending by hour was acquired from:

http://timlaqua.com/2009/12/trending-sql-server-agent-job-duration-by-hour/

Executing SSIS Packages from SQL without SSIS

I wanted to automate collecting logs from some production servers that I manage and the best way to get remote data is using SSIS. However, we typically do not install SQL Server Integration Services (SSIS) on our boxes – and I do not have access to any server that has this feature installed (its a new job, things are going to change!). So package store is not there, nor can you schedule the SSIS Package easily. DTEXEC came to rescue:

You can simply execute package via the SQL Server agent job by doing the following. Please note that you will have to have xp_cmdshell enabled if you want to use T-SQL.

 
EXEC xp_cmdshell 'dtexec /f "C:\SSISProjects\DBA\DBA\Test.dtsx"'

SSRS 2008 R2 Permissions

I spent a couple of hours this morning trying to figure out the permission with one of our SSRS 2008 R2 servers. Here is how the problem started, and the steps I took to fix the issue. When I RDP to the server, I can open the Reporting Services “Site Settings” and view the reports that I have published. However, when I am on my local computer, there was a permission issue which said the following:

“does not have required permissions. Verify that sufficient permissions have been granted and Windows User Account Control (UAC) restrictions have been addressed”

I was wondering what the problem was because I was already added to the “System Administrator” role in the site setting. That did not resolve the problem. After looking at the permission in the ReportServer Database, I added myself to RSExecRole. That still did not fix the problem. Finally it came down to Adding myself to the root level Folder Permission.

SSRS 2008 Permission

Automation of Backup Restores

This blog post is a continuation of my earlier post. Since BACKUP VERIFYONLY does not ensure (guarantee) that the backup will fully restore, and I also wanted to offload the DBCC CHECKDB activities into a secondary server, I wrote a Automated Restore Process that will:
1) RESTORE A DATABASE
2) Keep History

Since my test system is pretty much equivalent to my production boxes, it will also give me a better idea on how much time it takes for a particular database to restore. Script as follows:

Table Creation Script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
USE [DBATasks]
GO
 
/****** Object:  Table [dbo].[Admin_BackupRestores]    Script Date: 7/19/2012 1:49:43 PM ******/
SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
SET ANSI_PADDING ON
GO
 
CREATE TABLE [dbo].[Admin_BackupRestores](
	[BackupRestoreID] [int] IDENTITY(1,1) NOT NULL,
	[ServerName] [varchar](50) NOT NULL,
	[DatabaseName] [varchar](50) NOT NULL,
	[LastCopied] [varchar](50) NOT NULL,
	[DateCreated] [datetime] NOT NULL,
	[LastRestored] [varchar](50) NOT NULL,
	[DateRestored] [datetime] NOT NULL
) ON [PRIMARY]
 
GO
 
SET ANSI_PADDING OFF
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_ServerName]  DEFAULT ('') FOR [ServerName]
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_DatabaseName]  DEFAULT ('') FOR [DatabaseName]
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_LastCopied]  DEFAULT ('') FOR [LastCopied]
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_DateCreated]  DEFAULT (getdate()) FOR [DateCreated]
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_Restored]  DEFAULT ('') FOR [LastRestored]
GO
 
ALTER TABLE [dbo].[Admin_BackupRestores] ADD  CONSTRAINT [DF_Admin_BackupRestore_DateRestored]  DEFAULT ('1900-01-01') FOR [DateRestored]
GO

Stored Proc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
USE [DBATasks]
GO
/****** Object:  StoredProcedure [dbo].[Admin_RestoreVerifyOnly]    Script Date: 7/19/2012 11:55:45 AM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
 
ALTER PROC [dbo].[Admin_AutomaticRestores]
(
	@ServerName VARCHAR(50)
	, @Path VARCHAR(200)
	, @LogicalDataFileName VARCHAR(50)
	, @LogicalLogFileName VARCHAR(50)
)
 
AS
SET NOCOUNT ON;
 
/***************************************************************************
Usage:
EXEC [dbo].[Admin_AutomaticRestores]
@ServerNAme = 'Test-DB1'
, @Path = 'K:\Backup\DB1'
, @LogicalDataFileName = 'DBATasks'
, @LogicalLogFileName = 'DBATasks_log'
***************************************************************************/
 
/***************************************************************************
Author: Subhash Pant
Date Created: 07/19/2012
Date Modified:
Purpose: This process automatically restores the database so that DBCC 
		 can be run against it.
***************************************************************************/
 
DECLARE
	@DATE DATETIME ,
    @fileName NVARCHAR(75) ,
    @dir NVARCHAR(255) ,
    @loadPath NVARCHAR(500) ,
    @cmd1 NVARCHAR(330) ,
    @maxDate DATETIME ,
    @id INT ,
    @dateRestored DATETIME ,
    @restorePath NVARCHAR(500),   
    @lastRestored VARCHAR(50) ,
    @errMsg VARCHAR(500) ,
    @errSeverity INT ,
    @errState INT
/*********************************************************
Please configure these values
**********************************************************/
 
SET @loadPath = @Path
SET @restorePath = @Path
--SET @loadPath = '"K:\Backup\DB1"'
--SET @restorePath = 'K:\Backup\DB1'
 
/*********************************************************
No modification needed beyond this point
**********************************************************/
SET @dir = 'dir ' + @loadPath
SET @dateRestored = '1900-01-01'
 
IF OBJECT_ID('tempdb.dbo.#tmpDir') IS NULL
CREATE TABLE #tmpDir
(
	id INT IDENTITY ,
	ret NVARCHAR(1000)
)
ELSE
    TRUNCATE TABLE #tmpDir
 
INSERT #tmpDir
EXEC master.dbo.xp_cmdshell @dir
 
 
 
/*********************************************************
Remove unwanted values
**********************************************************/
DELETE #tmpDir
WHERE NOT ret LIKE '%.bak%'
OR ret IS NULL
 
 
IF EXISTS ( SELECT *
FROM #tmpDir AS tmp
LEFT OUTER JOIN [dbo].[Admin_BackupRestores] AS lr 
	ON LTRIM(RTRIM(RIGHT(tmp.ret, CHARINDEX(' ', REVERSE(tmp.ret))))) = LastCopied
WHERE tmp.ret LIKE '%.bak%'
	AND LastCopied IS NULL )
 
 
BEGIN
    INSERT into [dbo].[Admin_BackupRestores](ServerName, DatabaseName, LastCopied, DateCreated)
    SELECT @ServerName,
			LEFT(LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))), PatIndex('%[0-9]%', LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))) + '1') - 1),
				LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))) AS LastCopied,
            CASE WHEN tmp.ret LIKE '% PM %'
				THEN LEFT(tmp.ret, CHARINDEX(' PM ', tmp.ret) + 3)
            WHEN tmp.ret LIKE '% AM %'
				THEN LEFT(tmp.ret, CHARINDEX(' AM ', tmp.ret) + 3)
            END AS date_created
    FROM #tmpDir AS tmp
    LEFT OUTER JOIN DBATasks.dbo.Admin_BackupRestores AS lr 
	ON LTRIM(RTRIM(RIGHT(tmp.ret, CHARINDEX(' ', REVERSE(tmp.ret))))) = LastCopied
    WHERE tmp.ret LIKE '%.bak%'
		AND LastCopied IS NULL
END
 
WHILE EXISTS (SELECT *
              FROM [dbo].[Admin_BackupRestores]
              WHERE [DateRestored] = '1900-01-01')
BEGIN
SELECT TOP 1
	@id = [BackupRestoreID] ,
	@lastRestored = [LastCopied] ,
	@cmd1 = 'RESTORE DATABASE ' + SUBSTRING(LastCopied, 1, CHARINDEX('_', [LastCopied])-1) + ' FROM DISK = '''
                    + @restorePath + '\' + LastCopied
                    + ''' WITH MOVE ''' + @LogicalDataFileName + ''' TO ''' + 'K:\TestRestores\DB1\' + DatabaseName +  CONVERT(VARCHAR(20), GETDATE(), 112)  
					+ '_Data.mdf ' + '''' + ', MOVE ''' + @LogicalLogFileName + '''' +
					' TO ''' + 'K:\TestRestores\DB1\' + DatabaseName + CONVERT(VARCHAR(20), GETDATE(), 112) + '_log.ldf'''
FROM [dbo].[Admin_BackupRestores]
WHERE [DateRestored] = '1900-01-01'
ORDER BY [DateCreated]
 
BEGIN TRY
	--print @cmd1
	EXEC sp_executesql @cmd1
 
END TRY
BEGIN CATCH
	SELECT 
		@errMsg = ERROR_MESSAGE() ,
		@errSeverity = ERROR_SEVERITY() ,
		@errState = ERROR_STATE()
 
	RAISERROR(@errMsg, @errSeverity, @errState) WITH NOWAIT
GOTO cleanup
END CATCH
	UPDATE [dbo].[Admin_BackupRestores]
	SET 
		[LastRestored] = @lastRestored,
		[DateRestored] = GETDATE()
	WHERE 
		[BackupRestoreID] = @id
END
cleanup:
SET NOCOUNT OFF;

What to run

1
2
3
4
5
EXEC [dbo].[Admin_AutomaticRestores]
@ServerNAme = 'Servername'
, @Path = 'K:\Backup\DB1'
, @LogicalDataFileName = 'DBATasks'
, @LogicalLogFileName = 'DBATasks_log'

Automating Restore Verify operations

I am a fan of automation. The more I can automate, the more I have time to do productive [sic] things. I wanted to automate a RESTORE VERIFYONLY process and document this in a table. Without further explanation, script is as follows. As always, please feel free to use/modify this and let me know if you come across some points I might have missed.

Table Creation Script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
USE [DBATasks]
GO
 
/****** Object:  Table [dbo].[Admin_BackupVerify]    Script Date: 7/19/2012 8:58:33 AM ******/
SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
SET ANSI_PADDING ON
GO
 
CREATE TABLE [dbo].[Admin_BackupVerify](
	[BackupRestoreID] [int] IDENTITY(1,1) NOT NULL,
	[ServerName] [varchar](50) NOT NULL,
	[DatabaseName] [varchar](50) NOT NULL,
	[LastCopied] [varchar](50) NOT NULL,
	[DateCreated] [datetime] NOT NULL,
	[LastRestorable] [varchar](50) NOT NULL,
	[DateRestored] [datetime] NOT NULL,
	[Verified] [bit] NOT NULL
) ON [PRIMARY]
 
GO
 
SET ANSI_PADDING OFF
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_ServerName]  DEFAULT ('') FOR [ServerName]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_DatabaseName]  DEFAULT ('') FOR [DatabaseName]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_LastCopied]  DEFAULT ('') FOR [LastCopied]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_DateCreated]  DEFAULT (getdate()) FOR [DateCreated]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_Restored]  DEFAULT ('') FOR [LastRestorable]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupRestores_DateRestored]  DEFAULT ('1900-01-01') FOR [DateRestored]
GO
 
ALTER TABLE [dbo].[Admin_BackupVerify] ADD  CONSTRAINT [DF_Admin_BackupVerify_Verified]  DEFAULT ((0)) FOR [Verified]
GO

Actual Stored Proc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
USE [DBATasks]
GO
 
/****** Object:  StoredProcedure [dbo].[Admin_RestoreVerifyOnly]    Script Date: 7/19/2012 8:20:20 AM ******/
SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
ALTER PROC [dbo].[Admin_RestoreVerifyOnly]
(
	@Path VARCHAR(200)
	, @ServerName VARCHAR(50)
)
 
AS
SET NOCOUNT ON;
/***************************************************************************
Author: Subhash Pant
Date Created: 07/17/2012
Date Modified:
Purpose: This process automatically finds the files
in the backup directory, and then runs the RESTORE VERIFYONLY
***************************************************************************/
 
DECLARE
	@DATE DATETIME ,
    @fileName NVARCHAR(75) ,
    @dir NVARCHAR(255) ,
    @loadPath NVARCHAR(500) ,
    @cmd1 NVARCHAR(330) ,
    @maxDate DATETIME ,
    @id INT ,
    @dateRestored DATETIME ,
    @restorePath NVARCHAR(500),   
    @lastRestored VARCHAR(50) ,
    @errMsg VARCHAR(500) ,
    @errSeverity INT ,
    @errState INT
/*********************************************************
Please configure these values
**********************************************************/
 
SET @loadPath = @Path
SET @restorePath = @Path
--SET @loadPath = '"K:\Backup\DB1"'
--SET @restorePath = 'K:\Backup\DB1'
 
/*********************************************************
No modification needed beyond this point
**********************************************************/
SET @dir = 'dir ' + @loadPath
SET @dateRestored = '1900-01-01'
 
IF OBJECT_ID('tempdb.dbo.#tmpDir') IS NULL
CREATE TABLE #tmpDir
(
	id INT IDENTITY ,
	ret NVARCHAR(1000)
)
ELSE
    TRUNCATE TABLE #tmpDir
 
INSERT #tmpDir
EXEC master.dbo.xp_cmdshell @dir
 
/*********************************************************
Remove unwanted values
**********************************************************/
DELETE #tmpDir
WHERE NOT ret LIKE '%.bak%'
OR ret IS NULL
 
 
IF EXISTS ( SELECT *
FROM #tmpDir AS tmp
LEFT OUTER JOIN DBATasks.dbo.Admin_BackupVerify AS lr 
	ON LTRIM(RTRIM(RIGHT(tmp.ret, CHARINDEX(' ', REVERSE(tmp.ret))))) = LastCopied
WHERE tmp.ret LIKE '%.bak%'
	AND LastCopied IS NULL )
 
BEGIN
    INSERT  DBATasks.dbo.Admin_BackupVerify(ServerName, DatabaseName, LastCopied, DateCreated)
    SELECT @ServerName,
			Left(LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))), PatIndex('%[0-9]%', LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))) + '1') - 1),
			LTRIM(RTRIM(RIGHT(tmp.ret,
				CHARINDEX(' ', REVERSE(tmp.ret))))) AS LastCopied,
            CASE WHEN tmp.ret LIKE '% PM %'
				THEN LEFT(tmp.ret, CHARINDEX(' PM ', tmp.ret) + 3)
            WHEN tmp.ret LIKE '% AM %'
				THEN LEFT(tmp.ret, CHARINDEX(' AM ', tmp.ret) + 3)
            END AS date_created
    FROM #tmpDir AS tmp
    LEFT OUTER JOIN DBATasks.dbo.Admin_BackupVerify AS lr 
	ON LTRIM(RTRIM(RIGHT(tmp.ret, CHARINDEX(' ', REVERSE(tmp.ret))))) = LastCopied
    WHERE tmp.ret LIKE '%.bak%'
		AND LastCopied IS NULL
END
 
WHILE EXISTS (SELECT *
              FROM DBATasks.dbo.Admin_BackupVerify
              WHERE [DateRestored] = '1900-01-01')
BEGIN
SELECT TOP 1
	@id = [BackupRestoreID] ,
	@lastRestored = [LastCopied] ,
	@cmd1 = 'RESTORE VERIFYONLY ' + 'from disk = '''
	+ @restorePath + '\' + [LastCopied] + ''''               
FROM DBATasks.dbo.Admin_BackupVerify
WHERE [DateRestored] = '1900-01-01'
ORDER BY [DateCreated]
 
BEGIN TRY
	--print @cmd1
	EXEC sp_executesql @cmd1
 
END TRY
BEGIN CATCH
	SELECT 
		@errMsg = ERROR_MESSAGE() ,
		@errSeverity = ERROR_SEVERITY() ,
		@errState = ERROR_STATE()
 
	RAISERROR(@errMsg, @errSeverity, @errState)
GOTO cleanup
END CATCH
	UPDATE [dbo].[Admin_BackupVerify] 
	SET 
		[LastRestorable] = @lastRestored,
		[DateRestored] = GETDATE(), 
		[Verified] = 1
 
	WHERE 
		[BackupRestoreID] = @id
END
cleanup:
SET NOCOUNT OFF;
GO

Auto memory management in Cluster

I was musing into a problem. We currently have our production server in a two node cluster that has 128GB of RAM each. 1st node contains 4 instances of SQL Server running, and the 2nd node contains 1 instance of SQL Server running. These are active/active clusters, meaning that in the event of failure, the other node takes on the processing, which ultimately degrades the performance. In the SQL Server running into 2nd node, we use some 84GB of memory and leave the remaining for failure events and the OS. In the 1st node though, we leave more than 90 GB for Node2 to be able to failover and not cause memory problems. I totally agree with the concept of high availability, but, for me, leaving so much amount of memory is just waste of resources.

We are adding additional node now, and the configuration will slightly change to active/active/passive clustering. I gathered my thoughts around the idea of writing a stored proc to manage the memory and do some math and decided to poke around with auto-memory management and startup stored procedure to achieve what I wanted to do so that when a fail over happens, and the SQL Server shifts to another node, the startup stored proc checks to see which node the server is on, and sets the right amount of memory.

Please do not use it in production without thoroughly verifying that this is not going to cause any problems. Startup stored procs has to be created on the master database (master is the first one which recovers after a restart), and cannot contain any input or output variables. There are other gotchas too.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
create proc dbo.dba_startup
as
SET NOCOUNT ON;
DECLARE @hostName varchar(100)
DECLARE @tbl TABLE
(
      hostName varchar(100)
)
 
INSERT INTO @tbl
exec xp_cmdshell 'hostname'
 
delete from @tbl
where hostName is null
 
 
 
 
select @hostname = hostname from @tbl
 
--SELECT @hostName = HOST_NAME()
 
if @hostName = 'Node1'
BEGIN
 
            exec sys.sp_configure 'show advanced options', 1;
            RECONFIGURE
            EXEC sys.sp_configure 'max server memory', 20000;
            RECONFIGURE
            EXEC sys.sp_configure 'min server memory', 20000;
            RECONFIGURE
            EXEC sys.sp_configure 'show advanced options', 0;
            RECONFIGURE
 
 
END
if @hostName = 'Node2'
begin
      exec sys.sp_configure 'show advanced options', 1;
      RECONFIGURE
      EXEC sys.sp_configure 'max server memory', 2147483647;
      RECONFIGURE
      EXEC sys.sp_configure 'min server memory', 0;
      RECONFIGURE
      EXEC sys.sp_configure 'show advanced options', 0;
      RECONFIGURE
END
 
 
SET NOCOUNT OFF;
 
 
--Configuring this as startup stored proc
exec dbo.sp_procoption N'dba_startup', 'startup', 'on'

Tracking Database Growth and Trends over Time

I was looking for a solution where database sizes and growth (and autogrowth) can be tracked over a time period so that I can estimate the size requirements, see when the growth happened, and trace it down to an event, job, or regular operations. Autogrowth can be tracked via querying default traces, but default traces are valid only for a certain period of time, and does not track the database sizes, free spaces, etc, for a long period of time. In order to accommodate all of that, I decided to look for around to see if anyone has a custom solution or at least partial solution of what I wanted to do. Found a post at MSSQLTips that had similar implementation. My changes were to create a table and collect information there, run the collector as a stored proc, and write a query to check the autogrowth between 2 consecutive days.

Found the author: Tim Ford (b, t). Original post was at: http://www.mssqltips.com/tip.asp?tip=1426

I made a few changes to the original script, and put together a little bit more detailed mechanism, which collects information, and allows you to query it in different ways to get the result you want.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
USE [Database]
GO
 
CREATE TABLE [dbo].[DatabaseSpace](
	[DatabaseSpaceID] [int] IDENTITY(1,1) NOT NULL,
	[ServerName] [varchar](100) NULL,
	[DatabaseName] [varchar](100) NULL,
	[FileSizeMB] [int] NULL,
	[LogicalFileName] [sysname] NOT NULL,
	[PhysicalFileName] [nvarchar](520) NULL,
	[Status] [sysname] NOT NULL,
	[Updateability] [sysname] NOT NULL,
	[RecoveryMode] [sysname] NOT NULL,
	[FreeSpaceMB] [int] NULL,
	[FreeSpacePct] [varchar](7) NULL,
	[FreeSpacePages] [int] NULL,
	[PollDate] [datetime] NULL,
 CONSTRAINT [PK_DatabaseSpace] PRIMARY KEY CLUSTERED 
(
	[DatabaseSpaceID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
--Stored proc creation. This stored prod runs everyday via SQL Server Agent Job
USE [Database]
GO
/****** Object:  StoredProcedure [dbo].[dba_DatabaseSpace]    Script Date: 05/16/2011 23:18:24 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER proc [dbo].[dba_DatabaseSpace]
as
SET NOCOUNT ON;
 
DECLARE @command VARCHAR(5000)  
 
SELECT @command = 'Use [' + '?' + '] SELECT  
@@servername as ServerName,  
' + '''' + '?' + '''' + ' AS DatabaseName,  
CAST(sysfiles.size/128.0 AS int) AS FileSize,  
sysfiles.name AS LogicalFileName, sysfiles.filename AS PhysicalFileName,  
CONVERT(sysname,DatabasePropertyEx(''?'',''Status'')) AS Status,  
CONVERT(sysname,DatabasePropertyEx(''?'',''Updateability'')) AS Updateability,  
CONVERT(sysname,DatabasePropertyEx(''?'',''Recovery'')) AS RecoveryMode,  
CAST(sysfiles.size/128.0 - CAST(FILEPROPERTY(sysfiles.name, ' + '''' +  
       'SpaceUsed' + '''' + ' ) AS int)/128.0 AS int) AS FreeSpaceMB,  
CAST(100 * (CAST (((sysfiles.size/128.0 -CAST(FILEPROPERTY(sysfiles.name,  
' + '''' + 'SpaceUsed' + '''' + ' ) AS int)/128.0)/(sysfiles.size/128.0))  
AS decimal(4,2))) AS varchar(8)) + ' + '''' + '%' + '''' + ' AS FreeSpacePct,  
GETDATE() as PollDate FROM dbo.sysfiles'  
INSERT INTO dbo.DatabaseSpace  
   (ServerName,  
   DatabaseName,  
   FileSizeMB,  
   LogicalFileName,  
   PhysicalFileName,  
   Status,  
   Updateability,  
   RecoveryMode,  
   FreeSpaceMB,  
   FreeSpacePct,  
   PollDate)  
EXEC sp_MSForEachDB @command  
SET NOCOUNT OFF;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
--Sample Query to get the database growth between two consecutive days.
--Author: Subhash Pant
SELECT  
        a.DatabaseName ,
        a.FileSizeMB AS previousSize,
        a.LogicalFileName ,
        a.PhysicalFileName ,
        a.FreeSpaceMB ,
        a.FreeSpacePct ,
        a.PollDate,
        c.FileSizeMB AS newSize,
        c.PollDate,
        isGrowth = CASE WHEN a.FileSizeMB < c.FileSizeMB THEN 1 ELSE 0 END
FROM dbo.DatabaseSpace a 
CROSS APPLY
(
SELECT TOP 1 *
FROM dbo.DatabaseSpace b
WHERE a.PollDate < b.PollDate
AND a.DatabaseName = b.DatabaseName
AND a.LogicalFileName = b.LogicalFileName
ORDER BY b.PollDate
) c

Found the author: Tim Ford (b, t). Original post was at: http://www.mssqltips.com/tip.asp?tip=1426